Jump to content
Ultra Engine Community
  • entries
    943
  • comments
    5,899
  • views
    923,763

Upsampling Textures with AI

Josh

Entry posted by Josh ·

4,634 views

 Share

I did a little experiment with FPS Creator Pack #75 by upsampling the images with Gigapixel, which uses deep learning to upsample images and infer details that don't appear in the original pixels. The AI neural network does a pretty impressive job, generating results that are look better than a simple sharpen filter: I doubled the size of the textures to 1024x1024. Then I generated normal maps from the high-res images using AMD's TGA to DOT3 tool, and saved the normal maps with BC5 DDS compression. The diffuse textures were saved with BC7 DDS compression. The images below are using a 4x magnification to demonstrate the difference.

bicubic.thumb.png.7232bf7d8017ff6d12efcf8466ae5dd3.png

ai.thumb.png.89d32653a172cfd0ba353b2324e7b620.png

As you can see, the image that is upsampled with deep learning looks normal and the resized image looks like there is butter on the lens! It's hard to believe the above images came from a 256x128 section of an image.

The workflow was pretty tedious, as I had to convert images to TGA, then to uncompressed or BC5 DDS, and then to BC7 in Visual Studio. Each BC7 texture took maybe 5-10 minutes to compress! So while this set represents the optimum quality for 2019 game tech, and the format for assets we want to use in LE5, the workflow has a lot of room for improvement.

You can download the full package here:

FPSCPack75TexturesHD.zip

  • Like 1
 Share

6 Comments

Recommended Comments

gamecreator

It still has a long ways to go but it's definitely an improvement of just resizing up.  I would encourage you to get a true, crisp, 2048x2048 (or any dimension large image), downsize it, then use the downsized version to scale up.  Then compare your results to the original 2048x2048 texture.

Link to comment
gamecreator

Curious how it would do with this grass texture:

grass512.png.eec816a2579fc310912ea2628477b843.png

This is the original (from here)

grass2048.thumb.png.65a2fa19a17436b6c97580f8df3c33e5.png

Of course, the next natural question is how it does with upsizing seamless textures.  In other words, does it introduce seams?

Link to comment
Josh
3 hours ago, gamecreator said:

Curious how it would do with this grass texture:

It did not do very well with that one. Left to right, low-res, AI upsample, high-res:
1817611748_grasscompare.thumb.png.7f2caca59a751d4a7d5171f66918896a.png

  • Like 1
Link to comment
gamecreator

I think this is the situation with everything it does.  It does an OK job estimating (and much better than just upscaling) but it can't make up for all missing details.  The arch and your original image were blurry too; obviously the original images would be much more crisp and detailed.  That said, this might still be helpful if you already have a 1024x1024 image and you want to make it 2048x2048, or from there to 4092.  I suspect the more detail it already has to work with, the better it'll do.

Link to comment
Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
Copyright © 2024 Ultra Software.
All rights reserved.
×
×
  • Create New...