DXT5nm and 3Dc DDS compression

[BrokenPillar]

We discussed this briefly on a different thread, but that thread got a little long so I thought I would start this topic in a fresh one.

 

This link is from nVidia's developer site and describes several methods to try and get better DDS compression results for normal maps:

http://developer.nvidia.com/object/real-time-normal-map-dxt-compression.html

 

I do not know much about writing shaders so I was hoping someone could write one for the DXT5nm and 3Dc methods described in this paper. I would then be happy to create some tests and record some video to compare the results so that people can decide whether it is worth using. for DXT5nm, I think you might be able to still store the spec in the red or green channels, but for the 3Dc you would have to save the spec somewhere else.

 

Also, is 3Dc possible through shader language alone, or would something have to be added to the engine?

[Niosop]

I'll add in a DXT5nm shader snippet (almost done w/ it, just need to test/debug). For 3Dc I think it would need engine support to work fluidly w/ everything else. We can add our own test support by manually creating the texture and loading the data, but it wouldn't be automatically loaded by the engine's texture loading stuff, so it's not optimal.

[Niosop]

K, it seems to be working. One question: I've heard that "if" statements in fragment shaders are expensive. Would it be worth detecting if a normal map is in DXT5nm format (R and B channels should equal 1) and automatically switching, or would the 2 if statements slow it down enough that it's worth having a different shader define (LW_DXT5NM) and require the user to make a diffuse_bumpmap_swizzle.frag shader?

 

If anyone wants to try it out, here's the updated portion of mesh.frag:

 

#ifdef LW_BUMPMAP

	#ifdef LW_TERRAINNORMALS
		//Use terrain normals
		terraincoord=vec2(vertexposition.x,-vertexposition.z) / terrainsize + 0.5;
		terrainresolution = terrainsize / terrainscale.x;
		terraincoord += 0.5 / terrainresolution;
		vec3 worldNormal = ((texture2D(texture14,terraincoord).xyz - 0.5) * 2.0).xyz;
		normal = normalize(gl_NormalMatrix*worldNormal);
	#else
		vec4 bumpcolor = texture2D(LW_BUMPMAP,texcoord);

	    #ifdef LW_DXT5NM
		normal = bumpcolor.xyz;
		normal.xy = bumpcolor.ag * 2.0 - 1.0;
		normal.z = sqrt(1.0 - normal.x * normal.x - normal.y * normal.y);
	    #else
	        normal = bumpcolor.xyz * 2.0 - 1.0;
	    #endif
	#endif

	#ifdef LW_DETAIL
		normal += texture2D(LW_DETAIL,texcoord * 4.0).xyz * 2.0 - 1.0;
	#endif
	normal.z /= bumpscale;
	normal = T * normal.x + B * normal.y + N * normal.z;
	normal = normalize(normal);
	#ifdef LW_SPECULAR
		shininess = bumpcolor.a*specular;//*fOcclusionShadow
	#endif
	#ifdef LW_SPECULARMAP
		shininess = texture2D(LW_SPECULARMAP,texcoord).x*specular;//*fOcclusionShadow
	#endif

#else
	normal=normalize(normal);
#endif

 

It's used something like this:

 

diffuse_bumpmap_swizzle.frag

 

#define LW_DIFFUSE texture0
#define LW_BUMPMAP texture1
#define LW_DXT5NM

Include "mesh.frag"

[Masterxilo]

AFAIK, "if" statements in shaders USED TO BE expensive on old cards, since the whole code still got executed and the results were just deleted afterwards.

[Niosop]

Cool then. Can get rid of all the defines and such and just detect if r and b values are 1, and if so assume it's a DXT5nm and do the swizzling. Will update it now and see how well it works out.

[Josh]

What's the purpose of encoding normals this way? Do you have some other data to store in the leftover channel? An extra square root operation is something I try to avoid.

[Niosop]

Just for resolution. I think it's 5658, so by using the two highest precision channels and calculating the Z then you can squeeze a little more precision out of it (8/6/calculated Z). Whether it actually makes a noticeable difference in either visual quality or FPS I'll leave it to BrokenPillar to do the comparisons and let us know what he comes up with.

 

We could stick specular and something else in the R and B channels if we wanted, there are two 5 bit channels left over.

 

Auto detecting the format in the shader isn't working out very well so I'll leave it as a define so that people who want to use it can.

[BrokenPillar]

Thanks, I'll try to get some tests running later tonight when I get some time to work with the code you posted.

 

What's the purpose of encoding normals this way?

 

It's all explained in great detail in the link in my first post, but the watered down version looks something like this:

 

 

Source image on the left, compressed image using standard DXT1 or DXT5 method on the right.

 

 

 

but with saving a normal map using the DXTnm setting and using the shader Niosop wrote you can get this:

Source image on the left, compressed image using DXT5nm method on the right.

 

 

and if we could get 3Dc working you could get this:

Source image on the left, compressed image using 3Dc method on the right.

 

The 3Dc method is the best quality, but it would take more work to implement and you wouldn't be able to store the specular with the normal (it is basically turning all 3 RGB channels into one block similar to the Alpha in a DTX5, so you are left with 2 channels that are the highest possible quality.)

 

We could stick specular and something else in the R and B channels if we wanted, there are two 5 bit channels left over.

 

Any chance you could add that as well? It would be a better comparison to be able to see a model done the regular way (with diffuse, bump and spec) next to a model with DXT5nm (also with diffuse, bump and spec).

 

Thanks

[Niosop]

Another option is A8L8 (GL_HILO8_NV). It's not compressed, but since it uses only two 8-bit channels then you can get the same quality as an uncompressed RGB8 texture in 2/3's the space. You'd still have to calculate the Z component which requires use of sqrt, but hey, 1/3 off w/ no quality loss.

 

I think 3Dc is the best option if it could be supported, as you get 4:1 compression w/ very little quality loss and it's hardware accelerated.

[Niosop]

I updated the code to use .ag instead of .ga. I originally used .ga because it made my test model look a little better, but I think it's because my model had some issues. Feel free to try both versions out and see what you come up w/.

 

Yet another page showing decent pictures of the different methods in use:

 

http://www.ozone3d.net/tutorials/bump_map_compression.php

[BrokenPillar]

Ok, I got the shader to work, and did an initial test with a significant difference in quality. Here is the first screenshot:

 

 

the two columns on the left are using the standard mesh_diffuse_bumpmap.frag shader. The two columns on the right are with Niosop's new shader.

 

I will clean up my files and post them all for other people that want to look for themselves. I will also try this out on a different mesh that you would actually see in a game (like a character or something) and post those results.

 

Niosop (or anyone else that understands shader language better than me), any chance you could tweak the shader so that you can store the spec in the now unused Red or Blue channels so I can compare using that as well?

[BrokenPillar]

Left image DXT5, Right image DXT5NM with new shader

 

Apparently a little improved normal map compression goes a long way.

 

One important note is that you have to save to the DXT5NM from the source image. You cannot just save out from an existing DXT5 map. This may be obvious, but I thought it was worth making clear.

[Niosop]

Nice examples. Could you take the guy model and add in a non-compressed version as well so we can see how close DXT5nm comes to the original? I'll add in spec to the red channel, although getting it there in your texture editing program might be a pain. I'll think a bit on the best way to handle that.

[Niosop]

Here's the small addition to use the red channel instead of the alpha for the specular value.

 

In mesh.frag replace

 

	#ifdef LW_SPECULAR
		shininess = bumpcolor.a*specular;
	#endif

 

with

 

	#ifdef LW_SPECULAR
		#ifdef LW_DTX5NM
			shininess = bumpcolor.r*specular;
		#else
			shininess = bumpcolor.a*specular;//*fOcclusionShadow
		#endif
	#endif

[BrokenPillar]

getting the spec into the red channel isn't too bad. You can just copy the red channel into the alph, and then you can copy your spec into the Red and save it as a DXT5 instead of DXT5NM. The DXT5NM export just automatically copies your red channel into the alpha and then duplicates the Green into the Red and Blue, but still compresses using the same method as DXT5.

 

I'll do a comparison against non-compressed a little later, and with the spec in the red channel.

[BrokenPillar]

I edited my character comparison post to include a comparison of uncompressed vs the DXT5NM. I can barely tell any difference between the uncompressed and DXT5NM versions, but a huge difference between the standard DXT5 method and DXT5NM.

 

Niosop, I notices that the derived z channel code you posted gives pure black spots on the model. Your code looks like this:

 

normal.z = sqrt(1.0 - normal.x * normal.x - normal.y * normal.y);

 

but in the website you linked to the two values are added rather than subtracted, like this:

 

z = sqrt( 1 - x*x + y*y );

 

If I change the "-" to a "+" in my mesh.frag code, I no longer get the black spots and so the overall effect is improved. Is there a reason you switched it to a subtract instead of an add?

 

Still need to make a version with the spec in the red channel...

[Niosop]

It's quite possible it was just a typo. May account for some other issues I was seeing, thanks for catching this. I'll check it on my system and update the code above.

[Niosop]

Hmm, http://code.google.com/p/nvidia-texture-tools/wiki/NormalMapCompression and http://developer.nvidia.com/object/real-time-normal-map-dxt-compression.html have:

 

z = sqrt(1 - x*x - y*y)

 

I also found an article that states:

 

The Green and Alpha channels are used because in the DXT format they are compressed using somewhat higher bit depths than the Red and Blue channels. Red and Blue have to be filled with the same solid color because most DXT compressors use an algorithm that compares differences between the three color channels. If you try to store some kind of texture in Red and/or Blue (specular power, height map, etc.) then the compressor will create more compression artifacts because it has to compare all three channels.

 

So might be worth using a separate spec map, or supplying a shader that uses the alpha value from the diffuse instead of the normal for specular values.

[BrokenPillar]

Ok, here are my final results with the complete shader in place:

 

Left Image is with a separate Spec Map, the image on the right is with the Spec in the Red channel of the DXT5NM.

The Spec map gets pretty butchered in the red channel when compared to when it is stored in the Alpha (as expected) but it doesn't seem to have a big impact on the final product.

 

Left Image is using an Uncompressed DDS, the image on the right is using the new DXT5NM method.

 

Left image is the standard shader method, the image on the right is the new DXT5NM method with the spec in the Red channel. Both have exactly the same texture memory footprint. All textures are 1024x 1024 resolution.

 

For anyone that is curious, this is what the new normal map looks like:

I couldn't attach a DDS, so you can't see the alpha, but this is what the RGB channels look like

 

I noticed a couple of errors in the example SwizzleTest.zip file I posted early, plus it doesn't have the Spec in Red Channel code integrated. I will clean it up and re-post it.

[Niosop]

Instead of relying on articles I decided to do the math:

 

x*x + y*y + z*z = 1

 

z*z = 1 - x*x - y*y

 

z = sqrt(1-x*x-y*y)

 

So the way I have it is technically correct. Could differing coordinate systems require a change in this Josh?

[BrokenPillar]

hmmm, just ran out of time to mess around with this today, but I will post some images of what I get when it is a subtract instead of an add...

[Niosop]

On the cubes, are they holes or bumps?

[Niosop]

Could you include an OBJ version of the cube? Something is a little funky w/ the direction of the lighting w/ normal maps in general. I want to test in a couple other engines and see if it's related to the model/normal map or the way LE handles them.

[Niosop]

K, tested, looks like it's an issue w/ LE. I don't really want to touch any other normal mapping shader code until this is resolved as I won't know which are problems with my code and which w/ LE's.

[BrokenPillar]

Here are the updated files to look at the two cubes with the bumps side by side. It now includes the ability to put specular in the Red Channel and the fixed normal map:

SwizzleTest.zip

It has a mesh for a cube for all of you other non-software types that understand working with geometry better than drawing a cube and painting it with a material in code.

 

You will need to unpack your shader.pak file first and then backup the standard mesh.frag shader somewhere else (this may be obvious, but just in case). You should then be able to copy the folders over and then just load the SwizzleTest map. The meshes are called DXT5Box.gmf and DXT5nmBox.gmf if you would rather just create your own scene and drop them in.

 

They are supposed to be bumps, not holes. I'm not sure what is going on with the internal normal map calculations that you are referring to Niosop, but these are the versions of the files that gave me the best results (with the + instead of the - for now). Here is a shot of what I get when it is subtracted instead:

 

 

Notice the pure black spots along his collar. They show up in different places around the model. To me it looks like it is calculating the Z to be too dark when the values are subtracted.