print out shader values

[Rick]

Is there a way to print out values inside a shader to the output window? I'm curious to see some values in the fragment shader.

[klepto2]

Shader-Debugging is mostly done by mapping values or states to different output colors. You can try gdebugger (http://www.gremedy.com/) or NVidias NSight for Visual Studio. Otherwise the only way to get values is to write them to a rendertarget.

[Rick]

I'm trying to figure out what diffuse color values are coming from the editor to the Fragment shader. My assumption is that the editor is dividing whatever value is put in by 255 to get a value between 0 and 1. So I would think if I multiply by 255 in the shader I cold get the original value. ie. if I put 2 in the editor for red then the editor is doing 2 / 255 and then sending .0078 to the shader. Then if I multiply that by 255 I should end back up with 2. However this doesn't seem to be the case in the expected result I'm seeing (doing an atlas shader) vs when I just hardcode the red value of 2 in the shader itself. So was hoping to see these values in order to see what is going on. Unless anyone with shader experience has an idea on what is happening off the top of their head.

 

 

Maybe the precision getting sent to the shader can't handle such low values so maybe 1/255 is just getting sent as 0 and so multiplying it by anything obviously isn't going to do anything.

[klepto2]

It is aways a bad idea to compare float values directly lets take your exsample:

 

value set in the Editor = 2:

value sent to the shader = 2 / 255 = ~0.0078431372

 

calculation in the shader: 0.0078431372 * 255 = 1.9999 which is != 2

 

you may try something like:

 

vec3 intcolor = round(floatcolor * 255.0);

 

or you can make a comparison with a given tolerance.

[Rick]

There is no comparison being done. The value is just used for offsetting the giant texture so although 1.99 isn't 2 it's close enough for the texture offset.That's my understanding anyway.

 

// save off the color values as they really are used for texture offset
vec4 tileselect = ex_color;

vec4 outcolor = ex_color;

// set everything to 1 for color, which means we lose SetColor() functionality but I never use that anyway 
outcolor.x = 1.0;
outcolor.y = 1.0;
outcolor.z = 1.0;

// testing purposes
//tileselect.x = 0.0;
//tileselect.y = 1.0;
//tileselect.z = 0.5;

// the editor divides the diffuse color by 255 so we'll multiply it back out to allow us to enter valid row/col values
tileselect.x = tileselect.x * 255;
tileselect.y = tileselect.y * 255;
tileselect.z = 1 / (tileselect.z * 255);

vec4 color_specular = materialcolorspecular;

//Modulate blend with diffuse map
//outcolor *= texture(texture0,ex_texcoords0);
outcolor *= texture2D(texture0,vec2(ex_texcoords0.x*tileselect.z+tileselect.z*tileselect.x,ex_texcoords0.y*tileselect.z+tileselect.z*tileselect.y));

 

 

But again, entering a value of 0 or 1 in the editor doesn't change the visual of the model texture. If I comment the multiply by 255 code above and uncomment the texting code where I hardcode 0 or 1 then it works, but of course then it's not by model which is the requirement. This leads me to believe something about the x, y, & z values is not what I'm expecting. Below is the entire frag shader.

 

#version 400
#define BFN_ENABLED 1

//Uniforms
uniform sampler2D texture0;//diffuse map
uniform sampler2D texture1;//light map
uniform vec4 materialcolorspecular;
uniform vec4 lighting_ambient;
uniform samplerCube texture15;

//Inputs
in vec2 ex_texcoords0;
in vec4 ex_color;
in float ex_selectionstate;
in vec3 ex_VertexCameraPosition;
in vec3 ex_normal;
in vec3 ex_tangent;
in vec3 ex_binormal;

out vec4 fragData0;
out vec4 fragData1;
out vec4 fragData2;
out vec4 fragData3;

void main(void)
{
// save off the color values as they really are used for texture offset
vec4 tileselect = ex_color;

vec4 outcolor = ex_color;

// set everything to 1 for color, which means we lose SetColor() functionality but I never use that anyway 
outcolor.x = 1.0;
outcolor.y = 1.0;
outcolor.z = 1.0;

// testing purposes
//tileselect.x = 0.0;
//tileselect.y = 1.0;
//tileselect.z = 0.5;

// the editor divides the diffuse color by 255 so we'll multiply it back out to allow us to enter valid row/col values
tileselect.x = tileselect.x * 255;
tileselect.y = tileselect.y * 255;
tileselect.z = 1 / (tileselect.z * 255);

vec4 color_specular = materialcolorspecular;

//Modulate blend with diffuse map
//outcolor *= texture(texture0,ex_texcoords0);
outcolor *= texture2D(texture0,vec2(ex_texcoords0.x*tileselect.z+tileselect.z*tileselect.x,ex_texcoords0.y*tileselect.z+tileselect.z*tileselect.y));

//Normal map
vec3 normal = ex_normal;
//normal = texture(texture1,ex_texcoords0).xyz * 2.0 - 1.0;
normal = texture(texture1,vec2(ex_texcoords0.x*tileselect.z+tileselect.z*tileselect.x,ex_texcoords0.y*tileselect.z+tileselect.z*tileselect.y)).wyz * 2.0 - 1.0;
float ao = normal.z;
normal = ex_tangent*normal.x + ex_binormal*normal.y + ex_normal*normal.z;
normal=normalize(normal);

//Calculate lighting
vec4 lighting_diffuse = vec4(0);
vec4 lighting_specular = vec4(0);
float attenuation=1.0;
vec3 lightdir;
vec3 lightreflection;
int i;
float anglecos;
float diffspotangle; 
float denom;

//One equation, three light types
/*for (i=0; i<MAXLIGHTS; i++)
{
attenuation=1.0;

//Get light direction to this pixel
lightdir = normalize(ex_VertexCameraPosition - lightposition[i].xyz) * lightposition[i].w + lightdirection[i] * (1.0 - lightposition[i].w);

//Distance attenuation
attenuation = lightposition[i].w * max(0.0, 1.0 - distance(lightposition[i].xyz,ex_VertexCameraPosition) / lightrange[i]) + (1.0 - lightposition[i].w);

//Normal attenuation
attenuation *= max(0.0,dot(normal,-lightdir));

//Spot cone attenuation
denom = lightingconeanglescos[i].y-lightingconeanglescos[i].x; 
if (denom>-1.0)
{
anglecos = max(0.0,dot(lightdirection[i],lightdir));
attenuation *= 1.0 - clamp((lightingconeanglescos[i].y-anglecos)/denom,0.0,1.0);
}

lighting_diffuse += lightcolor[i] * attenuation;
}*/

//outcolor = (lighting_diffuse + lighting_ambient) * outcolor; 

//Blend with selection color if selected
fragData0 = outcolor;// * (1.0-ex_selectionstate) + ex_selectionstate * (outcolor*0.5+vec4(0.5,0.0,0.0,0.0));
//fragData0.xyz = normal*0.5+0.5;

#if BFN_ENABLED==1
//Best-fit normals
fragData1 = texture(texture15,normalize(vec3(normal.x,-normal.y,normal.z)));
fragData1.a = fragData0.a;
#else
//Low-res normals
fragData1 = vec4(normalize(normal)*0.5+0.5,fragData0.a);
#endif
fragData1.a = materialcolorspecular.r * 0.299 + materialcolorspecular.g * 0.587 + materialcolorspecular.b * 0.114;
int materialflags=1;
if (ex_selectionstate>0.0) materialflags += 2;
fragData2 = vec4(0.0,0.0,0.0,materialflags/255.0);
}

[shadmar]

You are converting floats to intergers.

 

        
tileselect.x = tileselect.x * 255;
tileselect.y = tileselect.y * 255;
tileselect.z = 1 / (tileselect.z * 255);

Change to :

 

        
tileselect.x = tileselect.x * 255.0;
tileselect.y = tileselect.y * 255.0;
tileselect.z = 1.0 / (tileselect.z * 255.0);

[Rick]

That didn't seem to make any difference when I made the change.

[shadmar]

Works on my 6x6 spritesheet, I can play thorugh all tiles using the material editor.

Can you show your texture?

[Rick]

So if I just isolate this to the z value I find it interesting. If the blue (z) value in the diffuse editor is set to 2 then in the shader

 

tileselect.z = 1.0 / (tileselect.z * 255.0);

 

produces the same result as if I did

 

tileselect.z = 1;

 

In that I see both textures in the first column in the atlas on the model

 

Somehow the formula is setting z to 1 instead of 0.5. I must be missing something basic here.

 

 

 

I can play thorugh all tiles using the material editor.

 

What do you mean material editor? I need to do this via the model color in the level editor (SetColor()). I thought that is how I did it before. If not is there a way to get the model color data into the shader?

 

Looking at my old code I'm calling SetColor() on the model itself not the material as the changes need to be done via model level.

 

In the level editor when I have the model selected under the Appearance tab in the Diffuse section is where I'm modifying my values for testing and the blue value does change the scale (not to what I'm expecting though) but red & green don't seem to be doing anything.

[shadmar]

What do you mean material editor? I need to do this via the model color (SetColor())

 

Well I just assumed since you were converting from the mat editors 0-255 colors, if not there is no point in going via 255.

 

Ex:

entity:SetColor(0.7,0.7,0.7,1.0) in lua/cpp code is going have the vaule in ex_color in the shader.

[Rick]

The way I originally used it a year ago was via entity:SetColor(). This time I need to use it via the editor. My assumption is that entity:SetColor() = Appearance Diffuse color via the editor when a model is selected.

 

So today since I'm trying to use the Appearance Diffuse color of a model I'm converting back from 255 since my assumption is the editor is taking whatever value is in those fields and dividing by 255 and that's what's getting sent to the shader.

 

Doing it via the level editor Appearance->Diffuse setting doesn't produce the correct results. Why would that be?

[shadmar]

Appearance->SetColor is multiplied by the materials color, so make sure material is all 255's (white), then it should work.

Brightness also needs to be 1.0

[Rick]

Ah OK, so that answers that mystery. They weren't all 255 in the material editor for some reason. Change that makes it work correctly when I attach a script to the model and call self.entity:SetColor().

 

However, from Appearance->Diffuse it still doesn't seem to work with values like (1, 1, 2).

[shadmar]

how does it look?

Works here, is Appearance brightness = 1.0 ?

[Rick]

So with this shader code:

 

tileselect.x = tileselect.x * 255.0;
tileselect.y = tileselect.y * 255.0;
tileselect.z = 1.0 / (tileselect.z * 255.0);

outcolor *= texture2D(texture0,vec2(ex_texcoords0.x*tileselect.z+tileselect.z*tileselect.x,ex_texcoords0.y*tileselect.z+tileselect.z*tileselect.y));

 

 

With the model Appearance->Diffuse of (1, 1, 2, 255) (alpha ignored anyway) and brightness set to 1 it looks like: