Ultra Engine testing

[klepto2]

the baseimageview is only one imageview, while this is enough for samplers, for images you need the imageview of each mipmap layer with all faces. and pass them separatly. 

[klepto2]

On 9/26/2022 at 7:55 AM, klepto2 said:

this is what i use to create the IBL map from the sky-cubemap:

#version 450

#extension GL_ARB_separate_shader_objects : enable
#extension GL_ARB_shading_language_420pack : enable

layout (local_size_x = 16, local_size_y = 16, local_size_z = 1) in;

layout (set = 0, binding = 0) uniform samplerCube envSkybox;
layout (set = 0, binding = 1, rgba32f) uniform imageCube envReflection;

layout (push_constant) uniform Contants
{
	vec4 data;
} constants;

const uint numSamples = 16;
#define PI 3.14159265359
float roughnessLevel = constants.data.x;

float RadicalInverse_VdC(uint bits)
{
	bits = (bits << 16u) | (bits >> 16u);
	bits = ((bits & 0x55555555u) << 1u) | ((bits & 0xAAAAAAAAu) >> 1u);
	bits = ((bits & 0x33333333u) << 2u) | ((bits & 0xCCCCCCCCu) >> 2u);
	bits = ((bits & 0x0F0F0F0Fu) << 4u) | ((bits & 0xF0F0F0F0u) >> 4u);
	bits = ((bits & 0x00FF00FFu) << 8u) | ((bits & 0xFF00FF00u) >> 8u);
	return float(bits) * 2.3283064365386963e-10; // / 0x100000000
}

// https://learnopengl.com/#!PBR/IBL/Specular-IBL
vec2 Hammersley(uint i, uint N)
{
	return vec2(float(i) / float(N), RadicalInverse_VdC(i));
}
// https://learnopengl.com/#!PBR/IBL/Specular-IBL
vec3 ImportanceSampleGGX(vec2 Xi, vec3 N, float roughness)
{
	float a = roughness*roughness;
	float phi = 2.0 * PI * Xi.x;
	float cosTheta = sqrt((1.0 - Xi.y) / (1.0 + (a*a - 1.0) * Xi.y));
	float sinTheta = sqrt(1.0 - cosTheta*cosTheta);
	// from spherical coordinates to cartesian coordinates
	vec3 H;
	H.x = cos(phi) * sinTheta;
	H.y = sin(phi) * sinTheta;
	H.z = cosTheta;
	// from tangent-space vector to world-space sample vector
	vec3 up = abs(N.z) < 0.999 ? vec3(0.0, 0.0, 1.0) : vec3(1.0, 0.0, 0.0);
	vec3 tangent = normalize(cross(up, N));
	vec3 bitangent = cross(N, tangent);
	vec3 sampleVec = tangent * H.x + bitangent * H.y + N * H.z;
	return normalize(sampleVec);
}

vec3 cubeCoordToWorld(ivec3 cubeCoord, vec2 cubemapSize)
{
    vec2 texCoord = vec2(cubeCoord.xy) / cubemapSize;
    texCoord = texCoord  * 2.0 - 1.0; // -1..1

    switch(cubeCoord.z)
    {
        case 0: return vec3(1.0, -texCoord.yx); // posx
        case 1: return vec3(-1.0, -texCoord.y, texCoord.x); //negx
        case 2: return vec3(texCoord.x, 1.0, texCoord.y); // posy
        case 3: return vec3(texCoord.x, -1.0, -texCoord.y); //negy
        case 4: return vec3(texCoord.x, -texCoord.y, 1.0); // posz
        case 5: return vec3(-texCoord.xy, -1.0); // negz
    }
    return vec3(0.0);
}
  
void main() 
{
	ivec3 cubeCoord = ivec3(gl_GlobalInvocationID);
	vec3 viewDir = normalize(cubeCoordToWorld(cubeCoord, vec2(imageSize(envReflection))));

	vec3 N = normalize(viewDir);
	vec3 R = N;
	vec3 V = N;

	float weight = 0;
	vec3 color = vec3(0);

	for (int samples = 0; samples < numSamples; samples++)
	{
		vec2 Xi = Hammersley(samples, numSamples);
		vec3 L = ImportanceSampleGGX(Xi, N, roughnessLevel); 

		float NdotL = dot(N, L);
		if (NdotL > 0)
		{
			color += texture(envSkybox, L).rgb;
			weight += NdotL;
		}
	}

	imageStore(envReflection,	
		cubeCoord,
		vec4(color / weight, 1.0));
}

on some discussions is mentioned, that you should always use computeshader when u write to cubemaps,  But from your code i don't think this is actually true, because in the discussions it is assumed, that you need to attach and render each face separatly in the fragment shader, but you just attach each face as a different target, so even if i prefer the compute way yours should be fast as well.

i use the above shader this to create the IBL texture, and this code below is the actual cpp code to setup the mipmapgeneration per roughness:

vector<EnvironmentSkyReflectionContants> reflShaders;
	for (int layer = 0; layer < reflectionTexture->CountMipmaps(); layer++)
	{
		shared_ptr<ComputeShader> refshader;
		int tx = 16;
		if (reflectionTexture->GetMipmapWidth(layer) < 16)
		{
			refshader = ComputeShader::Create("Shaders\\Environment\\env_reflection_gen_1.comp.spv");
			tx = 1;
		}
		else
		{
			refshader = ComputeShader::Create("Shaders\\Environment\\env_reflection_gen.comp.spv");
		}
		
		refshader->AddSampler(envSkyWithoutSun);
		refshader->AddTargetImage(reflectionTexture,layer);
		refshader->SetupPushConstant(sizeof(EnvironmentSkyReflectionContants));
		EnvironmentSkyReflectionContants data;
		data.reflectiondata.x = layer / reflectionTexture->CountMipmaps();
		data.reflectiondata.y = layer;
		
		refshader->BeginDispatch(world, Max(1,reflectionTexture->GetMipmapWidth(layer)) / tx, Max(1, reflectionTexture->GetMipmapHeight(layer)) / tx, 6, false, ComputeHook::RENDER, &data, sizeof(EnvironmentSkyReflectionContants));
		reflShaders.push_back(data);
	}

 

[Josh]

Okay...how do you use imageStore() with a cubemap? I see it takes an iVec3 for the coordinate. Is the third component the face number, between 0 and 5?

[klepto2]

exactly

[Josh]

Have you thought about creating your own imageviews that contain the exact data you want? The engine has a limited number of image slots and if I create different versions of each mipmap level, those numbers start getting very big. There's also an issue that another library or feature in my engine might need them to be slightly different again.

This is how the base image view is created:

	bool RenderTexture::CreateImageView()
	{
		viewInfo = {};
		viewInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO;
		viewInfo.image = vkimage;
		switch (type)
		{
			//case TEXTURE_1D:
			//	viewInfo.viewType = VK_IMAGE_VIEW_TYPE_1D;
			//	break;
		case TEXTURE_2D:
			viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D;
			break;
		case TEXTURE_3D:
			viewInfo.viewType = VK_IMAGE_VIEW_TYPE_3D;
			break;
		case TEXTURE_CUBE:
			viewInfo.viewType = VK_IMAGE_VIEW_TYPE_CUBE;
			if (isshadow) viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D_ARRAY;
			break;
		}
		viewInfo.format = imageInfo.format;
		switch (format)
		{
		case VK_FORMAT_D24_UNORM_S8_UINT:
		case VK_FORMAT_D32_SFLOAT:
		case VK_FORMAT_D32_SFLOAT_S8_UINT:
		case VK_FORMAT_D16_UNORM:
		case VK_FORMAT_D16_UNORM_S8_UINT:
			viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_DEPTH_BIT;
			break;
		default:
			viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
			break;
		}
		viewInfo.subresourceRange.baseMipLevel = 0;
		viewInfo.subresourceRange.levelCount = miplevels;
		viewInfo.subresourceRange.baseArrayLayer = 0;
		viewInfo.subresourceRange.layerCount = faces;
		viewInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY;
		viewInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY;
		viewInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY;
		viewInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY;

		VkAssert(vkCreateImageView(device->device, &viewInfo, NULL, &imageview));
		return true;
	}

 

[klepto2]

Maybe just an idea:

to hide the VkTexture things and maybe everything related to transfer data from your vulkan renderer to 3rd party code from the common user, you could add a new namespace: UltraEngine::Transfer

in this you could have something like: VkTexture GetTransferDataForTexture(shared_ptr<Texture> texture);

this could be added as pure functions or in a static class (which i would prefer) the benefit is that you don't need answer questions to everyone who looks into the texture class, but also everyone who needs access to it has a central point where to look at.

2 minutes ago, Josh said:

Have you thought about creating your own imageviews that contain the exact data you want? The engine has a limited number of image slots and if I create different versions of each mipmap level, those numbers start getting very big. There's also an issue that another library or feature in my engine might need them to be slightly different again.

This is how the base image view is created:

	bool RenderTexture::CreateImageView()
	{
		viewInfo = {};
		viewInfo.sType = VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO;
		viewInfo.image = vkimage;
		switch (type)
		{
			//case TEXTURE_1D:
			//	viewInfo.viewType = VK_IMAGE_VIEW_TYPE_1D;
			//	break;
		case TEXTURE_2D:
			viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D;
			break;
		case TEXTURE_3D:
			viewInfo.viewType = VK_IMAGE_VIEW_TYPE_3D;
			break;
		case TEXTURE_CUBE:
			viewInfo.viewType = VK_IMAGE_VIEW_TYPE_CUBE;
			if (isshadow) viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D_ARRAY;
			break;
		}
		viewInfo.format = imageInfo.format;
		switch (format)
		{
		case VK_FORMAT_D24_UNORM_S8_UINT:
		case VK_FORMAT_D32_SFLOAT:
		case VK_FORMAT_D32_SFLOAT_S8_UINT:
		case VK_FORMAT_D16_UNORM:
		case VK_FORMAT_D16_UNORM_S8_UINT:
			viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_DEPTH_BIT;
			break;
		default:
			viewInfo.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
			break;
		}
		viewInfo.subresourceRange.baseMipLevel = 0;
		viewInfo.subresourceRange.levelCount = miplevels;
		viewInfo.subresourceRange.baseArrayLayer = 0;
		viewInfo.subresourceRange.layerCount = faces;
		viewInfo.components.r = VK_COMPONENT_SWIZZLE_IDENTITY;
		viewInfo.components.g = VK_COMPONENT_SWIZZLE_IDENTITY;
		viewInfo.components.b = VK_COMPONENT_SWIZZLE_IDENTITY;
		viewInfo.components.a = VK_COMPONENT_SWIZZLE_IDENTITY;

		VkAssert(vkCreateImageView(device->device, &viewInfo, NULL, &imageview));
		return true;
	}

I don't know how this would break the image slots? I would guess the way your currently doing it will brake the image slots much faster

Lets take a cubemap with 1024*1024 and the assumed 6 layers: while your appoach currently generates 66 imageviews for this, the correct size would be 10. 

So adding the last 10 wouldn't be that hard

also the vulkan spec states that there is no limit for imageviews, though multiple people stated that theyx tested it and it breaks after around 520k but that should be more than enough ;).

[Josh]

In my shaders, the imageviews and samplers are all stored in one big array for each type of texture or for each image. This allows the engine to access all textures at all times, something the terrain and lighting features make use of. The engine and shaders allocate a fix size array, so there is an upper limit of how many textures can be loaded at once, so there is a limited number of slots.

If you are sending the image views to your own shader yourself, then none of that affects you.

Why not do something like this?:

class WhateverObjectIsStoredInExtraParameter : public Object
{
    std::map<shared_ptr<Texture>, std::vector<VkImageView> > imageviewsfortexture;
}

You could create the image views and associate them with that texture:

if (o->imageviewsfortexture[mytexture].empty())
{
  for (int m = 0; m < miplevels; ++m)
  {
    vkCreateImageView(&imageview);
    o->imageviewsfortexture[mytexture].push_back(imageview);
  }
}

 

[klepto2]

i just had a similar idea. This also is just needed for cubemaps if i understand you correctly as the other textures will behave the same way as before?

[Josh]

1 minute ago, klepto2 said:

This also is just needed for cubemaps if i understand you correctly as the other textures will behave the same way as before?

2D textures, yes, but I think volume textures are not getting imageviews for individual mip levels right now.

[klepto2]

That is currently no problem as this is not yet needed

[Josh]

You probably don't need to be too concerned with cleanup with something like this, but every time your hook is called the "extra" parameter gets bound to a list of objects that isn't cleared until that frame is finished rendering. So it should be safe to delete the imageviews or any other resources in your class destructor, if you wish to, and it won't delete objects that are still in use by the GPU.

[klepto2]

what would be nice, would be if you could at the imageInfo to the VkTexture, there should be everything included which is need for the view creation.

[klepto2]

maybe a stupid question, but maybe you can share the code you use for the mipmap Imageview? i can't get it to work correctly.

[klepto2]

nevermind, got it

viewInfo.subresourceRange.baseMipLevel = mipLevel;
	viewInfo.subresourceRange.levelCount = VK_REMAINING_MIP_LEVELS;
	viewInfo.subresourceRange.baseArrayLayer = 0;
	viewInfo.subresourceRange.layerCount = texture->CountFaces();

 

[klepto2]

small bug or change report:

the directional light is now much brighter than in the previous builds. (like factor 10 or so)

terrain is not affacted by light in any way.

[Josh]

The CreateProbe example in the documentation appears normal to me, and it uses a directional light.

One of the reasons you see a lot of little bugs every time I change shaders is because the entity data is packed as tightly into some very complicated structures, where I am trying to save every bit possible to minimize the data transferred.

[Josh]

1 hour ago, klepto2 said:

maybe a stupid question, but maybe you can share the code you use for the mipmap Imageview? i can't get it to work correctly.

In my code I just use a level count of one, otherwise I think it is the same as yours:

		if ((((flags & TEXTURE_STORAGE) != 0 or (flags & TEXTURE_BUFFER) != 0)))// and miplevels > 1)
		{
			if (type == TEXTURE_2D or type == TEXTURE_CUBE)
			{
				viewInfo.viewType = VK_IMAGE_VIEW_TYPE_2D;
				viewInfo.subresourceRange.levelCount = 1;
				viewInfo.subresourceRange.layerCount = 1;
				int mipcount = CountMipmaps();
				mipimageview.resize(mipcount * faces);
				for (int l = 0; l < faces; ++l)
				{
					viewInfo.subresourceRange.baseArrayLayer = l;
					for (int n = 0; n < mipcount; ++n)
					{
						viewInfo.subresourceRange.baseMipLevel = n;
						VkAssert(vkCreateImageView(device->device, &viewInfo, NULL, &mipimageview[l * mipcount + n]));
					}
				}
			}
		}

 

[Yue]

Me watching all this passively.

 

[klepto2]

yeah, it all works as expected, but somehow the brightness was lower in the previous builds. I have set the lightcolor vec3(2,2,2) for the same which is now: vec3(0.5,0.5,0.5). 

Anyway: probes are working awesome:

[Josh]

It looks like signed distance fields fits this need exactly. Initial testing indicates they can be generated on the CPU in not too much time, and saved as a DDS volume texture.

[klepto2]

Signed distance field would be a great addition, they are pure gold for a lot of advanced effects.

[klepto2]

By the way, it should be possible to create a full featured real-time sdf with the voxeldata of your vxrt implementation.

[klepto2]

Hi Josh,

I have a small and i think simple feature request. If my assumption from your previous posts are correct, then all textures created within the context of ultraengine are already available in all shaders. This is similar to my investigations while debugging in the app. eg: the renderthreadmanager holds a reference for every texture and its type. I also have seen, that the underlying rednertexture has an ID which seems to be the correct index for the texturearray (2d,3d, cube). 

If that is correct, it would be nice to have this id directly accessable in the Texture class. Then you could easily pass that id too a posteffect shader. With the AddPostEffectParameter you mentioned in this thread.

[Josh]

18 minutes ago, klepto2 said:

If that is correct, it would be nice to have this id directly accessable in the Texture class. Then you could easily pass that id too a posteffect shader. With the AddPostEffectParameter you mentioned in this thread.

Yeah, I was thinking the command would accept the texture object and then automatically pass the integer ID by converting the integer bits to a float (and then back in the shader).

I looked more into signed distance fields for reflections. The performance would be faster than my voxel raytracing, but the maximum sharpness of the reflections would be nowhere close to what screen-space reflection could provide, and it wouldn't work with particles. So I'm afraid signed distance fields are not a magic bullet we want to use everywhere. I think it's probably best to focus on fast SSR, and getting that to match the look of the environment probes as closely as possible. (This is identical to what id  Tech 7 does.)

[klepto2]

signed distance fields can be used for much more than that. you can use it for every effect which needs to know here it collides with an object. eg:  flow direction, or foam generation:

they even can be used to make fast SSAO and other effects.

So you should keep this in mind for later