Ultra Engine testing

[klepto2]

Here you may see it more clearly:

and without the floor:

 

[klepto2]

I could fix it for me when i generate the diffuse and reflection map with inversed z-component.  Maybe this gives you the right hint.

[Josh]

Okay, I am doing a lot of work with this right now so I think I will find it soon.

[klepto2]

Here is a new Video, showing some improvements to the sky:

- using more exposure

- added a ground sphere to simulate the earth

- fixed the cubemap distortion (not completely,, but much better)

 

[SpiderPig]

Looks great, good job.  What's the FPS like?

[Josh]

With the 64-bit float build, you'll be able to render the earth, moon, other planets, and the sun all to-scale.

[klepto2]

31 minutes ago, SpiderPig said:

Looks great, good job.  What's the FPS like?

I get around 150 fps but this is because the rendering of the diffuse irradiance is very heavy and needs to be optimized or another faster way needs to be found. Through the nature of the precomputation, the sky and reflections are rendered in ~1ms (maybe faster). 

22 minutes ago, Josh said:

With the 64-bit float build, you'll be able to render the earth, moon, other planets, and the sun all to-scale.

yes, while this is possible, i will currently focus on 1 planet. multiple planets at once will be supported later.

Now i am starting withthe cloudsystem

[SpiderPig]

1 minute ago, klepto2 said:

I get around 150 fps but this is because the rendering of the diffuse irradiance is very heavy and needs to be optimized or another faster way needs to be found. Through the nature of the precomputation, the sky and reflections are rendered in ~1ms (maybe faster). 

150 is pretty good.

2 minutes ago, klepto2 said:

Now i am starting withthe cloudsystem

Volumetric?

 

@Josh What's your plans for sprites now?  I see they are created in 3D space rather than 2D as before...

[Josh]

5 minutes ago, SpiderPig said:

What's your plans for sprites now?  I see they are created in 3D space rather than 2D as before...

Right. There's no 2D primitives, just orthographic rendering. So if you want a rectangle, create a sprite and render with an orthographic camera. If you set the camera position like I showed you, the entity XY position will correspond to screen coordinates, with 0,0 at the bottom-left corner of the screen.

[SpiderPig]

Do I need to assign the sprite to the ortho camera?  I have one setup and creating a sprite result in this;

auto sprite = CreateSprite(world, font, "test", font_size);

EDIT : Or is this a "2D game only" thing?

[klepto2]

16 minutes ago, SpiderPig said:

150 is pretty good.

Volumetric?

@Josh What's your plans for sprites now?  I see they are created in 3D space rather than 2D as before...

For sure

This was done with leadwerks for a while: 

 

[Josh]

@SpiderPig There are some examples posted earlier in this thread.

[SpiderPig]

Sprite::SetRenderLayers()

Got it drawing but it draws like this;

[Josh]

I'm going to implement that fragment shader from Khronos for blurring / downsampling cubemaps, but it will work as a compute shader. Since real-time cubemap reflections are just not a good idea, I would rather focus on the quality of their reflections, and it's a lot easier to work with a compute shader than to set up a rendering pass. Environment probes should just get rendered once when the level loads.

@klepto2 What inputs would you need for the post-processing system? What would the hook look like?

void PostProcessHook(VkRenderer renderer, VkTexture depth, VkTexture color, VkTexture screennormals...?, shared_ptr<Object> extra)

 

[klepto2]

3 hours ago, Josh said:

I'm going to implement that fragment shader from Khronos for blurring / downsampling cubemaps, but it will work as a compute shader. Since real-time cubemap reflections are just not a good idea, I would rather focus on the quality of their reflections, and it's a lot easier to work with a compute shader than to set up a rendering pass. Environment probes should just get rendered once when the level loads.

@klepto2 What inputs would you need for the post-processing system? What would the hook look like?

void PostProcessHook(VkRenderer renderer, VkTexture depth, VkTexture color, VkTexture screennormals...?, shared_ptr<Object> extra)

It depends what verengerter is and what it data it holds. I assume color texture would be the later framebuffer / output? I don‘t know how the internal pix system works, but I to work flawlessly I needs to be similar to the way Leadwerks does it. Eg we need to load a shader (fragment or compute) and add not only textures but also variables ( push constants or uniform buffers). I must admit that I haven‘t fully thought about that yet. I know speed goes over flexibility but to get people into shader or effect programming it should be easy but flexible as possible.

[Josh]

Yeah, the system will allow custom properties in post-process effects on a per-camera basis. I have it worked out, I just haven't finalized it yet because of time.

I am wondering what additional features someone would implement that can't already be done in the post-effects system?

[klepto2]

eg: a cheap fog using real atmospheric lookups. Currently, this would need (and is probably the better way) a modification of the PBR shaders and additional textures and parameters to the material system. There are tons of pfx which may not only rely on down scaled 2d textures but also on custom 3d textures etc.. I haven't tried the pfx system yet that much, so i can't really say how flexible this will be. To be honest i think we can't compare Leadwerks and Ultraengine on this topic. While it was easy to render a sky as a posteffect (or water like shadmar did) as it would break the PBR rendering. 

But one sample comes into my mind: underwater fx. if the water/ocean has a flexible height we need to pass the ocean heightfield and the parameters to the underwater posteffect to mask out the areas where the camera is partially above the water. 

[Josh]

3 minutes ago, klepto2 said:

But one sample comes into my mind: underwater fx. if the water/ocean has a flexible height we need to pass the ocean heightfield and the parameters to the underwater posteffect to mask out the areas where the camera is partially above the water. 

It will work like this:

camera->SetPostEffectParameter(0, "waterheightmap", texture);
camera->SetPostEffectParameter(0, "waterheight", 10.0f);

 

[klepto2]

that should work

[Josh]

The JSON file will contain the names and types, which the engine will map to integer offsets in a 4x4 matrix, and then the shader will be responsible for evaluating that like this:

mat4 params = ExtractCameraPostEffectsParameters(CameraID);
float waterheight = params[0][0];
int texID = floatBitsToInt(params[0][1]);

 

[klepto2]

first take on volumetric clouds (converted from my LE-Implementation)

[Josh]

I'm going through some render-to-cubemap stuff and I finally realized what you were saying about the whole-texture-imageview. I didn't realize an imageview can include multiple miplevels and layers! How confusing.

Do you need the imageview that contains all layers and mipmaps, or is it okay to return an array of imageviews that each specify one miplevel and one cubemap face (layer)?

[Josh]

Here you can see rendering to six cubemap faces at once for images processing, using solid colors to test output. Next step is to pass the correct parameters to the Khronos shader, and then I can get nice filtering of the probe cubemap:

[klepto2]

From my experience, you only need the miplevels. (At least i just needed the layercount) the layer for the cubemap are accessed by the z component. In the sky rendering I always render to the full cubemap at once. Well for the clouds I currently try to implement temporal reprojection, this allows rendering of 1/16th of the whole cubemap in one frame without a noticeable lag or noise. ( the clouds in the above screenshot let the framerate drop from hundreds to just 20 fps so there is need for optimization)

[Josh]

Skies are already a soft gradient, so filtering might not make as big a difference there, but for environment probes it removes sharp edges that simply downsampling each face won't eliminate. It's this old problem:

Downsampled:

Filtered:

The filter shader is pretty intensive and I don't know if running it in real-time or even intermittently is a good idea. By default it uses 1024 samples per pixel. I tried setting the sample count to 16 and that seems okay:

This is my code:

	void RenderContext::BlurCubemap(shared_ptr<RenderTexture> src, shared_ptr<RenderTexture> dst)
	{
		commandbuffers[currentFrame]->BindResource(src);
		commandbuffers[currentFrame]->BindResource(dst);

		auto engine = UltraCore::GameEngine::Get();
		auto shader = engine->renderingthreadmanager->iblfiltershader;;
		if (shader == NULL) return;
		commandbuffers[currentFrame]->BindResource(shader);

		FilterParameters pushconstants = {};
      
		// 0: Lambertian (diffuse)
		// 1: GGX (specular)
		// 2: Charlie (sheen)
		pushconstants.distribution = 1;
		pushconstants.lodBias = 0;
		pushconstants.sampleCount = 1024;
		pushconstants.textureID = src->id;

		VkRect2D scissor = {};
		VkViewport viewport = {};
		viewport.maxDepth = 1.0f;
		VkImageCopy region = {};

		RenderPipelineSettings rendersettings = {};
		rendersettings.facecount = 1;// only one vertex output
		rendersettings.colorattachments = 6;// six fragment outputs
		for (int n = 0; n < 6; ++n)
		{
			rendersettings.colorformat[n] = dst->format;
		}

		VkRenderingInfoKHR renderinfo = {};
		renderinfo.sType = VK_STRUCTURE_TYPE_RENDERING_INFO_KHR;
		renderinfo.layerCount = 1;
		renderinfo.colorAttachmentCount = 6;
		
		int w = dst->size.x;

		//Incoming memory barrier
		VkImageMemoryBarrier barrier = {};
		barrier.sType = VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER;
		barrier.oldLayout = VK_IMAGE_LAYOUT_GENERAL;
		barrier.newLayout = VK_IMAGE_LAYOUT_GENERAL;
		barrier.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT;
		barrier.dstAccessMask = VK_ACCESS_SHADER_READ_BIT,
		barrier.subresourceRange.aspectMask = VK_IMAGE_ASPECT_COLOR_BIT;
		barrier.subresourceRange.layerCount = 6;
		barrier.subresourceRange.baseMipLevel = 0;
		barrier.subresourceRange.levelCount = src->CountMipmaps();
		VkPipelineStageFlags srcstagemask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
		VkPipelineStageFlags dststagemask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
		barrier.image = src->vkimage;
		vkCmdPipelineBarrier(commandbuffers[currentFrame]->commandbuffer, srcstagemask, dststagemask, 0, 0, NULL, 0, NULL, 1, &barrier);

		auto layout = VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL;
		commandbuffers[currentFrame]->TransitionImageLayout(src, layout, VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT, VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT);

		int mipcount = dst->CountMipmaps();
		for (int m = 0; m < mipcount; ++m)
		{
			pushconstants.currentMipLevel = m;
			pushconstants.width = dst->size.x;
			pushconstants.roughness = float(m) / float(mipcount - 1);

			VkRenderingAttachmentInfo colorattachment = {};
			colorattachment.sType = VK_STRUCTURE_TYPE_RENDERING_ATTACHMENT_INFO;
			colorattachment.imageLayout = VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL;
			colorattachment.loadOp = VK_ATTACHMENT_LOAD_OP_DONT_CARE;

			std::array<VkRenderingAttachmentInfo, 6> colorattachments;
			for (int n = 0; n < 6; ++n)
			{
				colorattachment.imageView = dst->mipimageview[n * mipcount + m];
				colorattachments[n] = colorattachment;
			}

			viewport.width = w;
			viewport.height = w;
			scissor.extent.width = w;
			scissor.extent.height = w;
			renderinfo.renderArea.extent.width = w;
			renderinfo.renderArea.extent.height = w;
			renderinfo.pColorAttachments = colorattachments.data();

			auto pipeline = shader->GetPipeline(rendersettings);
			Assert(pipeline);
			commandbuffers[currentFrame]->BindResource(pipeline);
			
			vkCmdBeginRendering(commandbuffers[currentFrame]->commandbuffer, &renderinfo);
			vkCmdPushConstants(commandbuffers[currentFrame]->commandbuffer, device->pipelineLayout, VK_SHADER_STAGE_ALL, 0, sizeof(pushconstants), &pushconstants);
			vkCmdBindPipeline(commandbuffers[currentFrame]->commandbuffer, VK_PIPELINE_BIND_POINT_GRAPHICS, pipeline->pipeline);
			vkCmdSetViewport(commandbuffers[currentFrame]->commandbuffer, 0, 1, &viewport);
			vkCmdSetScissor(commandbuffers[currentFrame]->commandbuffer, 0, 1, &scissor);
			vkCmdSetDepthBias(commandbuffers[currentFrame]->commandbuffer, 0.0f, 0.0f, 0.0f);
          
			//Vertex positions and texcoords are generated in the vertex shader from the index, so this really just needs six indices and the vertices they point to don't matter
			vkCmdDrawIndexed(commandbuffers[currentFrame]->commandbuffer, engine->renderingthreadmanager->fullscreenmesh->indiceCount, 1, engine->renderingthreadmanager->fullscreenmesh->indiceoffset, engine->renderingthreadmanager->fullscreenmesh->vertexoffsethandle->GetID(), 0);
          
			vkCmdEndRendering(commandbuffers[currentFrame]->commandbuffer);

			w /= 2;
		}

		barrier.image = dst->vkimage;
		barrier.subresourceRange.levelCount = dst->CountMipmaps();
		dststagemask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
		vkCmdPipelineBarrier(commandbuffers[currentFrame]->commandbuffer, srcstagemask, dststagemask, 0, 0, NULL, 0, NULL, 1, &barrier);
	}

And here is the shader. PostEffect.vert.spv can be used for the vertex shader module with no changes:

IBLFilter.zip