[Advice Request] Cubemap capture and custom render pass shader groups

[Vladimir Sabantsev]

Started by looking into new shader families and inspecting shader output layout.
Default shader layout is defined in "Shaders/Base/Fragment.glsl"

layout(location = 0) out vec4 outColor[8];

Is it right that values from "Shaders/PBR/Fragment.glsl" outColor[attachmentindex] can be accessed if you attach a texture array to color attachment 0 and define appropriate RenderFlags uniform? Seems great, less collision opportunities with custom shader layouts!

The example code from Camera::SetRenderTarget gives an artifact, although the only moving object is a cam2, it feels as if all 3 objects in the scene are getting rotated and casting shadows to the red ball.
UPD: Ok, it's because cone and sphere object also have the same material as the white box, not a bug.

 

 

[Vladimir Sabantsev]

@Josh 
Here comes an example of using multiple color attachments in texture buffer.
It almost works out the box, which is super great! Please check the notes at the very end.

You can create a new shader family inside the editor:
Project > + > New Shader Family

Shader layout is declared inside Shaders/Base/Fragment.glsl

layout(location = 0) out vec4 outColor[8];

By default, PRB shader outputs additional information to the texture buffer color attachment with index in [1, 7] based on the 'RenderFlags' value
see: Shaders/PBR/Fragment.glsl
see: outColor[attachmentindex]

In our custom user-hook-shader we can output something by our own choice, e.g. the surface normal obtained from the PBR shader:

...
void UserHook(inout Surface surface, in Material material)
{
    outColor[1] = vec4(surface.normal.xyz, 1.0);
...

C++ code:

#include "Leadwerks.h"

using namespace Leadwerks;

int main(int argc, const char* argv[])
{
  auto displays    = GetDisplays();
  auto window      = CreateWindow("Ultra Engine", 0, 0, 1280, 720, displays[0], WINDOW_CENTER | WINDOW_TITLEBAR);
  auto world       = CreateWorld();
  auto framebuffer = CreateFramebuffer(window);
  auto cam1        = CreateCamera(world);

  cam1->SetClearColor(0.125);
  cam1->SetPosition(0, 0, -3);
  cam1->SetFov(70);

  //Create scenery
  auto light = CreateBoxLight(world);
  light->SetRange(-10, 10);
  light->SetRotation(15, 15, 0);
  light->SetColor(2);

  auto box1 = CreateBox(world);
  auto box2 = CreateBox(world);
  auto cone = CreateCone(world);
  auto sphr = CreateSphere(world);

  box1->SetColor(1, 1, 1);
  box2->SetColor(1, 1, 1);
  cone->SetColor(0, 0, 1);
  sphr->SetColor(1, 0, 0);

  box1->SetPosition( 0.00, +0.50, 0.00);
  box2->SetPosition( 0.00, -0.50, 0.00);
  cone->SetPosition(+1.25,  0.00, 0.00);
  sphr->SetPosition(-1.25,  0.00, 0.00);

  //Create render target with texture buffer which has 2 color attachment textures
  auto texbuffer = CreateTextureBuffer(256, 256, 2, true, 0);
  auto texbase   = CreateTexture(TextureType::TEXTURE_2D, 256, 256);
  auto texnorm   = CreateTexture(TextureType::TEXTURE_2D, 256, 256);
  texbuffer->SetColorAttachment(texbase, 0);
  texbuffer->SetColorAttachment(texnorm, 1);

  //Create camera with render target attached
  auto cam2 = CreateCamera(world);
  cam2->SetClearColor(1, 1, 1);
  cam2->SetRenderTarget(texbuffer);

  //Configure render layers to render
  //1. cone and shpere only to cam2
  //2. boxes only to cam1
  cam1->SetRenderLayers(0b01);
  box1->SetRenderLayers(0b01);
  box2->SetRenderLayers(0b01);

  cam2->SetRenderLayers(0b10);
  cone->SetRenderLayers(0b10);
  sphr->SetRenderLayers(0b10);

  //Create render target material with custom shader family which has 2 color outputs
  auto shfcust = LoadShaderFamily("Shaders/Example-TextureBuffer-SetColorAttachment.fam");
  auto mtlcust = CreateMaterial();
  mtlcust->SetShaderFamily(shfcust);

  //Create render target debug output fragment base color material
  auto mtlbase = CreateMaterial();
  mtlbase->SetTexture(texbase);

  //Create render target debug output fragment normal material
  auto mtlnorm = CreateMaterial();
  mtlnorm->SetTexture(texnorm);

  //Apply custom material to cone and sphere
  cone->SetMaterial(mtlcust);
  sphr->SetMaterial(mtlcust);

  //Apply render target debug output materials to boxes
  box1->SetMaterial(mtlbase);
  box2->SetMaterial(mtlnorm);

  //Main loop
  while (window->Closed() == false and window->KeyDown(KEY_ESCAPE) == false)
  {
    //Orient the texturebuffer camera
    cam2->SetPosition(0, 0, 0);
    cam2->Turn(0, 1, 0);
    cam2->Move(0, 0, -3);

    world->Update();
    world->Render(framebuffer);
  }
  return 0;
}

Result:

NOTES:

1. I had to add #include "../Base/Fragment.glsl" inside custom Fragment.glsl to have access to outColor
2. I had to add #include guard inside /Base/Fragment.glsl to avoid duplicate declarations with PBR/Fragment.glsl
3. Example didn't work with texbuffer->GetColorAttachment(1) without explicitly setting texbuffer->SetColorAttachment(texnorm, 1)
   May be a bug or some misunderstanding.
4. My initial though to add "layout(location = 1) out vec4 outCustom;" didn't work.
   Is it because the overlap with the default "layout(location = 0) out vec4 outColor[8];"?
5. Added #ifndef USER_HOOK section to avoid overwriting outColor[attachmentindex] in PBR/Fragment.glsl, but I'm not sure if it's actually needed.
6. I notice some kind of a stall on application close

[Josh]

Hmmmm, this is interesting. The number of color attachments will vary if the camera uses post-processing effects, or refraction, or SSR. But if you are not adding any post-processing effect to that camera I suppose it would be okay to just output to all those color attachments in one shader.