Setting vertex colors

[Canardia]

I want to color a mesh using vertex colors so I can have smooth color gradients. Is this possible with Ultra?

I want also to modify the vertex coordinates so I can make deformations, for example from physics impact.

[Josh]

Ultra does not support vertex colors because the size of the vertex structure does make a big difference in performance, especially with very detailed models, like polygonalized CAD models and 3D scans, and vertex colors are rarely used nowadays.

However, the vertex bone weights could be used to store colors, and a custom shader could read that information and modify the output color of the vertex shader.

You can modify the position of an existing vertex:
https://www.ultraengine.com/learn/Mesh_SetVertexPosition?lang=cpp

You cannot adds new vertices or indices to a mesh once it is initialized, due to the asynchronous rendering.

[Genebris]

What do you mean Josh? Vertex color is used constantly to define material blending. Including in AAA. Just recently I made a system to create holes in the ship sails by vertex color in real time. All sorts of shader effects might need vertex color. And what if you need that on a skinned mesh? You can't just discard it.

[Canardia]

If vertex coloring is not possible, I could perhaps draw a 2D OpenGL rectangle and render it as texture?

[Josh]

Well, meshes need to be explicitly set to be skinned if they use vertex weighting:
https://www.ultraengine.com/learn/Mesh_SetSkinned?lang=cpp

So it would probably be easy for me to modify the default vertex shader to display vertex colors stored in bone weights, if the mesh is not skinned.

1 hour ago, Genebris said:

And what if you need that on a skinned mesh?

There is also a second texcoord set, which is intended for use with GI lightmaps. An animated mesh could use that if it was really needed. Four bytes for RGBA can be packed into a single float value.

[Josh]

I need to think about this, because there is also the possibility of reusing the second texcoord set for the bone indices and weights. It would eliminate 8 bytes from the structure, which is worth considering.

[Genebris]

You typically need second set for detail mapping. then there's third for lightmaps and fourth is also available usually.

[Canardia]

I am trying to make a game which is mixed reality, it will be to one half based on newton dynamics which is deterministic and have ordinary textures.
The other half will have dream like experiences.
Therefore I need to have conventional graphics and also the possibility to shape that reality like in quantum mechanics.
 

[SpiderPig]

You might be able to store a vertex colour in a texture and write a custom shader to use the vertex index as some sort of lookup value.  I'm thinking out loud so don't know exactly how to code this.  Probably using gl_VertexID.

[Josh]

23 hours ago, Genebris said:

What do you mean Josh? Vertex color is used constantly to define material blending. Including in AAA. Just recently I made a system to create holes in the ship sails by vertex color in real time. All sorts of shader effects might need vertex color. And what if you need that on a skinned mesh? You can't just discard it.

The new forward renderer allows a lot of things that were not possible before. Although decals are not yet implemented, they will be able to modify the color (including alpha), normal, displacement, and other values. You could use decals to punch see-through holes in the sail.

I'm going to go ahead and implement vertex colors as an option when mesh skinning is not in use, but I am not 100% sure this is the final configuration of the vertex layout, and it is possible there could be some kind of small behavioral changes in the future. It could be possible to store colors one component of the second UV set, but only if they are 32-bit floats that can be unpacked to bytes. In earlier builds I was using half-floats for position and texcoords, but some GPUs do not support this. Vertex data is packed into the GPU-ready format on the main thread, because it usually has time to spare and this relieves the rendering thread from any extra overhead. However, the renderer doesn't find out whether the GPU support half-float vertex data until the first render, which occurs after meshes have already been packed. So changing that up dynamically based on the GPU capabilities is a bit tricky, and I am not sure if it will ever happen.

With half-floats we could eliminate 12 bytes, probably more, and that does make a difference when the vertex pipeline is the bottleneck.

[Genebris]

3 hours ago, Josh said:

You could use decals to punch see-through holes in the sail.

In that case I had to use some data within a vertex (naturally vertex color) because the sail is doing cloth simulation and every vertex can move individually. Just placing a decal would result in hole staying in place while the cloth moves around. The mesh was also skinned as required for cloth simulation. Using some UV channel could be enough if they are easily editable in runtime and if you aren't limited to just 2 of them.

[Josh]

That is an excellent point, and it applies to any skinned mesh. We would probably want decals to be applied based on the unskinned vertex position, so that the decals stretches with the mesh.

[Josh]

An update is available now. Make sure your sync your project to get the shader changes:
https://www.ultraengine.com/learn/Mesh_SetVertexColor?lang=cpp

[Canardia]

Awesome, it works! Thanks!

I have one little question though: are the corners of a CreateBox() mesh using 3 vertices instead of 1 for all 3 sides?
Anyway, I think this is good like it is, since I can now choose if I want to have each side differently colored:

[Josh]

Yes, this is necessary so that the each face has flat normals.

I noticed in my example the sphere seems to be using unique vertices for each face. This might be unnecessary but I'm not sure.

[Canardia]

My FPS display stopped working after the shader update, it shows nothing, not even the text: FPS: 

[Josh]

Okay, this is actually caused because I was storing per-vertex displacement in the fourth bone weight component....I need to rethink this.

[Josh]

Okay, it's fixed. This example shows both working at once:
 

#include "UltraEngine.h"

using namespace UltraEngine;

int main(int argc, const char* argv[])
{
    //Get the displays
    auto displays = GetDisplays();

    //Create a window
    auto window = CreateWindow("Ultra Engine", 0, 0, 1280, 720, displays[0], WINDOW_CENTER | WINDOW_TITLEBAR);

    //Create a world
    auto world = CreateWorld();

    //Create a framebuffer
    auto framebuffer = CreateFramebuffer(window);

    //Create a camera
    auto camera = CreateCamera(world);
    camera->SetClearColor(0.125);
    camera->SetPosition(0, 0, -4);

    //Create a light
    auto light = CreateBoxLight(world);
    light->SetRotation(45, 35, 0);
    light->SetRange(-10, 10);
    light->SetColor(2);

    //Create a model
    auto model = CreateSphere(world);
    for (int v = 0; v < model->lods[0]->meshes[0]->vertices.size(); ++v)
    {
        model->lods[0]->meshes[0]->SetVertexColor(v, Random(), Random(), Random());
    }

    auto font = LoadFont("Fonts/arial.ttf");
    auto sprite = CreateSprite(world, font, "Hello", 18);
    sprite->SetPosition(2, 0, 0);
    sprite->SetScale(1.0f / 18.0f);

    //Main loop
    while (window->Closed() == false and window->KeyDown(KEY_ESCAPE) == false)
    {
        model->Turn(0, 1, 0);
        world->Update();
        world->Render(framebuffer);
    }
    return 0;
}