klepto2

This is a feature only a minority will ever need, but can increase the performance a lot in some cases: custom refraction / reflection posteffects which are meant as an overlay or backdrop What i mean exactly: Currenty each camera performs its own culling, even when the camera shares the same attributes as postion, rotation and range. My idea is to add a property to a camera where you can assign a main or parent camera. Now the main or parent camera performs the culling and the "child" cameras just consume the same visibility lists or data as the main camera. Even when pure gpu culling will be introduced this will increase performance as it can save some gpu processing which can be used for other things.

I think the scale of the dandelion model is wrong. I think it is of by a factor of 10.

Works great now, no crash, no memleak.

ok, maybe this helps to track the error down: i first thought the error is related to the density, but made some small tests and i found the following things: The reason for the error seems to be more complex than i thought: Create a terrain of any size to get the error it should be bigger than 512*512 otherwise the memory will raise but in handable sizes (7-10gb) Add a Layer: No error and instant visualisaton (no memory usage as well) change the density only to 1.0 or any value you like Click Fill Layer is displayed instant no mentionable increase in memory Now you can clear the layer, change density and fill it again, still instant and no memory usage With error and high memory usage Case: 1 make the same steps as above, but instead of clearing the layer after the first steps change the density to a lower value memory usage goes up to crazy amounts ( i have raised my virtual memory to 100gb for testing this) resize error occurs. With error and high memory usage Case: 2 This case is more like the "No Error" steps this time we will change the maxslope to lets say 20 before clicking Fill same result as in case 1

ok, tested it with the latest build. The good news: on smaller maps up to 1024 * 1024 i can't trigger the buffer resize error anymore. the virtual memory goes up to a crazy amount but after finishing the memory goes down again The bad news: on larger maps the virtual memory still raises beyond the maximum and triggers the buffer resize error the error now appears a lot later but still appears when the error message appears, the memory goes back to normal but the app still crashes thats far better than before as now the app doesn't "kill" the whole os anymore

I played around with some other terrain sizes, and even with a 512*512 sized terrain and i get the same results when switching the density from 2.0 to 1.0 and back. I can do this either by the spinner or by directly entering the density in the textfield, so in theory it should be only triggered once. With the 512 sized terrain, the virtual memory is always increased by 10gb as soon as I switch to "1.0" density. Switching back to a higher value doesn't lower the virtual memory, but then switching back to 1.0 it grows again by 10gb. Also loading a map ingame will lead to the same issue.

I would suggest to add some Infos and editable options to the terrain. As it is not in the scene-tree, it would be nice to have some of the entity related stuff baked into the terrain panel like: Tags Name components (i believe this was mentioned before) maybe collision stuff the appeareance stuff shadow, hidden, etc the color stuff is not needed also some infos about the terrain itself would be nice: eg: the terrain size, once created you need to remember the size lets say for importing heightmaps.

Maybe a small hint, when changing the density, the virtual memory goes up to more than 32gb and then the app crashes with the error. these are some virtual memory stats i measured: Starting the Editor: virtual memory ~ 1gb Creating the terrain (4k): virtual memory ~ 2.4gb Adding a layer with 3 variations: virtual memory ~7gb Decreasing the density to 1.0 : virtual memory stays at ~7gb Using Fill (no crash): virtual memory stays at ~7gb Increasing the density back to 2.0: virtual memory increases up to max available (here 32gb) and the app crashes.

with the same steps as i mentioned above.

Details: Terrain-Size is 4k used memory of the app is around 1.3 gb and there is more than 10 gb free mem availbale. the first fill with a density of 2.0 is working, lowering it to 1.0 leads to the state above. if i remember correctly it the allocation is 4gb not 256mb as you stated. [Addition:] It also seems to be dependent on how the terrain is layouted. In the above case it is just a fresh created flat terrain. With a heightmap applied and some height and slope settings the error comes earlier, sometimes already with the initial fill command. It also depends how many layers are already there. The number of variations seems to play no role.

I can reproduce it with the above steps everytime after a fresh restart. Sometimes it needs a bigger terrain or more dense vegetation.

Ok, yes it is only 16gb, but it also occurs on another machine where i have 32gb (just abit later). The bigger problem, is that even maps loaded ingame will crash with this error and it can't be workaround. This increaes the ram requirement to massive amounts for the engine and the published games. While i agree that 16 gb is not much, what will happen, if you later integrate streaming terrains or full planets? There must be a solution, maybe splitting the workload to slower chunks of memory? as it seems a ram problem, not agpu problem it should be solvable.

this might be rarely usecase, but when you run this code: #include "UltraEngine.h" #include "ComponentSystem.h" //#include "Steamworks/Steamworks.h" using namespace UltraEngine; SIZE_T PrintMemoryInfo() { auto myHandle = GetCurrentProcess(); //to fill in the process' memory usage details PROCESS_MEMORY_COUNTERS pmc; //return the usage (bytes), if I may if (GetProcessMemoryInfo(myHandle, &pmc, sizeof(pmc))) return(pmc.WorkingSetSize); else return 0; } shared_ptr<Entity> main_instance; vector<shared_ptr<Entity>> instances; void UpdateHook(shared_ptr<Object> source, shared_ptr<Object> extra) { auto world = extra->As<World>(); if (world != NULL) { instances.clear(); for (int i = 0; i < 100; i++) { instances.push_back(main_instance->Instantiate(world)); } } } int main(int argc, const char* argv[]) { #ifdef STEAM_API_H if (not Steamworks::Initialize()) { RuntimeError("Steamworks failed to initialize."); return 1; } #endif RegisterComponents(); auto cl = ParseCommandLine(argc, argv); //Load FreeImage plugin (optional) auto fiplugin = LoadPlugin("Plugins/FITextureLoader"); //Get the displays auto displays = GetDisplays(); //Create a window auto window = CreateWindow("Ultra Engine", 0, 0, 1280 * displays[0]->scale, 720 * displays[0]->scale, displays[0], WINDOW_CENTER | WINDOW_TITLEBAR); if (!AttachConsole(ATTACH_PARENT_PROCESS)) { if (AllocConsole()) { freopen("conin$", "r", stdin); freopen("conout$", "w", stdout); freopen("conout$", "w", stderr); } } else { auto consoleHandleOut = GetStdHandle(STD_OUTPUT_HANDLE); auto consoleHandleIn = GetStdHandle(STD_INPUT_HANDLE); auto consoleHandleErr = GetStdHandle(STD_ERROR_HANDLE); if (consoleHandleOut != INVALID_HANDLE_VALUE) { freopen("conout$", "w", stdout); setvbuf(stdout, NULL, _IONBF, 0); } if (consoleHandleIn != INVALID_HANDLE_VALUE) { freopen("conin$", "r", stdin); setvbuf(stdin, NULL, _IONBF, 0); } if (consoleHandleErr != INVALID_HANDLE_VALUE) { freopen("conout$", "w", stderr); setvbuf(stderr, NULL, _IONBF, 0); } } //Create a framebuffer auto framebuffer = CreateFramebuffer(window); //Create a world auto world = CreateWorld(); auto camera = CreateCamera(world); camera->SetClearColor(0.125); camera->SetFov(70); camera->Move(0, 2, -8); world->RecordStats(true); main_instance = CreatePlane(world, 10.0, 10.0, 256, 256); world->AddHook(HOOKID_UPDATE, UpdateHook, world); //Main loop while (window->Closed() == false and window->KeyDown(KEY_ESCAPE) == false) { world->Update(); world->Render(framebuffer); #ifdef STEAM_API_H Steamworks::Update(); #endif window->SetText("Instances: " + String(world->renderstats.instances) + " MEM: " + String(PrintMemoryInfo() / 1024) + " kb"); } #ifdef STEAM_API_H Steamworks::Shutdown(); #endif return 0; } You can observe the following: The memory rises a lot without going down. After some seconds, normally when the memory is above 300000kb the app spams Error: Invalid value While the memory has not so much impact, the Invalid value error leads to flickering (not so much in this sample, but in the original source i use the instances gets rendered totally random) and wrong rendering of the instances.

When using the vegetation system with large terrains it happens very fast that this error occurs: I can reproduce this with these simple steps: Create a terrain with at least 4096 * 4096 Add a foliage layer (u can use the crate.gltf) Fill the terrain with that layer (this should work) Set the density to 1.0 The editor works some time then the error message will appear It looks like a system memory problem, not a gpu problem. From time to time i get this error as well, when using eg: to many probes but with the vegetation system it is the safest way to reproduce it.

currently i can only use 2D textures as a rendertarget as there is no way to specify the current layer. It would be nice to have the ability to directly render to a specific cubeface or 3d layer.

It would be nice to have the same functionality to manually set uniform values as in Postprocess effects. This would highly improve the flexibility of the material system. Currently we need to either misuse the existing properties like glossiness etc. Which on the other hand removes the possibility to use these values in its original means or we need to provide a "settings" texture and read the values for each fragment which works, but is a much slower way.

I think this might resolve automatically if volumetric lights are implemented. In theory this would have a similar effect (when configured correctly) like colored probes.

just a hint, i get these artifacts only when i use the bloom posteffect.

As temporary fix you could try this: camera->SetBackfaceCulling(0); this solved it for me.

small correction: void main() { ivec2 coord = ivec2(gl_FragCoord.x, gl_FragCoord.y); outColor = texelFetch(ColorBuffer, coord,gl_SampleID); outColor.rgb = aces(outColor.rgb); } this also uses the sampleID, which might be better in this case.

Some bugs i found while playing with the terrain meshlayer system in the editor: Per-Vertex Alignment has wrong lighting: fix in Vertex.glsl line 148: the if statement needs to be removed Slope and Height settings are not working and not updating the already filled instances (update might be intendedm but feels unnatural) slope see image slope (completly undefined) height see image 2 (it seems like the min height is an offset from left and right, the maxheight acts a bit similar to the slope settings) Weight settings are not intentional When you add a new variation: the previous variations are divided by 2 and the new variation gets the remaining possible weight Samples: 1 Layer: weight = 100 2 Layers: L1 = 50, L2 = 50 (ok) 3 Layers: L1 = 25, L2 = 25, new L3 = 50 4 Layers: L1 = 13, L2 = 13, L3 = 25, new L4 = 50 ... this also leads to problems when editing the weight per variation as you can not exactly make everything lets say 25 percent as the other variations are also updated and readjusted. to solve this, I would suggest the following: for each variation, let the weight be a range of 0 to 100 (or whatever value someone would like to set up) for distribution, calculate the probability by summing up all variation weights and then calculate the real probability by dividing the specific variation weight by the total weight of all variations Sample 1: 4 Layers each have a probability of 100% total weight = 400 weight per layer = 100 / 400 = 0.25 Sample 2: 4 layers: L1 = 30 L2 = 100 L3 = 10 L4 = 70 total weight = 210 weight per Layer: L1 = 30 / 210 = 0.142 L2 = 100 / 210 = 0.476 L3 = 10 / 210 = 0.047 L4 = 70 / 210 = 0.33 Total: ~ 1.0 Image (Slope): Image (Height):

When using the skybox together with the angular fog you get artifacts in the corners like this: This can be fixed with something like this in the sky.frag: if (ExtractCameraFogSettings(CameraID, fogcolor, fogrange, fogangle)) { float slope = degrees(asin(normalize(texCoords).y)); if (slope < fogangle.y) { float l = clamp(1.0f - ((slope - fogangle.x) / (fogangle.y - fogangle.x)), 0.0f, 1.0f) * fogcolor.a; outColor[0].rgb = outColor[0].rgb * (1.0f - l) + fogcolor.rgb * l; } } just normalize the texcoords before calculating the slope:

Just an idea, you can use 2 cameras for this. 1. the normal realtime camera 2. a non realtime camera where you set the rendertarget to a new TextureBuffer. to the second camera you you attach the blur posteffect and assign the colorbuffer from the first (original) framebuffer via SetUnifrom("color", fb->GetColorAttachment(0)). whenever you open the inventory, you render the second camera once and the Texturebuffer of the second camera will have the processed blurred image. This could be extended to also work in realtime. as the 2nd camera can be using a complete empty world or a comeplere unique renderlayer, so no processing power is lost.

In the current builds (steam and standalone) the imposter generation is not working. If i use the "Tool->Generate imposter" the model is added correctly, but no imposter textures are generated. It seems that the required post effects (basecolor and world normals) for imposter generation are not provided.

With the latest terrain and selection changes the terrain editing has some small bugs: 1. no indicators are shown in the viewport. (the sculpt and paint ranges) 2. instead of the indicator, the multiselection box is drawn. But the editing in both cases works, just no overlay is shown. A side question: Now that i can choose the texture size. Are the brushes (or textures) resized internally to match the size? I ask because the default dirt layer has textures with 1k size, but when i create a terrain with 2k or 512 textures it just works but the texture size in the material is still 1k.