Memory Issues

[xtom]

When I open my map the windows task manager says leadwerks editor is using 1,269,748 K - am I right in saying this is over 1GB? Anyway I was getting error messages a lot today and I think it might be because of memory usage on the map so I don't think I will be able to add anymore content to my map without causing further editor crashes. I got a bank resize error, cant set clipmap and shadow map texture errors It's a 1024 terrain map with about 18 buildings. Is there anyway to make the editor use less memory? When I run the game in debug mode it says it's only using about 375MB vram.

[Admin]

What GPU do you have? The integrated graphics are somewhat more limited.

 

It's hard to assess your problem without seeing your project.

[xtom]

AMD A10 5800K APU (Integrated HD7660D) 8GB System Ram with 1GB dedicated as video ram. The project folder is 2GB+ but I'm probably only using 10% of the assets so I'll have to try and copy the project and trim it down later.

[Ronald Buanno]

Might I recommend linux? Lol jk, But trimming down your assets might help a bit. However If you have a gajillion assets in your scene that is understandable. How are your buildings constructed also? Anyway you could post pictures? or even better your .map?

 

More information makes diagnosis and troubleshooting more accurate

[xtom]

My buildings are all csg but with very few models in them at the moment. I don't have a massive amount in there to be honest so if I have to trim down on assets then there is no hope of me continuing this game. I asked Josh to have a look at it, I'm hoping it's something dodgy in the map so i can fix it and continue.

 

I just tried to take a screenshot but the editor crashed again as I zoomed out a view..

 

[martyj]

It's not a memory issue. Your task manager reports memory used in RAM, not video ram.(Although in your case it's the same thing, but not as far as the OS is concerned). 1 GB of ram is a ton of memory for your game to be using.

 

If I had to guess, I would say the issue could be a driver issue. I know that AMD drivers are a pain to work with as far as development goes.

[xtom]

1 GB of ram is not my fault I don't think.

 

281k Start leadwerks new blank map

363k add 1 csg house prefab (all one 512x512 texture)

397k (+34K) copy house

442k (+45K) copy house

490K (+48K) copy house

524K (+34K) Copy house

 

Blank map - 20 Houses (all 1 512x512 texture) - 1,118,988K

 

Now if I delete 19 of those houses and leave 1 the editor is still using - 1,009,808K (sorry 570K missed a few)

 

This is over a GB for just 20 simple houses and nothing else, no lights, just the one texture. The crashes might not be because of memory but I think it makes them more likely to happen the more that's in the map.

 

[YouGroove]

Could you post your prefab if made out of CSG only, i would like to test it.

I got also lot of model houses (no BSP) in a test scene , and i didn't ecountered memory growing or problems.

[xtom]

I've attached the prefab - I just added the blue developer texture to the whole thing so there wouldn't be missing textures.

houseprefab.zip

[xtom]

I just came across another post and it seems instanced model buildings perform better over csg buildings memory wise. Noooooo. So I think csg is only suited for indoor quake style maps then. That sucks because making maps and buildings in the editor is so easy and so much fun.

[shadmar]

You can obj export this, and bring it back as model, and instance it.

[xtom]

Ok I just tried exporting as .obj and then converted to .fbx and then leadwerks made a .mdl - but the mdl comes in without any textures. I guess this means I would need to re-texture it all over again in blender or something. Blender and texturing aghh not looking forward to that.

 

I tried placing over 120 of these in the scene though and there is barely any hit on memory compared to the csg version. Maybe a quick way to convert from csg to a textured mdl would be a great thing to have.

 

20 CSG houses1.2GB memory

 

120 mdl houses only 360,000K !

[shadmar]

I tested your house, noticed the editor getting sluggish very fast. That house is made from hundreds of csg items.

An export with the uvmap would be great.

[YouGroove]

I tried placing over 120 of these in the scene though and there is barely any hit on memory compared to the csg version. Maybe a quick way to convert from csg to a textured mdl would be a great thing to have.

Yes for outdoor games the way to go is models.

 

LE3 couldn't load your house when i drag and drop it in the scene.

I noticed you have very detailled objects in your house also, this is bad practice.

You would need to display group of big objects only when the player is near the house and display small detail objects only when the player is inside.

Also windows and doors on backfaces of the house when not visible in game, they could be hidden by script. Like hidding a group of doors and windows per side of the house.

[xtom]

That house is made from hundreds of csg items.

 

It's not made of hundreds of csg items, I guesstimate less than 200 anyway. But I could be wrong.

 

I noticed you have very detailled objects in your house also, this is bad practice.

You would need to display group of big objects only when the player is near the house and display small detail objects only when the player is inside.

 

I wanted to do set interior stuff to "Near" but apparently you can't do this with csg either. The detailed objects in the wireframe screenshot are models I forgot to remove, but I removed them and it made very little difference. The real memory killer is the simple csg and not the detailed/instanced models.

 

Also windows and doors on backfaces of the house when not visible in game, they could be hidden by script. Like hidding a group of doors and windows per side of the house.

 

Really I think this is something the engine itself should be handling. But suffice to say if I stay away from lots of csg buildings I probably won't need to worry about this.

[Josh]

The problem is the number of entities. This is sort of an extreme case, but you've convinced me to add a "convert to model" or "export MDL" option.

 

 

My preferred style of mapping is to model the doorframes, window frames, stairs, and other little pieces and use CSG for walls, floors, roofs, etc., but I understand some people want to work exclusively with the CSG modeling. It's just that you are demonstrating the worst possible scenario...highly detailed CSG that are repeatedly created 20 times.

[xtom]

Convert/export model sounds good if it keeps me in the editor more and helps streamline the process. At the moment my head is bit scrambled with rethinking new approaches. I think I too have run out of memory for the day

[Olby]

I can see this becoming a major problem once carving is finally implemented. Cut a cylinder through box and you instantly got a huge number of small objects. But 3DWS and even CShop can handle it easily so theoretically there should be a way to do this in LE without collapsing the CSG into models.

[Josh]

One thing we run into a lot with Leadwerks 3 is people come up against limitations they don't realize existed in old products, because the old products never really got used to the same extent as the new one. I suspect that may be the case here.

[YouGroove]

I can see this becoming a major problem once carving is finally implemented. Cut a cylinder through box and you instantly got a huge number of small objects

Games like HL2 are indoor games use really few and simple BSP as possible, all details are models, this is to keep performance

 

 

Also BSP incovenients :

-Produces double vertexs and inner faces compared to traditionnal modeling

-Can't use LOD

[Olby]

Guys I know for a fact that 3DWS could easily handle extremely large levels. So did Worldcraft/Hammer. Tons of detailed BSP maps and they all work, the technology has been around for nearly 2 decades. Don't tell me it can't be optimised. Simply nobody's interested that's a fact.

 

@YG - LOD for BSP? Are you kidding? How are you going to reduce a box with 8 vertices

 

 

 

 

[Josh]

I would wager his scene with 20 houses contains far more brushes than any of the screenshots above. Just because it looks simple in the distance doesn't mean it is. Open his map and look at the scene tree, then imagine that multiplied 20 times.

 

BTW, the latest build on the beta branch will use less memory, and there is another change I can make to improve this, but I don't want to make any drastic changes right now.

 

An extreme solution would be to implement brushes as a special type of builder object that isn't an entity. This would introduce a lot of limitations for brushes, but could reduce their memory usage substantially.

[YouGroove]

Why not some option that would turn some BSP sub tree in some model in the editor directly.

Perhaps the final model would no more be editable as BSP, but you could still keep the original BSP somewhere if you would need to edit it.

 

 

 

Tons of detailed BSP maps and they all work, the technology has been around for nearly 2 decades.

In your screenshot, it's BSP : i mean it's ver cubic look , while models allows you lot more details withotu looking cubic.

 

Take a closer look at more recent games made with Source Engine.

 

Today modern technology don't uses anymore BSP, i can use Crysis SDK even on my average laptop, i doubt it would be the case if the engine was based on BSP or was using BSP for buildings and structures

Also a modeler is really not so different from using BSP, you can still use Cubes and easy modify edges or faces , UV is very easy , you can use auto Unwrap and just give some angle and the modeler will make all seams for you.

 

YG - LOD for BSP? Are you kidding? How are you going to reduce a box with 8 vertices

This is what i said, keep BSP for very simple thinsg like some floor, some wall etc ... but when it is a complete house, the way to go is a model with LOD, you can't imagine how much power you will save using some 3 good LOD stages. Any today game uses LOD.

[Josh]

It's not an issue of polygon count, it's an issue of entity count.

 

Even though the house map looks very simple, it looks like it's composed of perhaps 5-6000 entities, which is far more than The Zone. I usually use pre-made models for those window and door frames. It's sort of an extreme case, he's modeling every little detail with CSG (which is fine) but expecting it to be copyable across a large area.

 

I think there's improvements that can be made here still...this only really came to my attention 2-3 days ago. But in the meantime you've got to keep track of how many brushes you're actually using. Just because another high-density enclosed scene looks artistically better doesn't mean it's using less objects. Too often I see people just copy a lot of polygons and expect it to be "better", then act surprised when a lower detail scene with better composition looks more detailed.

[Guppy]

Just out of curiosity why isn't adjacent BSP collapsed into a model at runtime? ( I actually thought they were )

 

I mean even if something like dual contouring surface reconstruction is of of scope, wouldn't simply fusing vertices within tolerance X and stuffing it into one batch/model help loads with performance?

 

I relalize that there would be a lot of polygons that would never be visible but the overall poly count is still fairly low in the csg house