Runtime Assets Management ?

[Guy Rabiller]

Hello,

 

A quick question before I decide to dig deeper with Leadwerks Engine and possibly buy a licence ( with pureLight ).

 

I'm currently looking at several 'engines' or 'realtime 3d renderers' for the purpose of creating a 'machinima' applicaiton.

 

So far the big problem I encouter is that a lot of 'game engines' or '3d engines' out there do not allow to manage assets at runtime, because everything must be pre-processed somehow and loaded as 'level' or 'universe' in the engine at runtime. Wich means one can't add/remove/modify or create or load assets ( characters, props, sets, etc.. ) at runtime.

 

This is obviously a showstopper for machinima applications wich are not games but uses game technologies.

 

So my big question here is:

 

Does Leadwerks Engine allows dynamic loading of assets at runtime ?

 

Is it possible to 'launch' the engine then programaticaly add 3d models, lights, cameras, etc.. define their 'materials', behaviors, etc.. once the engine is running so one can 'construct' iteratively, progressively his own 'level' not as a pre-processed file or 'level' but as a dynamic world ( and possibly save it on file eventualy ) ?

 

Thanks a lot in advance for a precise answer to this question.

 

Guy.

[Canardia]

Yes, you can load and unload all assets in realtime. Loading is even cached and instanced, so there is no disk action when you load the same model multiple times.

 

Loading and saving a model programmatically is very easy:

// load a model
model1=LoadModel("models/model1.gmf");

// unload a model
FreeEntity(model1);

[Guy Rabiller]

Hi,

 

Thanks for your answer.

 

Starting from here is it possible to load a model while the engine is runing the loop, I mean loading the model in a background task ( another thread perhaps ) wihtout affecting the current realtime rendering, then get notified somehow that the model has been loaded successfuly, then start including it in the current realtime rendering loop ?

 

In short, can we load a model at any point in time without blocking the current rendering ?

 

Guy.

[Rick]

The short answer is no, but you can preloading that model before hand. The models are instanced so once you load 1 specific model, loading others will be close to instant and not cause any slowdown. So the idea is to load a model you know you'll need to load at the start and just hide that one. That preload model doesn't have to be used in the game really but just used to get the model from file so other loads are "instant". Hope that makes sense.

[Guy Rabiller]

Hi Rick,

 

Thanks for your input.

 

For a machinima application, the user may be presented with some kind of repository with hundreds if not thousends of assets. In this case it would be point less to preload all the repository ( if even possible due to memory constraints ) then only show the needed assets.

 

I'm wondering though : what would happen if, while the engine is running and rendering some scene, I use another thread to load a model ?

 

Will the engine simply crash or something ? Are you aware of any test(s) of that kind ?

 

Guy.

[Rick]

LE is not thread safe. It will crash. Do a search and you will see a bunch of posts on this topic.

[caustic]

Hi - I'm in no way a LE expert but from my 'workarounds' around some products I would guess you could do the following:

 

1. if the game mechanics see that they need another model to load you send a trigger via pipe (or network or whatever you hip kids today use) to another process running in the background

2. the other process will load the file just to read it

3. the other process pipes (or whatever) the file back to LE and when LE comes to see what input is in the pipe it can pipe it to a memory file object, loading the model from there (removing the object afterwards (or GC)).

 

Don't know if that would be feasible for LE - maybe some LE expert should comment on that first.

[Canardia]

Like Rick said, loading models is instant since they are instanced after you have loaded them once.

I have made a demo with 100 million models in a SQLite3 database which runs on another thread and that way you can stream models infinitely without any delays.

The database is queried with a spherical 3D distance formula: sqr(dx^2+dy^2+dz^2) for the distance between the camera and each model in the database, and even with 100 million models in the database this takes only a few milliseconds, and since it's on another thread, it doesn't affect the FPS in the main thread at all, and the actual model load routines checks the results of the SQL query only every 10 seconds (there's no point checking it more often, since those models which are loaded/unloaded are still way further than the camera can see).

[VeTaL]

Hm, quite interesting idea... How did you achieved that?

 

//I have made a demo with 100 million models in a SQLite3 database which runs on another thread and that way you can stream models infinitely without any delays

 

Well, you have some models in separate database - thats ok... you have a second loop - thats ok.. but how did you load model from database to engine without delays?

Or you're talking about instanced models (as Rick said, you preload one model. and get coordinates of other models from SQL)?

[Canardia]

I have 2 different models, and once they are loaded during the game startup intro screen, all the other 100 million models load instantly when using the LoadModel() command.

It works also if you load 100 models in the game startup screen, then it takes a bit longer depening how big models and how many models you preload. But 100 normal models should load also in a few seconds, at maximum maybe 20 seconds.

[Laurens]

Yeah but since he expressed a concern regarding loading hundreds of assets beforehand, I think we can safely assume he meant different assets, in which case there is no possible way to properly load them in a separate thread.

[Guy Rabiller]

Hi,

 

Thanks all for your inputs and ideas.

 

I understand the pre-loading models Lumooja's idea, but it's still 'pre-loading', and Laurens is rightly assuming that it means the developper of the final application makes some assumptions about the content of what will be rendered by the engine.

 

This is a fundamental difference between a game application and a machinima application.

 

I can talk about this issue with a different perspective : Imagine a machinima application in a professional environment where several peoples are working together at the same time. While a camera operator is doing some rehearsal work with his camera - preparing dolly movement, etc.. according to the current set and some actions - the DP who is working on another computer ( but 'connected' to the same scene and rendering the same scene ) wants to add some lights, perhaps point light but may be object lights too or shadow casters, the set designer wants to add some props, etc..If the rendering must stop whenever something is loaded it's obvious that this will become annoying. Basicaly this means the all team will 'freeze' whenever someone wants to add something in the scene.

 

This is the kind of applications and situations I'm talking about.

 

Unfortunately, nowadays very little attention is given to this kind of situations by engines developpers who have a 'game centric' vision about their development. Crytek will release their 'Cinema' sand box version this summer, and according to what I've heard, I'm not sure they will even address this king of issues.

 

Perhaps Josh could be the first engine developper to take this kind of issues into consideration, that would be wonderfull because I think the Leadwerks engine has a lot of potentials and is taking the right path.

 

To add further to the discussion and the differences between game applications and machinima applications, lets considere the use of tools like pureLight wich I'm considering.

 

In a game environment, pureLight - a lightmapper - is considered as a pre-processing tool. In a machinima environment the same tool is considered as a post-processing tool ( to render the final images ).

 

In the second case, the user of the application is doing the lighting, not the game developper/designer. For this to work, the application developper then must have some way to expose the functionalities of the lightmapper to the user ( preview, etc.. ) wich seems to not be the case at the moment because the lighmapper developpers have a 'game centric vision' too.

 

The thing is that all the needed technologies nowadays exist to create wonderfull ( and professional ) machinima applications but in fact it is actualy very difficult to do so, not because it would be difficult to adapt the technologies, but just because of this 'game centric vision' adopted by middle-ware developpers.

 

Lets hope 'someone' is listening :-]

 

Guy.

[caustic]

To put this kind of stuff in another process without having the hazzle to restruct ((un-)serializing) your objects you could use some kind of shared objects.

 

If you know you need a new model to be loaded in the next cycles you just instance the super-class of all models with the name of the model you need and the other process would fill in the data part while the engine could carry on with drawing until the model data has been filled.

 

I read that you can use this kind of shared objects (in shared memory or something) also with non-thread-save modules because the shared object library would handle all the needed processing properly.

[Canardia]

Any professional studio needs several minutes to startup.

Each music workstation takes around a minute to startup, the PC takes 5 minutes to startup and load Cubase6, an musician takes around 30-60 minutes to startup with coffee, cigarettes and beers.

 

When the studio is loaded, everything is realtime.

LE2 can be networked in a studio, so each change is synced in realtime to all other LE2 clients and other apps.

[Guy Rabiller]

Any professional studio needs several minutes to startup.

Each music workstation takes around a minute to startup, the PC takes 5 minutes to startup and load Cubase6, an musician takes around 30-60 minutes to startup with coffee, cigarettes and beers.

 

When the studio is loaded, everything is realtime.

LE2 can be networked in a studio, so each change is synced in realtime to all other LE2 clients and other apps.

 

I'm not talking about 'starting up'. I'm talking about managing content after a startup.

 

I see in your profile you use Cubase and you are talking here about music workstation so I bet you know well this kind of stuff.

 

Well, very good, because I know very well this too, I also do sound design.

 

So we are on the same line here and perhaps I'll better succeed in explaining to you what I'm talking about :

 

I'm using Live from Ableton, I guess you know this application. Once Live is started, once you have loaded a song, then you can add any sample file at any time while the song is playing without disrupting the realtime audio playing.

 

And you don't have to pre-load hundreds of gibabytes of waveforms you may have on your hard drive for this.

 

You just load the audio on the fly.

 

Well, lets considere a model as an audio file, and a 3d scene as the song. Once the song is playing - the scene is rendering - , I would like to load any audio file at anytime - any model at anytime - and make it played by the audio engine mixed with the song - make it rendered with the current scene - without the need to stop the playing of the song - stopping the renderer loop.

 

Do you better see what I mean now ?

 

Guy.

[Guy Rabiller]

To put this kind of stuff in another process without having the hazzle to restruct ((un-)serializing) your objects you could use some kind of shared objects.

 

If you know you need a new model to be loaded in the next cycles you just instance the super-class of all models with the name of the model you need and the other process would fill in the data part while the engine could carry on with drawing until the model data has been filled.

 

I read that you can use this kind of shared objects (in shared memory or something) also with non-thread-save modules because the shared object library would handle all the needed processing properly.

 

Hi Caustic,

 

This sounds interresting, do you have more details regarding this approach and how this could be done in the context of the engine ?

 

Guy.

[Canardia]

I wonder if I could manage to write a multi-threading C++ version of LoadModel(). I could check what needs to be done inside LoadModel() to make it work exactly like the LE2 LoadModel() command.

[caustic]

Hi Caustic,

 

This sounds interresting, do you have more details regarding this approach and how this could be done in the context of the engine ?

 

Guy.

 

Hi Guy,

Im a noob in c++ but I would strongly recommend to use it there as this is the natural choice for programming with LE.

Though I know quite some possibilities to 'share objects' between processes I cannot promise that this stuff exists for C++ (for easy usage) also.

 

I'm totally on your side regarding streaming content into the engine. I wouldn't want to wait for a loading screen which loads stuff I wouldn't even probably need later anyway.

Also I hate when engines just use one core of my system and everytime they need something from disk they begin to freeze.

Just have a look at some "web2.0" apps - the golden rule is: preload all "place-holders" (fast), then display them when you can't load the content fast enough, BUT DISPLAY something at the place for which you are loading something!

This is something where many devs still have to learn (or the managers of them) - I think.

 

As for LE - you know when you need a model - plus: you know it in advance (most of the time).

e.g. If there are 'enemy units' coming in and they appear on the radar first and then they come into the range of sight.

 

So I would propose the stuff above to be able to load stuff while you are playing without letting the FPS down.

 

As I said I suck at C++ (atm) so I don't know what would be the fastest and most efficient lib to use here.

Generally said: You can use nearly every lib which gives you inter-process communication.

Just take the network architecture for example (which is quite slow and maybe some firewalls will cry).

You could for instance do a small file-loading server service.

Define (load_object(), is_object_loaded() and get_object()) and start it up.

 

When LE is running and enemies would come into "radar range" you would lookup if you already have that model loaded.

If not, you call the is_object_loaded("name") of the server service and ask if it has the object. It would say "no" maybe.

So you send a load_object("name") and continue with LE displaying the world and being smooth and fast responding.

In the next cycle you ask the server service again is_object_loaded("name") and this time it would say yes.

Then you call the get_object("name") and put this stream of data right into a pre-defined file_object. (like filestream (I hope this exists for C++)).

You would stream some kilobytes from the network buffer, and then in the next cycle more and then the rest somewhere in the future.

 

When the object is there you just feed the filestream object to the engine, thus loading the model from RAM.

 

So it all depends on which interface you use to communicate between the processes.

Maybe it's fast enough to give the network buffer as a filestream into LE within one Frame. I don't know.

Maybe it's better to use PIPEs or some kind of shared mem (some quick google: IPC / OpenMP) or shared objects lib or in-mem database or whatever!

Or Remote Procedure Calls (Java: Remote Method Invocation) or somekind of distributed object technology - there has to be some open and easy

lib for c++ somewhere.

In Python I would use http://www.xs4all.nl/~irmen/pyro3/ .

 

You would just need to search a lib where you can have another process fill a buffer or which can send some kilobytes per object per call

and thus you would be able to keep FPS high, load the model in the background (give the process low prio can work also but you won't need it I guess),

and 'stream' the model into the engine on the fly.

 

Hopefully this helps somehow.

[Pixel Perfect]

However you look at it the problem is a simple one, as the engine is not inherently thread safe you cannot have two threads access the same shared memory without locking it in some way. This is expensive in its own right but for something like model loading (which is intrinsically a lengthy process) it might be worth while. Whether this can be done without altering the existing source code I'm not sure.

[Guy Rabiller]

../.. Whether this can be done without altering the existing source code I'm not sure.

 

Yes, that's my concern.

 

I've red somewhere on this site that the Leadwerks engine does not use acceleration ( space partitioning ) structure and rely on hardware occlusion culling, to avoid pre-processing datas.

 

If the engine does not need preprocessed datas and if those can already be loaded on the fly ( in a blocking way for now though ) it sounds logical to me that the next step would be to be able to load those same datas in a non-blocking way while the engine is runing.

 

I'm not sure yet if the SDK expose the concept of scene graph but this could be done quite easily if the engine could allow us to create scene graph nodes independently of the current running scene graph, then at some point could allow us to 'attach' this node to the current scene graph. The first part could be intensive and long but that would be no problem because the engine wouldn't care about this node yet. Attaching the node could be then done very quickly. With this approach, it could work even if the engine is non-thread safe because the loading of the datas and the creation of the node could be done in another thread without any impact on the current scene graph, then attaching the node would be done in the main engine thread very quickly without significant impact on the FPS.

 

This does not sounds as a technical challenge to me, more a 'game centric vision' issue.

 

Well, hopefully perhaps this is planed somehow for a future release.

 

Guy.

[Rick]

Well, hopefully perhaps this is planed somehow for a future release.

 

I've never heard Josh talk about giving us more control over the scene graph, but he does plan on letting us load models on another thread. It would be another parameter to the LoadModel() method. We are hoping for some kind of callback then that let's us know when the model was fully loaded.

[Guy Rabiller]

I've never heard Josh talk about giving us more control over the scene graph, but he does plan on letting us load models on another thread. It would be another parameter to the LoadModel() method. We are hoping for some kind of callback then that let's us know when the model was fully loaded.

 

Ah! That's it!

 

Wonderfull news, thanks Rick, you made my day

 

Guy.

[caustic]

Can someone please please compile a list of all the features which are planned for LE3?

It can be unconfirmed, of course. Just so that I can see where the direction is pointed at.

[Canardia]

This is the official Roadmap for LE3 (or was):

http://www.leadwerks.com/werkspace/index.php?/page/roadmap

[VeTaL]

When LE is running and enemies would come into "radar range" you would lookup if you already have that model loaded.

If not, you call the is_object_loaded("name") of the server service and ask if it has the object. It would say "no" maybe.

So you send a load_object("name") and continue with LE displaying the world and being smooth and fast responding.

In the next cycle you ask the server service again is_object_loaded("name") and this time it would say yes.

Then you call the get_object("name") and put this stream of data right into a pre-defined file_object. (like filestream (I hope this exists for C++)).

You would stream some kilobytes from the network buffer, and then in the next cycle more and then the rest somewhere in the future.

 

When the object is there you just feed the filestream object to the engine, thus loading the model from RAM.

+1, i was talking about loading models from RAM in previous summer