Neutrino Game Engine

[Pixel Perfect]

Most of you know me; I have been a member of the forums and user of Leadwerks technology from the beginning of Leadwerks 2 and have been slowly developing a game engine and toolset based on Leadwerks 2 in my spare time. I have two potential games in progress, an FPS/RPG called 'STRANDED - The Game' and an Adventure/RPG game called 'The Kingdom Of Soul'. Both of these have had the underlying game engine functionality built, to a certain extent, in preparation for the construction of the games themselves and my engine is designed to support both genres seamlessly.

 

Previously I have maintained separate threads for each but as the two games now share a common engine I'm starting this thread to consolidate all output based on my Neutrino game engine development.

 

Early trial space scene from my STRANDED game:

 

 

As some of you will already know I developed my own game editor for the purposes of facilitating the design of my path finding solution and AI and to act as a test bed for the engines functionality. The game engine is embedded into the editor and runs in real time within in it. I added FSM (Finite state machine) support to allow AI development and animation driven motion and blending support. I have been working more recently on integrating the EKIOne path finding/AI product from Artificial Technology in Germany as it offers far better tools than I could hope to develop. I believe once I have EKIOne on board I will be able to expand the AI design far beyond what I can currently manage with my simple implementation of FSMs.

 

However, whilst I've been waiting for an updated SDK from Artificial Technology (which I have now received) I went back to playing with my Cover Point functionality in my game engine and now have that working to a point where I can demo some of it. It’s still pretty basic and needs further refinement but the foundations are there and I hope to export this functionality for use with EKIOne once I have that fully embedded.

 

The input to this thread is likely to remain sporadic for the meantime as I develop further elements of the engine and toolsets but I'll try and update it with progress as and when I complete further goals or have anything interesting to show.

 

So, simple cover point functionality running in my game editor (Note: the NPCs do not have the ability to kill the player yet so it's a little one sided, even if this was coded it would be switched off for testing purposes). Again ... best watched in 720p in full screen:

 

[Roland]

Looking great Pixel. All built with LE2 ? C++ and Lua?

[Pixel Perfect]

Thanks Roland. Yes, just LE2 and C++. I have never needed to use Lua yet.

[Aily]

Impressive bots! My dream is to make level models for such project.

 

The first thing that i start make with Leadwerks is FPS shooter, and on bots it was over. It's one of hardest part of FPS i thinking. And you maked it! With levels now must be easier.

 

Good that you still continue working on it B)

[Josh]

I like how they run for cover. You'll get more long-term views if you post your video here:

http://www.leadwerks.com/werkspace/page/videos/_/le2

[Pixel Perfect]

Impressive bots! My dream is to make level models for such project.

 

The first thing that i start make with Leadwerks is FPS shooter, and on bots it was over. It's one of hardest part of FPS i thinking. And you maked it! With levels now must be easier.

 

Good that you still continue working on it

Thanks Aily. It does require a lot of code and functionality to be in place before you can start to put this sort of behaviour together but once it's there it's fairly easy to construct the AI. But to increase the complexity I really do need the better tools to make it easier to manage; which is where EKIOne will come in.

 

I like how they run for cover. You'll get more long-term views if you post your video here:

http://www.leadwerks.com/werkspace/page/videos/_/le2

I've posted a copy to the video gallery, thanks Josh!

[Pixel Perfect]

Cover points now enabled accross the entire level. The type of scenario shown in the video can be played out anywhere in the level. Just need to code the partial cover points now where NPCs will crouch behind smaller objects for cover and randomly stand up to fire; to add some more variety and realism

[Paul Thomas]

Nice work, Pixel. Was waiting for another progress update from you.

[Pixel Perfect]

Thanks Paul. Just saw your R.A.T.S. project, that's looking well cool.

[Paul Thomas]

Thanks. Not sure how you found it but glad you found it, lol. It's the project that went to E3 2011 I mentioned in a status (that I was preparing for at the time) I wrote a little while ago. We've also acquired our UE3 license.

[Pixel Perfect]

Having looked briefly at The Game Creators 'Dark Voices' package a few months back I'd decided I'd have a go at getting it working in Leadwerks when I had some free time. This is the result. The coding was quite easy but converting their demo model proved to be a nightmare.

 

They provide a tool (licenced from Annosoft LLC in the States) which converts WAV files containing speech into a list of sequenced phonemes which you can then use to drive a model which has had the phonemes animated. Although the outputted file provides support for all 44 phonemes in the English language the example code maps these to just 9 basic phoneme shapes making the model animation far easier.

 

This is the result using a few speech examples from the Dark Voices package (the voice sync has gone out slightly since loading the video to You Tube, the original is pretty much bang on). I'm quite happy this is usable and intend to integrate it into my engine soon and provide speech support for my character models

 

[cassius]

Nice work. Enjoyed watching. Encouraging too.

[DigitalHax]

It's really great to see things such as this being made at such speed!

[Guest Red Ocktober]

impressive Pixel... if you get a free second, you may want to take a look at the truespace 5/7 facial animator to see if there's anything there that you can use...

 

--Mike

[macklebee]

good work pixel... very nice

[Pixel Perfect]

Thanks guys, it was good to get this done as it's not an expensive package and does work pretty well. The core code is really easy to implement in Leadwerks. The difficult thing is getting the character model heads rigged and animated but given that the rest is plain sailing!

 

 

if you get a free second, you may want to take a look at the truespace 5/7 facial animator to see if there's anything there that you can use...

Thanks for the pointer Mike, I am about to embark on a series of modelling tutorials for Blender but where this specific requirement is concerned I'll consider anything that looks like it might get the job done without too much pain!

[Pixel Perfect]

By the way, I’m not planning on doing any formal release of the code as I’ve already moved on to working on the completion of the Leadwerks end of the EKI One project, but if anyone buys Dark Voices and wants help with this just PM me and I’ll provide advice and some sample code if need be.

[flachdrache]

Very nice work ... perfectness screams "move your head" though - but in a cutscene one blending this into the skeleton setup i assume.

[Pixel Perfect]

Yeah, this demo was literally thrown together in five minutes having got the lip syncing working and never meant to be a polished work of art (which it clearly isn't ).

 

You're quite right, in game the facial animation would just be part of the overall skeletal animation and the head would move to make it look more animated and natural. There are a few more facial animations that I've not shown on this particular model, however, for anyone rigging and animating their own model there are no limits to what you could achieve.

 

If you were prepared to animate the entire 44 phonemes too (as opposed to the sub set of 9 used here) then the speech animation would pretty much approach real life quality. But to be honest, for most games the 9 are probably acceptable.

[paramecij]

This will be a very fine addition to your engine, I'm sure. Nicely done!

 

I had a brief look at the Dark Voices pack.. My assumptions are that the real value of the pack is in the provided wav-to-phonemes tool, or rather the pack -IS- the tool, and the head is thrown in as a bonus, correct?. And there are no additional dependencies for the game or limitations regarding the model/rig/animation set-up, so one could use morph instead of bone driven animation for example, or whatever, and the only "glue code" needed is for parsing the generated phoneme file (which should be easy and very cheap to process)? And it seems what makes a difference is the quality of the rig and animations themselves, not so much the quality of the extracted phonemes data.

 

You did good work on rigging and animating the head, it looks very usable, and I'm sure you'll get even better results with some practice ! How did you do your expression and phoneme animations, are they essentially single key poses, which you blend between at 'playback', something else?

 

Cheers

[Pixel Perfect]

You are absolutely correct on all counts! The only reason for buying the pack is to gain the wav-to-phonemes tool. As you point out once you have this then it can be used to drive any form of animated speech, not just skeletal. I load, parse and sequence already converted phoneme files in real time but it also comes with a dll which would allow you to create the phoneme data on the fly too if that is required.

 

I didn't rig or animate the head as it comes ready animated with the package, however, it proved impossible to convert their X file successfully without breaking quite a bit of it so it had to be re-done in Fragmotion which was what has taken the time (as I'm not a modeller/rigger/animator). The animations are, as you suggest, simply single frame poses which using variable interpolation during playback result in the smooth animation.

 

There are some FPSC character packs that come ready rigged for dark voices but I've no idea what the quality is like as they don't even advertise the fact that they are Dark Voices compatible! I will have to rig my own models unless I can get someone to do this for me. I'm assuming not all models will be suitable unless some thought has gone into the topology of the head.

[paramecij]

.. And thanks for doing and posting this, I admit I haven't given this subject much thought before, but It always seemed to me that facial animation was a bit out of my reach. Because most of the pro models (even some specifically made for games) I considered buying for my game had the facial expressions and/or phoneme poses done as morphs, I thought this the only way.. so I would at the very least need to write some form of morph animation shader, a custom exporter for max, and figure out a lot of stuff in between..

 

But you have shown that skeletal facial animation can go a long way, and that lip-sync in real time is both easy* and cheap (in terms of cpu cycles and money), even for amateur/indie developers! (again)

 

 

* easy only if you're not as rigging impaired as me

[Pixel Perfect]

But you have shown that skeletal facial animation can go a long way, and that lip-sync in real time is both easy* and cheap (in terms of cpu cycles and money), even for amateur/indie developers! (again)

Thanks paramecij, that was certainly the intention for the demo. I think this should be very achievable by any one intending to develop games using Leadwerks.

 

I have taken a FPSC character model today which is supposedly Dark Voices compatible and analysed it. It is indeed rigged for facial animation containing the necessary bone structure and vertex weightings. They contain no animated poses though, just the standard character animation for the main skeleton so I'm guessing that FPSC supports lip synced speech by directly manipulating the facial bones in real time.

 

I have started today adding the necessary facial poses to the model which, although time consuming, is proving quite successful and enjoyable. Once I have all the poses done I should be able to simply export the animations and then import them to update the remaining models in the pack.

 

Although they are pretty low poly and not really of high enough quality for Leadwerks it will at least serve as a vehicle to test out some more advanced lip synced scenes and good practice for when I want to rig and animate a better set of models.

 

I'll update this thread with the results once it's completed.

[Pixel Perfect]

My facial animation process is progressing. I have successfully animated quite a few of the poses and tested the export and re-import of the animations onto the other characters. This all works which is great, although again the overall quality of the models is poor.

 

It will undoubtedly allow me to put some sort of experimental cut scene together though to fully evaluate how much work is involved in automating this. Looks like I might need to code a cut scene editor function into my exiting editor and code the framework support for handling cut scenes seamlessly.

 

I will still need support for simple real time triggered speech response though as the requirement for scripted sequences might be limited.

[Pixel Perfect]

Successfully tested the first character tonight, it's talking away quite nicely. Will have to give some serious thought now to how I will proceed with this. The advantage to using a unified skeletal structure through out all my required character models is apparent but would mean pretty much abandoning all the models I have purchased so far. I need to start looking at what facial rigging systems are out there and get an idea of what the technology offers and the associated costs.