I am a fan of the recent bodycam-style movement videos that have come out. It's an idea I have had before, and then seeing it working was very cool. Today I spent a little time trying out my own implementation to see how well it would work.
My idea was to use my VR headset as a mocap system to record myself walking and get the headset orientation for a few seconds, then play that back in a loop when the first-person player moves around. I started with a simple scene. The strip on the ground
Recently I came across a problem where HDRI images I loaded and converted into cube maps looked simultaneously too bright and too dark.
I found that a simple linear to sRGB conversion made the images look as I would expect.
I have read about sRGB and linear color spaces before, but for some reason this time it just clicked and I understood. Maybe since I am not longer dealing with low-level Vulkan nonsense, I have extra mental capacity to spend on more high-level concepts l
At long last, the engine that felt like it would never be done is here. This is all the result of an idea I had and wanted to put to the test. Of course I never anticipated it was possible to deliver a 10x improvement in rendering performance, but here we are with the benchmarks to prove it.
Research & Development
The last four years of research and development were an opportunity to rethink and redesign what a game engine should be. Along the way there were a few ideas I had that
Attention: The addition of our custom model file format has changed the way this system works. Colliders are now saved into the Ultra MDL file.
I haven't been blogging much because I am making so much progress that there would be too much to cover. In this article I will talk about one little system I spent the last week on and show the improvements I have made to the workflow over Leadwerks.
In the asset editor, when the root of a model is selected, a "Collider" field appears in the p
Not long ago, I wrote about my experiments with AI-generated textures for games. I think the general consensus at the time was that the technology was interesting but not very useful in its form at the time. Recently, I had reason to look into the OpenAI development SDK, because I wanted to see if it was possible to automatically convert our C++ documentation into documentation for Lua. While looking at that, I started poking around with the image generation API, which is now using DALL-E 2. Ste
As I have explained before, I plan for Ultra Engine to use glTF for our main 3D model file format, so that your final game models can be easily loaded back into a modeling program for editing whenever you need. glTF supports a lot of useful features and is widely supported, but there are a few missing pieces of information I need to add into it. Fortunately, this JSON-based file format has a mechanism for extensions that add new features and data to the format. In this article I will describe th
I have two goals for the art pipeline in the new game engine.
Eliminate one-way 3D model conversions.
Use the same game asset files on all platforms.
In Leadwerks, 3D models get converted into the proprietary MDL format. Although the format is pretty straightforward and simple, there is no widespread support for loading it back into 3D modeling programs. This means you need to keep a copy of your 3D model in FBX and MDL format. You may possibly want to keep an additional fi
Package plugins are now supported in Ultra Engine 1.0.2. This allows the engine to handle other package formats besides just ZIP. In order to test this out, I created the Quake Loader plugin, which currently supports the following formats used in the original Quake game:
PAK
WAD
BSP (textures)
SPR
LMP
Why Quake? Well, the original Quake game is what got me into game development through modding, but Quake is also great for testing because it's so weird.
I wanted to take some time to investigate geospatial features and see if it was possible to use GIS systems with Ultra Engine. My investigations were a success and resulted in some beautiful lunar landscapes in a relatively short period of time in the new game engine.
I have plans for this, but nothing I can say anything specific about right now, so now I am working on the editor again.
Leadwerks had a very well-defined workflow, which made it simple to use but also somewhat limi
Midjourney is an AI art generator you can interact with on Discord to make content for your game engine. To use it, first join the Discord channel and enter one of the "newbie" rooms. To generate a new image, just type "/imagine" followed by the keywords you want to use. The more descriptive you are, the better. After a few moments four different images will be shown. You can upsample or create new variations of any of the images the algorithm creates.
And then the magic begins:
As I have stated before, my goa for this game enginel is not to build a marketplace of 3D models, but instead to just make sure our model loading code reliably loads all 3D models that are compliant with the glTF specification. I started testing more 3D models from Sketchfab, and found that many of them are using specular/gloss materials. At first I thought I could just fudge the result, but I wasn't getting very good results, and the Windows 10 3D Object Viewer was showing them perfectly. This
I've been working hard getting all the rendering features to work together in one unified system. Ultra Engine, more than any renderer I have worked on, takes a lot of different features and integrates them into one physically-based graphics system. A lot of this is due to the excellent PBR materials system that Khronos glTF provides, and then there are my own features that are worked into this, like combined screen-space and voxel ray traced reflections.
Anyways, it's a lot of di
Autodesk 3ds Max now supports export of glTF models, as well as a new glTF material type. The process of setting up and exporting glTF models is pretty straightforward, but there are a couple of little details I wanted to point out to help prevent you from getting stuck. For this article, I will be working with the moss rocks 1 model pack from Polyhaven.
Getting geometry into 3ds Max is simple enough. I imported the model as an FBX file.
To set up the material, I opened the compa
As the first release of Ultra Engine approaches, it seems clear that the best way to maximize its usefulness is to make it as compatible as possible with the Leadwerks game engine. To that end, I have implemented the following features.
Native Loading of Leadwerks File Formats
Ultra Engine loads and saves DDS, glTF, and OBJ files. Other formats are supported by plugins, both first and potentially third-party, for PNG, JPG, BMP, TGA, TIFF, GIF, HDR, KTX2, and other files. Additionally,
HDR skyboxes are important in PBR rendering because sky reflections get dampened by surface color. It's not really for the sky itself, but rather we don't want bright reflections to get clamped and washed out, so we need colors that go beyond the visible range of 0-1.
Polyhaven has a large collection of free photo-based HDR environments, but they are all stored in EXR format as sphere maps. What we want are cubemaps stored in a single DDS file, preferably using texture compression.
We'
I'm wrapping up the terrain features now for the initial release. Here's a summary of terrain in Ultra Engine.
Terrains are an entity, just like anything else, and can be positioned, rotated, or scaled. Non-square terrains are supported, so you can create something with a 1024x512 or whatever resolution. (Power-of-two sizes are required.)
Editing Terrain
Ultra Engine includes an API that lets you modify terrain in real-time. I took something very complicated and distilled it down
I've actually been doing a lot of work to finalize the terrain system, but I got into tessellation, and another rabbit hole opened up. I've been thinking about detailed models in VR. Tessellation is a nice way to easily increase model detail. It does two things:
Curved surfaces get smoother (using point-normal triangles or quads)
A displacement map can be used to make small geometric detail to a surface.
These are really nice features because they don't require a lot of mem
Ultra Engine makes much better use of class encapsulation with strict public / private / protected members. This makes it so you can only access parts of the API you are meant to use, and prevents confusion about what is and isn't supported. I've also moved all internal engine commands into separate namespaces so you don't have to worry about what a RenderMesh or PhysicsNode does. These classes won't appear in the intellisense suggestions unless you were to add "using namespace UltraRender" to y
Happy Friday! I am taking a break from global illumination to take care of some various remaining odds and ends in Ultra Engine.
Variance shadow maps are a type of shadowmap filter technique that use a statistical sample of the depth at each pixel to do some funky math stuff. GPU Gems 3 has a nice chapter on the technique. The end result is softer shadows that run faster. I was wondering where my variance shadow map code went, until I realized this is something I only prototyped in OpenGL a
I'm finalizing the shaders, and I was able to pack a lot of extra data into a single entity 4x4 matrix:
Orthogonal 4x4 matrix
RGBA color
Per-entity texture mapping offset (U/V), scale (U/V), and rotation
Bitwise entity flags for various settings
Linear and rotational velocity (for motion blur)
Skeleton ID
All of that info can be fit into just 64 bytes. The shaders now use a lot of snazzy new function like this:
void ExtractEntityInfo(in uint
I have more than one light bounce working now, and it looks a lot nicer than single-bounce GI. The ambient light here is pure black. All light is coming off from the direct light, and bouncing off surfaces.
It will take some time to get the details worked out, and more bounces will require more memory. I'm actually kind of shocked how good looking it is. This is just a single 128x128x128 volume texture at 0.25 meters per voxel. Light leaks seem to be not a problem, even at that low re
Previously I wrote about introducing latency to the voxel cone step tracing realtime global illumination system. The idea here is to improve performance and quality, at the cost of a small delay when the GI calculation gets updated. The diffuse GI lighting gets cached so the final scene render is very fast.
Here's what a gradual GI update does. Of course, this will be running unseen in the background for the final version, but this shows what is actually happening:
My new video pro
Since previously determining that voxels alone weren't really capable of displaying artifact-free motion, I have been restructuring the GI system to favor speed with some tolerance for latency. The idea is that global illumination will get updated incrementally in the background over the course of a number of frames, so the impact of the calculation per frame is small. The new GI result is then smoothly interpolated from the old one, over a period of perhaps half a second. Here's a shot of the r
I have not used the engine outside the editor in a while, but I needed to for performance testing, so now I am back to real-time rendering. During the development of the GI system I broke most of the light types, so I had to spend a couple of days getting those to work again. While doing this, I decided to resolve some longstanding issues I have put off.
First, the PBR lighting will use a default gradient in place of the skybox, if no reflection map is set for the world. This is based off t
Before proceeding with multiple GI volumes, I decided to focus on just getting the lighting to look as close to perfect as possible, with a single stage.
Injecting the ambient light into the voxel data made flat-lit areas appear much more "3D", with color bleeding and subtle contours everywhere.
Lighting only:
Lighting + albedo
Some adjustments to the way the sky color is sampled gave a more lifelike appearance to outdoor lighting.
Before:
After. N
