Advanced Terrain Building in Leadwerks Game Engine 5

Entry posted by Josh · March 2, 2020

3,019 views

In Leadwerks Game Engine 4, terrain was a static object that could only be modified in the editor. Developers requested access to the terrain API but it was so complex I felt it was not a good idea to expose it. The new terrain system is better thought out and more flexible, but still fairly complicated because you can do so much with it. This article is a deep dive into the inner workings of the new terrain system.

Creating Terrain

Terrain can be treated as an editable object, which involves storing more memory, or as a static object, which loads faster and consumers less memory. There isn't two different types of terrain, it's just that you can skip loading some information if you don't plan on making the terrain deform once it is loaded up, and the system will only allocate memory as it is needed, based on your usage. The code below will create a terrain consisting of 2048 x 2048 points, divided into patches of 32 x 32 points.

local terrain = CreateTerrain(world, 2048, 32)

This will scale the terrain so there is one point every meter, with a maximum height of 100 meters and a minimum height of -100 meters. The width and depth of the terrain will both be a little over two kilometers:

terrain:SetScale(1,200,1)

Loading Heightmaps

Let's look at how to load a heightmap and apply it to a terrain. Because RAW files do not contain any information on size or formats, we are going to first load the pixel data into a memory buffer and then create a pixmap from that with the correct parameters:

--We have to specify the width, height, and format then create the pixmap from the raw pixel data.
local buffer = LoadBuffer("Terrain/2048/2048.r16")
local heightmap = CreatePixmap(2048, 2048, TEXTURE_R16, buffer)

--Apply the heightmap to the terrain
terrain:SetHeightMap(heightmap)

Because we can now export image data we have some options. If we wanted we could save the loaded heightmap in a different format. I like R16 DDS files for these because unlike RAW/R16 heightmap data these images can be viewed in a DDS viewer like the one included in Visual Studio:

heightmap:Save("Terrain/2048/2048_H.dds")

Here is what it looks like if I open the saved file with Visual Studio 2019:

After we have saved that file, we can then just load it directly and skip the RAW/R16 file:

--Don't need this anymore!
--local buffer = LoadBuffer("Terrain/2048/2048.r16")
--local heightmap = CreatePixmap(2048, 2048, TEXTURE_R16, buffer)

--Instead we can do this:
local heightmap = LoadPixmap("Terrain/2048/2048_H.dds")

--Apply the heightmap to the terrain
terrain:SetHeightMap(heightmap)

This is what is going on under the hood when you set the terrain heightmap:

bool Terrain::SetHeightMap(shared_ptr<Pixmap> heightmap)
{
	if (heightmap->size != resolution)
	{
		Print("Error: Pixmap size is incorrect.");
		return false;
	}
	VkFormat fmt = VK_FORMAT_R16_UNORM;
	if (heightmap->format != fmt) heightmap = heightmap->Convert(fmt);
	if (heightmap == nullptr) return false;
	Assert(heightmap->pixels->GetSize() == sizeof(terraindata->heightfielddata[0]) * terraindata->heightfielddata.size());
	memcpy(&terraindata->heightfielddata[0], heightmap->pixels->buf, heightmap->pixels->GetSize());
	ModifyHeight(0,0,resolution.x,resolution.y);
	return true
}

There is something important to take note of here. There are two copies of the height data. One is stored in system memory and is used for physics, raycasting, pathfinding, and other functions. The other copy of the height data is stored in video memory and is used to adjust the vertex heights when the terrain is drawn. In this case, the data is stored in the same format, just a single unsigned 16-bit integer, but other types of terrain data may be stored in different formats in system memory (RAM) and video memory (VRAM).

Building Normals

Now let's give the terrain some normals for nice lighting. The simple way to do this is to just recalulate all normals across the terrain. The new normals will be copied into the terrain normal texture automatically:

terrain:BuildNormals()

However, updating all the normals across the terrain is a somewhat time-consuming process. How time consuming is it? Let's find out:

local tm = Millisecs()
terrain:BuildNormals()
Print(Millisecs() - tm)

The printed output in the console says the process takes 1600 milliseconds (1.6 seconds) in debug mode and 141 in milliseconds release mode. That is quite good but the task is distributed across 8 threads on this machine. What if someone with a slower machine was working with a bigger terrain? If I disable multithreading, the time it takes is 7872 milliseconds in debug mode and 640 milliseconds in release mode. A 4096 x 4096 terrain would take four times as long, creating a 30 second delay before the game started, every single time it was run in debug mode. (In release mode it is so fast it could be generated dynamically all the time.) Admittedly, a developer using a single-core processor to debug a game with a 4096 x 4096 terrain is a sort of extreme case, but the whole design approach for Leadwerks 5 has been to target the extreme cases, like the ones I see while working on virtual reality projects at NASA.

What can we do to eliminate this delay? The answer is caching. We can retrieve a pixmap from the terrain after building the normals, save it, and then load the normals straight from that file next time the game is run.

--Build normals for the entire terrain
terrain:BuildNormals()

--Retrieve a pixmap containing the normals in R8G8 format
normalmap = terrain:GetNormalMap()

--Save the pixmap as an uncompressed R8G8 DDS file, which will be loaded next time as a texture
normalmap:Save("Terrain/2048/2048_N.dds")

There is one catch. If you ran the code above there would be no DDS file saved. The reason for this is that internally, the terrain system stores each point's normal as two bytes representing two axes of the vector. Whenever the third axis is needed, it is calculated from the other two with this formula:

normal.z = sqrt(max(0.0f, 1.0f - (normal.x * normal.x + normal.y * normal.y)));

The pixmap returned from the GetNormalMap() method therefore uses the format TEXTURE_RG, but the DDS file format does not support two-channel uncompressed images. In order to save this pixmap into a DDS file we have to convert it to a supported format. We will use TEXTURE_RGBA. The empty blue and alpha channels double the file size but we won't worry about that right now.

--Build normals for the entire terrain
terrain:BuildNormals()

--Retrieve a pixmap containing the normals in R8G8 format
normalmap = terrain:GetNormalMap()

--Convert to a format that can be saved as an image
normalmap = normalmap:Convert(TEXTURE_RGBA)

--Save the pixmap as an uncompressed R8G8 DDS file, which will be loaded next time as a texture
normalmap:Save("Terrain/2048/2048_N.dds")

When we open the resulting file in Visual Studio 2019 we see a funny-looking normal map. This is just because the blue channel is pure black, for reasons explained.

In my initial implementation I was storing the X and Z components of the normal, but I switched to X and Y. The reason for this is that I can use a lookup table with the Y component, since it is an unsigned byte, and use that to quickly retrieve the slope at any terrain point:

float Terrain::GetSlope(const int x, const int y)
{
	if (terraindata->normalfielddata.empty()) return 0.0f;
	return asintable[terraindata->normalfielddata[(y * resolution.x + x) * 2 + 1]];
}

This is much faster than performing the full calculation, as shown below:

float Terrain::GetSlope(const int x, const int y)
{
	int offset = (y * resolution.x + x) * 2;
	int nx = terraindata->normalfielddata[offset + 0];
	int ny = terraindata->normalfielddata[offset + 1];
	Vec3 normal;
	normal.x = (float(nx) / 255.0f - 0.5f) * 2.0f;
	normal.y = float(ny) / 255.0f;
	normal.z = sqrt(Max(0.0f, 1.0f - (normal.x * normal.x + normal.y * normal.y)));
	normal /= normal.Length(); 
	return 90.0f - ASin( normal.y );
}

Since the slope is used in expensive layering operations and may be called millions of times, it makes sense to optimize it.

Now we can structure our code so it first looks for the cached normals image and loads that before performing the time-consuming task of building normals from scratch:

--Load the saved normal data as a pixmap
local normalmap = LoadPixmap("Terrain/2048/2048_N.dds")

if normalmap == nil then

	--Build normals for the entire terrain
	terrain:BuildNormals()

	--Retrieve a pixmap containing the normals in R8G8 format
	normalmap = terrain:GetNormalMap()

	--Convert to a format that can be saved as an image
	normalmap = normalmap:Convert(TEXTURE_RGBA)

	--Save the pixmap as an uncompressed R8G8 DDS file, which will be loaded next time as a texture
	normalmap:Save("Terrain/2048/2048_N.dds")
	
else

	--Apply the texture to the terrain. (The engine will automatically create a more optimal BC5 compressed texture.)
	terrain:SetNormalMap(normalmap)

end

The time it takes to load normals from a file is pretty much zero, so in the worst-case scenario described we just eliminated a huge delay when the game starts up. This is just one example of how the new game engine is being designed with extreme scalability in mind.

Off on a Tangent...

Tangents are calculated in the BuildNormals() routine at the same time as normals, because they both involve a lot of shared calculations. We could use the Terrain:GetTangentMap() method to retrieve another RG image, convert it to RGBA, and save it as a second DDS file, but instead let's just combine normals and tangents with the Terain:GetNormalTangentMap() method you did not know existed until just now. Since that returns an RGBA image with all four channels filled with data there is no need to convert the format. Our code above can be replaced with the following.

--Load the saved normal data as a pixmap
local normaltangentmap = LoadPixmap("Terrain/2048/2048_NT.dds")

if normaltangentmap == nil then

	--Build normals for the entire terrain
	terrain:BuildNormals()

	--Retrieve a pixmap containing the normals in R8G8 format
	normaltangentmap = terrain:GetNormalTangentMap()

	--Save the pixmap as an uncompressed R8G8 DDS file, which will be loaded next time as a texture
	normaltangentmap:Save("Terrain/2048/2048_NT.dds")
	
else

	--Apply the texture to the terrain. (The engine will automatically create a more optimal BC5 compressed texture.)
	terrain:SetNormalTangentMap(normaltangentmap)

end

This will save both normals and tangents into a single RGBA image that looks very strange:

Why do we even have options for separate normal and tangent maps? This allows us to save both as optimized BC5 textures, which actually do use two channels of data. This is the same format the engine uses internally, so it will give us the fastest possible loading speed and lowest memory usage, but it's really only useful for static terrain because getting the data back into a format for system memory would require decompression of the texture data:

--Retrieve a pixmap containing the normals in R8G8 format
normalmap = terrain:GetNormalMap()
tangentmap = terrain:GetTangentMap()

--Convert to optimized BC5 format
normalmap = normalmap:Convert(TEXTURE_BC5)
tangentmap = tangentmap:Convert(TEXTURE_BC5)

--Save the pixmaps as an compressed BC5 DDS file, which will be loaded next time as a texture
normalmap:Save("Terrain/2048/2048_N.dds")
tangentmap:Save("Terrain/2048/2048_T.dds")

When saved, these two images combined will use 50% as much space as the uncompressed RGBA8 image, but again don't worry about storage space for now. The saved normal map looks just the same as the uncompressed RGBA version, and the tangent map looks like this:

Material Layers

Terrain material layers to make patches of terrain look like rocks, dirt, or snow work in a similar manner but are still under development and will be discussed in detail later. For now I will just show how I am adding three layers to the terrain, setting some constraints for slope and height, and then painting the material across the entire terrain.

--Add base layer
local mtl = LoadMaterial("Materials/Dirt/dirt01.mat")
local layerID = terrain:AddLayer(mtl)

--Add rock layer
mtl = LoadMaterial("Materials/Rough-rockface1.json")
local rockLayerID = terrain:AddLayer(mtl)
terrain:SetLayerSlopeConstraints(rockLayerID, 35, 90, 25)

--Add snow layer
mtl = LoadMaterial("Materials/Snow/snow01.mat")
local snowLayerID = terrain:AddLayer(mtl)
terrain:SetLayerHeightConstraints(snowLayerID, 50, 1000, 8)
terrain:SetLayerSlopeConstraints(snowLayerID, 0, 35, 10)

--Apply Layers
terrain:SetLayer(rockLayerID, 1.0)
terrain:SetLayer(snowLayerID, 1.0)

Material layers can take a significant time to process, at least in debug mode, as we will see later. Fortunately all this data can be cached in a manner similar to what we saw with normals and tangents. This also produces some very cool images:

Optimizing Load Time

The way we approach terrain building depends on the needs of each game or application. Is the terrain static or dynamic? Do we want changes in the application to be saved back out to the hard drive to be retrieved later? We already have a good idea of how to manage dynamic terrain data, now let's look at static terrains, which will provide faster load times and a little bit lower memory usage.

Terrain creation is no different than before:

local terrain = CreateTerrain(world, 2048, 32)
terrain:SetScale(1,200,1)

Loading the heightmap works the same as before. I am using the R16 DDS file here but it makes absolutely no difference in terms of loading speed, performance, or memory usage.

--Load heightmap
local heightmap = LoadPixmap("Terrain/2048/2048_H.dds")

--Apply the heightmap to the terrain
terrain:SetHeightmap(heightmap)

Now here is where things get interesting. Remember how I talked about the terrain data existing in both system and video memory? Well, I am going to let you in on a little secret: We don't actually need the normal and tangent data in system memory if we aren't editing the terrain. We can load the optimized BC5 textures and apply them directly to the terrain's material, and it won't even realize what happened!:

--Load the saved normal data as texture
local normaltexture = LoadTexture("Terrain/2048/2048_N.dds")

--Apply the normal texture to the terrain material
terrain.material:SetTexture(normaltexture, TEXTURE_NORMAL)

--Load the saved tangent data as texture
local tangenttexture = LoadTexture("Terrain/2048/2048_T.dds")

--Apply the normal texture to the terrain material
terrain.material:SetTexture(tangenttexture, TEXTURE_TANGENT)

Because we never fed the terrain any normal or tangent data, that memory will never get initialized, saving us 16 megabytes of system memory on a 2048 x 2048 terrain. We also save the time of compressing two big images into BC5 format at runtime. In the material layer system, which will be discussed at a later time, this approach will save 32 megabytes of memory and some small processing time, Keep in mind all those numbers increase four times with the next biggest terrain.

In debug mode the static and cached dynamic test are not bad, but the first time the dynamic test is run there is a long delay of 60 40 25 15 seconds (explanation at the end of this section). We definitely don't want that happening every time you debug your game. Load times are in milliseconds.

Dynamic Terrain (Debug, First Run)

Loading time: 15497

Dynamic Terrain (Debug, Cached Data)

Loading time: 1606

Static Terrain (Debug)

Loading time: 1078

When the application is run in release mode the load times are all very reasonable, although the static mode loads about five times faster than building all the data at runtime. Memory usage does not vary very significantly. Memory shown is in megabytes.

Dynamic Terrain (Release, First Run)

Loading time (milliseconds): 1834
Memory usage (MB): 396

Dynamic Terrain (Release, Cached Data)

Loading time: 386
Memory usage: 317

Static Terrain (Release)

Loading time: 346
Memory usage: 311

The conclusion is that making use of cached textures and only using dynamic terrains when you need them can significantly improve your load times when running in debug mode, which you will be doing during the majority of the time during development. If you don't care about any of these details it will be automatically handled for you when you save your terrain in the new editor but if you are creating terrains programmatically this is important to understand. If you are loading terrain data from the hard drive dynamically as the game runs (streaming terrain) then these optimizations could be very important.

While writing this article I found that I could greatly decrease the loading time in debug mode when I replaced STL with my own sorting routines in some high-performance code. STL usually runs very fast but in debug mode can be onerous. It's scary stuff, but I actually remember doing this same routine back when I was using Blitz3D, which if I remember correctly did not having any sorting functions. I found this ran slightly faster than STL in release mode and much faster in debug mode. I was able to bring one computationally expensive routine down from 20 seconds to 4 seconds (in debug mode only, it runs fine in release either way).

//Scary homemade sorting
firstitem = 0;
lastitem = mtlcount - 1;
sortcount = 0;
while (true)
{
	minalpha = 0;
	minindex = -1;
	for (n = firstitem; n <= lastitem; ++n)
	{
		if (listedmaterials[n].y == -1) continue;
		if (minindex == -1)
		{
			minalpha = listedmaterials[n].x;
			minindex = n;
		}
		else
		{
			if (listedmaterials[n].x < minalpha)
			{
				minalpha = listedmaterials[n].x;
				minindex = n;
			}
		}
	}
	if (minindex == -1) break;
	if (minindex == firstitem) ++firstitem;
	if (minindex == lastitem) --lastitem;
	sortedmaterials[sortcount] = listedmaterials[minindex];
	listedmaterials[minindex].y = -1;
	++sortcount;
}

There may be some opportunities for further performance increase in some of the high-performance terrain code. It's just a matter of how much time I want to put into this particular aspect of the engine right now.

Optimizing File Size

Basis Universal is the successor to the Crunch library. The main improvement it makes is support for modern compression formats (BC5 for normals and BC7 to replace DXT5). BasisU is similar to OGG/MP3 compression in that it doesn't reduce the size of the data in memory, but it can significantly reduce the size when it is saved to a file. This can reduce the size of your game's data files. It's also good for data that is downloaded dynamically, like large GIS data sets. I have seen people claim this can improve load times but I have never seen any proof of this and I don't believe it is correct.

Although we do not yet support BasisU files, I wanted to run the compressable files through it and see what how much hard drive space we could save. I am including only the images needed for the static terrain method, since that is how large data sets would most likely be used.

Uncompressed (R16 / RGBA): 105 megabytes
Standard Texture Compression (DXT5 + BC5): 48 megabytes
Standard Texture Compression + Zip Compression: 18.7 megabytes
BasisU + Standard Texture Compression: 26 megabytes
BasisU + Standard Texture Compression + Zip Compression: 10.1 megabytes

If we just look at one single 4096x4096 BC3 (DXT5) DDS file, when compressed in a zip file it is 4.38 megabytes. When compressed in a BasisU file, it is only 1.24 megabytes.

4096x4096 uncompressed RGBA: 64 megabytes
4096x4096 DXT5 / BC3: 16 megabytes
4096x4096 DXT5 / BC3 + zip compression: 4.38 megabytes
4096x4096 BasisU: 1.24 megabytes

It looks like we can save a fair amount of data by incorporating BasisU into our pipeline. However, the compression times are longer than we would want to use for terrain that is being frequently saved in the editor, and it should be performed in a separate step before final packaging of the game. With the open-source plugin SDK anyone could add a plugin to support this right now. There is also some texture data that should not be compressed, so our savings with BasisU is less than what we would see for normal usage. In general, it appears that BasisU can cut the size of your game files down to about a third of what they would be in a zip file.

A new update with these changes will be available in the beta tester forum later today.

1 Comment

Recommended Comments

Add a comment...

× Pasted as rich text. Paste as plain text instead

Only 75 emoji are allowed.

× Your link has been automatically embedded. Display as a link instead

× Your previous content has been restored. Clear editor

× You cannot paste images directly. Upload or insert images from URL.

Insert image from URL

Sign In

Development Blog

Advanced Terrain Building in Leadwerks Game Engine 5

1 Comment

Recommended Comments

Josh 15,482

Link to comment

Browse

Activity

Store

Support