GMF2, Plugins, and Modding
Entry posted by Josh ·
6,303 views
I wanted to get a working example of the plugin system together. One thing led to another, and...well, this is going to be a very different blog.
GMF2
The first thing I had to do is figure out a way for plugins to communicate with the main program. I considered using a structure they both share in memory, but those always inevitably get out of sync when either the structure changes or there is a small difference between the compilers used for the program and DLL. This scared me so I went with a standard file format to feed the data from the plugin to the main program.
GLTF is a pretty good format but has some problems that makes me look for something else to use in our model loader plugins.
- It's text-based and loads slower.
- It's extremely complicated. There are 3-4 different versions of the file format and many many options to split it across multiple files, binary, text, or binary-to-text encoding.
- It's meant for web content, not PC games.
- Tons of missing functionality is added through a weird plugin system. For example, DDS is supported through a plugin, but a backup PNG has to be included in case the loaded doesn't support the extension.
The GMF file format was used in Leadwerks. It's a custom fast-loading chunk-based binary file format. GMF2 is a simpler flat binary format updated with some extra features:
- Vertices are stored in a single array ready to load straight into GPU memory.
- Vertex displacement is supported.
- Compressed bitangents.
- Quad and line meshes are supported.
- LOD
- Materials and textures can be packed into the format or loaded from external files.
- PBR and Blinn-Phong materials
- Mesh bounding boxes are supported so they don't have to be calculated at load time. This means the vertex array never has to be iterated through, making large meshes load faster.
I am not sure yet if GMF2 will be used for actual files or if it is just going to be an internal data format for plugins to use. GLTF will continue to be supported, but the format is too much of a mess to use for plugin data.
Here's a cool five-minute logo:
The format looks something like this:
char[4] ID "GMF2" int version 200 int root //ID of the root entity int texture_count //number of textures int textures_pos //file position for texture array int materials_count //number of materials int materials_pos //file position for materials int nodes_count //number of nodes int nodes_pos //file position for nodes
As you can see, it is really easy to read and really easy to write. There's enough complexity in this already. I'm not bored. I don't need to introduce unnecessary complexity into the design just so I can show off. There are real problems that need to be solved and making a "tricky" file format is not one of them.
In Leadwerks 2, we had a bit of code called the "GMF SDK". This was written in BlitzMax, and allowed construction of a GMF file with easy commands. I've created new C++ code to create GMF2 files:
//Create the file GMFFile* file = new GMFFile; //Create a model node = new GMFNode(file, GMF_TYPE_MODEL); //Add an LOD level GMFLOD* lod = node->AddLOD(); //Add a mesh GMFMesh* mesh = lod->AddMesh(3); //triangle mesh //Add a vertex mesh->AddVertex(0,0,0, 0,0,1, 0,0, 0,0, 255,255,255,255); mesh->AddVertex(0,0,1, 0,0,1, 0,0, 0,0, 255,255,255,255); mesh->AddVertex(0,1,1, 0,0,1, 0,0, 0,0, 255,255,255,255); //Add a triangle mesh->AddIndice(0); mesh->AddIndice(1); mesh->AddIndice(2);
Once your GMFFile is constructed you can save it into memory with one command. The Turbo Plugin SDK is a little more low-level than the engine, so it includes a MemWriter class to help with this, since the engine Stream class is not present.
As a test I am writing a Quake 3 MD3 import plugin and will provide the project and source as an example of how to use the Turbo Plugin SDK.
Packages
The ZIP virtual file system from Leadwerks is being expanded and formalized. You can load a Package object to add new virtual files to your project. These will also be loadable from the editor, so you can add new packages to a project, and the files will appear in the asset browser and file dialogs. (Package files are read-only.) Plugins will allow packages to be loaded from file formats besides ZIP, like Quake WADs or Half-Life GCF files. Notice we keep all our loaded items in variables or arrays because we don't want them to get auto-deleted.
//Load Quake 3 plugins auto pk3reader = LoadPlugin("Plugins/PK3.dll"); auto md3loader = LoadPlugin("Plugins/MD3.dll"); auto bsploader = LoadPlugin("Plugins/Q3BSP.dll");
Next we load the game package files straight from the Quake game directory. This is just like the package system from Leadwerks.
//Load Quake 3 game packages std::wstring q3apath = L"C:/Program Files (x86)/Steam/steamapps/common/Quake 3 Arena/baseq3"; auto dir = LoadDir(q3apath); std::vector<shared_ptr<Package> > q3apackages; for (auto file : dir) { if (Lower(ExtractExt(file)) == L"pk3") { auto pak = LoadPackage(q3apath + L"/" + file); if (pak) q3apackages.push_back(pak); } }
Now we can start loading content directly from the game.
//Load up some game content from Quake! auto head = LoadModel(world, "models/players/bitterman/head.md3"); auto upper = LoadModel(world, "models/players/bitterman/upper.md3"); auto lower = LoadModel(world, "models/players/bitterman/lower.md3"); auto scene = LoadScene(world, "maps/q3ctf2.bsp");
Modding
I have a soft spot for modding because that is what originally got me into computer programming and game development. I was part of the team that made "Checkered Flag: Gold Cup" which was a spin-off on the wonderful Quake Rally mod:
I expect in the new editor you will be able to browse through game files just as if they were uncompressed in your project file, so the new editor can act as a modding tool, for those who are so inclined. It's going to be interesting to see what people do with this. We can configure the new editor to run a launch script that handles map compiling and then launches the game. All the pieces are there to make the new editor a tool for modding games, like Discreet's old Gmax experiment.
I am going to provide official support for Quake 3 because all the file formats are pretty easy to load, and because gmax can export to MD3 and it would be fun to load Gmax models. Other games can be supported by adding plugins.
So here are some of the things the new plugin system will allow:
- Load content directly from other games and use it in your own game. I don't recommend using copyrighted game assets for commercial projects, but you could make a "mod" that replaces the engine and requires the player to own the original game.
- You could probably safely use content from any Valve games and release a commercial game on Steam.
- Use the editor as a tool to make Quake or other maps.
- Add plugin support for new file formats.
This might all be a silly waste of time, but it's a way to get the plugin system working, and if we can make something flexible enough to build Quake maps, well that is a good test of the robustness of the system.
-
4
12 Comments
Recommended Comments