Josh

The development of Leadwerks3D seemed like an impossible task at first. Not only did I need to write a new 3D engine entirely in C++ mostly by myself, but I had to make it run on four platforms (Android, iOS, Windows, and Mac), with a scalability all the way from the fixed-function pipeline up to the very latest hardware tessellation features. All logical sense told me this was impossible. Even large companies with 100 programmers or more haven't yet been able to put out a 3D engine with the range of platforms and scalability I am targeting. What makes me think I can do it? As we approach the initial release of Leadwerks3D, what I thought was impossible seems much more feasible now. During the course of development of Leadwerks3D, I've learned some important ideas that give a lot of hope to me and to indie game developers everywhere. Systemic InterdependenceWhen managing complex tasks there's something called systemic interdependence. This refers to the idea that when you have many people working on one project, they are rarely isolated from one another. They often have to stop and wait for another person's job to be done before they can continue with their own, and programmers often end up stepping on each other's toes when they work together. Even though only Aria and I are working on the source right now, we have experienced this. It's very important we keep busy with separate aspects of the engine. Aria works on platform implementation issues, I work on the core engine design, and we try our best to keep the source code repository in sync. So far this works well for us, but I could easily see a lot of problems arising if we had more people working on parts of the engine that aren't so easily compartmentalized. Think about your own game. What if I told you I was willing to give you as much money as you wanted to hire programmers, (as long as your final project gave me a big return on my money). Sound great, right? So you go out and hire ten programmers. They come in every weekday, 9 to 5. They're willing to work, but you have to instruct them and keep them busy and productive. Don't have anything for them to do at the moment while you finish one little part? Too bad, they're still on the clock and you still have to pay them. With ten programmers, would your game get done ten times faster? Almost certainly not. In fact, as you add programmers, your work output will probably look something like the graph below. With one programmer, we have an output of 100 arbitrary units. We gain efficiency with the first few programmers, but the effect lessens with each additional worker. They have to wait for another person to finish something. Now the code repository is conflicted, and you have to figure out how who added what code. You also have an increasing number of relationships and communication between individuals. Once we get past five programmers, additional programmers actually have a detrimental effect to our output. If we continue adding more workers, we can reach levels of output that are below even that of a single programmer. Imagine if 30 people were working on your game. It would be chaos, and that whole time you would be burning money as they came in every day. Meanwhile, your investors would be sitting impatiently, expecting you to get back a lot more money than they put in. My chart above isn't precise, and the number of programmers a project can support will vary based on its ease of compartmentalization, but the overall idea stands: Software development does not scale very well with additional manpower. This is why small companies are continually springing up and outmaneuvering larger ones. Android came from a small company. It was bought by Google, but only after a small team had built the foundation. Minecraft came from a very small company. Kinect originally came from a small company. Small companies with focused goals are able to compete with much larger companies because of the effect of diminishing returns. Performance and MotivationAnother problem big companies have is motivation. In a professional software development environment, managers like to have measurable performance metrics to evaluate employee efficiency. These are used for performance evaluations, and are the basis for salary increases, promotions, and disciplinary actions. Managers set goals and milestones that can be easily measured after a period of time. Workers respond to this by doing exactly what management tells them because they get rewarded for it. If a worker has an idea for a new technique that might or might not work, they're not as likely to spend time on it in this environment. If it works, the company gets to keep their idea and they get nothing. If it fails, they've wasted time on something that doesn't fit into their manager's ideas of measurable performance. Now, a lot of tech companies do try to encourage innovation, but if a worker has an idea for something they really believe in, they are more likely to form their own company where they can be in complete control of their vision. The professional software development environment does not encourage risk-taking and innovation the same way small companies do, because they need easily measurable metrics to assess performance. They only get out of employees what they reward, and it's hard to measure the value of new ideas and attention to detail. ConclusionThese ideas I've learned during the development of Leadwerks3D are directly applicable to your own game projects. Don't get discouraged when you run into a tough problem. Even if you had more help, you would still have to figure out a solution, one way or another, and you wouldn't be able to shift that responsibility off onto employees. You can do it, just be wise about where you focus your efforts and plan carefully. --Edit-- I posted this article on GameDev.net and got a few interesting responses:

Just download the Lua library, include it in your project, and call the Lua functions directly.

That's pretty amazing.

That's why they call him macklebee...

I think you'll get the best results with BlitzMax + Leadwerks Engine, but everyone has their own preferences.

Yep, Objects in BlitzMax are like a memory-managed pointer. I had trouble with this concept when I started with C++.

I spent most of today getting the Android library more polished, especially when handling application switching. It's a little tricky because Android doesn't automatically manage your sound channels and OpenGL resources, so these need to be reloaded when an app regains focus. Here's a video, which I obviously had way too much fun with in iMovie:

Vec3 vs. Vector3 is a 43% reduction in size. App vs. Application is a 73% reduction. Besides, "Apps" are for everything. Didn't Steve Ballmer tell you? PCs today have new form factors and touchscreens, and Windows 8 will be a powerful operating system capable of running the Apps that consumer want, like "Weather"! It will be CHROMELESS so it won't confuse your pretty little head! CHROMELESS!!! WITHOUT CHROME!!! As you know, teens are very social creatures!!!

Real programmers use Notepad.

The windows class on OSX has been a bit neglected, so i spent some time getting it ready for the final release. Creating a full-screen window on Mac is a little funny. There's a fullscreen mode that's been around a while, but it locks the system so that CMD+TAB doesn't work. That means if your application crashes, or you don't program it to close with the escape key, you'll be stuck in fullscreen mode, with no way out but a computer restart! You can create a window on top of everything else, and CMD+TAB will still work, but the borderless style you use for this makes it so no keyboard input is recognized! If you press escape, the system just beeps and the window doesn't catch an event. :o There's also a new full-screen mode for apps in OSX Lion, but it's for GUI apps, and I wanted this to work on Snow Leopard. I knew it was possible because Left 4 Dead 2 on Mac does this. Using the Google, I was finally able to find the information I needed. I had to create a new subclasses NSWindow class to get keyboard input working. Another problem I had was that when a window was resized, the NSOpenGLContext wasn't sizing with it. Many, many search queries later I found what I needed, and added a couple of methods to my OpenGL view class. Objective-C may be the greatest thing in the world for writing GUI applications for Mac, but I found it very complex for simply creating a window. It takes about 700 lines of code to create a window that has OpenGL rendering and keyboard and mouse input. Out of all that code, you guys end up with one or two functions that do exactly what you need to create a window and get events. B) Let's talk events. Event queues are the most compatible with all languages, but a callback is really better. When a window is being resized, the application loses control and doesn't get it back until the user lets up the mouse. That means an event queue system doesn't allow the window to be re-rendered as it is sized, and you get ugly artifacts as it is sizing. Therefore, I think we need a hook system where you can add your own hooks like this: System::AddHook(int hookid, char* funcptr, Object* extra) System::AddHook(HOOK_EVENT, myfunction, NULL) I think we're also going to set up the App class so that it has a ProcessEvent method, which automatically gets added as a hook: App::ProcessEvent(int eventid, int data, iVec2 position) { switch (eventid) { case EVENT_WINDOWSIZE: world->Render(); context->Sync(); } } Your thoughts?

Eclipse is very slow to compile C++ code. I recommend developing your program in Visual Studio or Xcode, and then just using Eclipse to build when you are ready to run on Android. The rest of it went over my head.

Well, we're through the "OMG will this even work" phase. Aria got FreeType to build on all platforms, so you can load TTF files directly on Windows, Mac, Android, and iOS devices. I set up the accelerometer input on iOS today, so you can call Device::GetAcceleration() and retrieve a Vec3 telling you which way is down. Acceleration is returned relative to the screen orientation, so you don't have to worry about which way the device is rotated. This isn't my proudest screenshot, but it demonstrates that it works. Gravity is pointing in the negative Y direction, relative to the landscape screen orientation. Multitouch is working on iOS and Android. I am not 100% happy with the way events are handled at this point, but the hard parts are done. I also got cross-platform threads working. I like the Windows implementation of threads better than POSIX, but our Thread and Mutex classes work the same on everything. I use these internally to optimize routines in the engine, and you are welcome to try them, but be warned! Multithreaded programming is not for the faint of heart, and there's a lot of ways you can hurt yourself! I generally avoid use of mutexes at all, and just try to design my routines so that I send a bunch of data to get processed, then get the results. Otherwise it's just too complicated. I had an interesting discussion last week with one developer here, and it led to the decision to start using static class functions for creation and loading commands. For example, instead of CreateThread(), you will call Thread::Create() in C++, Lua, and C#. This gives us a uniform API across all three languages, and it resolves the problems that arise from C#'s lack of global functions. The goal right now is to get all the platform-specific stuff totally resolved and 100% tested so that we can move on and be totally platform-agnostic. That means from now on I can just write code without worrying about whether it's running on Android, Mac, Windows, or whatever. Not much else to say at the moment. Here's a screenshot of the code log to show what we've been working on:

That's why I like Xcode.

Consider it done.

It just depends on the hardware. There is no engine limitation.

I don't know what happened to your post. I actually have a game pad I bought in order to implement this. There is existing code on this forum to do it with C++ or BMX in Leadwerks Engine, and Leadwerks3D will have it built-in.

I'll add this.

Yep. Visual and physical geometry have nothing to do with one another. AFAIK Bullet is one guy.

I implemented direct WAV file loading, and will add OGG as well. The MP3 file format has licensing issues, and we are not going to support it. However, if you use a third-party MP3 converter, the editor will detect whatever converters you add and use them (see more below). We're using OpenAL for sound, but have a more in-depth implementation than in the past. Sound source management is important because you only get so many sound channels on most cards, and this is handled automatically, based on sound range. It's actually quite elegant because sounds are stored in the world octree, and only sounds that intersect the listener are evaluated. This makes it very very fast, and sounds can have any arbitrary range value. You can also get the length of a sample and skip to a certain time. I decided it was important to be able to load TTF files directly. The FreeType library is quite large and isn't set up to compile for ARM processors. Aria has been working on that, which is good because dealing with compiling issues and other people's code is not my strength. I just got word that it was working on the last platform, so I will check it out tomorrow. The direct loading of WAV, OGG, and TTF files got me thinking about textures...could we justifiably load images files directly, instead of automatically converting them in the background? I wrote a BMP texture loader, and although I got the loading and mipmap generation down to 28 milliseconds for a 1024x1024 BGR image, I don't think it's the right way to do things. Textures and models are complex files, and it's best to have the editor automatically convert them to our own TEX texture format when a change on the hard drive is detected. The only disadvantage is that if you are coding and working in Photoshop, you have to either leave the editor running or open and close it, if you want your TEX files updated. Or you could drag the image file onto the appropriate converter, but we know from experience that is unacceptable. I think automatic conversion of these files is best because: Not all conversion code can be included in the engine library. The Autodesk FBX SDK, for example, isn't even available for ARM systems. Some libraries like FreeImage, which is needed to load PSD files, are enormous. FreeImage compiled as a static library is 17 mb! That is way too much bloat for a mobile app. Automatic conversion means you can add support for new file formats just by dropping a converter in the "Tools" folder. I want to show you how the directories are laid out in the Leadwerks3D installation folder. Instead of switching your asset path around in the options whenever you want to load assets from a directory, we're going to use a concept of "projects". When you select a project, Leadwerks3D starts loading all assets from that directory, and sets the project's "Maps" folder as the default open/save directory for maps. When you are ready to make a distributable game, Leadwerks3D will read all the maps and scan source code files for asset files, and copy only the asset files you use into your final distributable game. Root directory: Templates for all supported languages: A sample C++ project: C++ and C# project files for Visual Studio, Xcode, and Eclipse are stored in a "Build" directory, each with a subfolder for the platform. This is how we manage the Leadwerks3D source code, because all those compilers can create a ton of extra files you wouldn't want residing in your main project folder: Finally, I started the OpenGL2 renderer tonight, which is aimed to be identical to the OpenGLES2 renderer. This is my primary focus for now, as it is important to have one uniform renderer that looks the same on all platforms. Of course advanced graphics is our specialty, and I have already done all the research for new effects when I finish the OpenGL 3.2 and 4 renderers. Deferred rendering will work on OSX Lion, multisampled deferred rendering is already working on Windows, and other stuff like per-pixel motion blur is done. We're nearly to the point of having a usable game programming library for four platforms, and only have one more library to compile for mobile. It's coming along nicely! B)

It's done like this in Lua: test --[[ test test test ]]-- test

It works here.

I would, but that involves another file format change. If I knew that was going to happen, I would not have relied on it in the first place.

Please post your files so we can try ourselves.