Kinematic's Custom Engine
When aspiring game engineers have asked me for advice on how to improve their skills, my go-to advice is "make a game from scratch". That is, don't use Unity, Unreal, Godot, etc. Not because those engines are bad; nothing could be further from the truth. But there's so much to be learned from trying to build the entire game yourself, rather than relying on a component system, game loop, or physics engine that someone else wrote. Having given this advice so many times, I realized it was pretty hypocritical of me to have never done it myself. Therefore, in 2018, when I started working on a hobby project - a tower defense / incremental game - I decided to build the game from scratch in C++.
That project, Eternity TD, didn't come to fruition. I built a fair amount of code, but ran out of steam long before finishing the project. But that code formed a natural starting point when I left AAA development to work full-time on indie development. This post is going to walk through how some of the characteristics of that early tower defense work has influenced the engine for my forthcoming 2D indie game, Kinematic.
The core technical challenge of making an incremental (aka idle) tower defense game is how to simulate what would have occurred while you weren't playing. I'm dissatisfied with the two solutions I've seen (require you to keep the game running at all times, or require you to keep it running while you clear waves, then earn income based on which wave you've cleared). I wanted to make a game with the typical idle game core loop of fire the game up, poke it for a couple minutes, then close it again for a few hours. Thus I needed a way to be able to rapidly simulate the effect of the changes you've made to your defense. I decided to make it possible to have multiple instances of the game running simultaneously on different threads. Thus you could start to play normally at 60 FPS on the main thread, while the update thread goes off at ~1000 FPS to figure out what a typical ~10 minutes of play would have looked like, and extrapolate the result for however many hours you've been away. I could even mask this delayed computation of the results with little e.g. "+500 gold" popups streaming in while the extrapolation is being computed.
In order for this scheme to work, I needed all of the mutable game state to be contained within a knowable blob of memory. Read-only Definition data can be shared by all running instances, but each game needs to be isolated from all others. I decided to make a monolithic Game class that contains all of the game state within itself.
This yielded a nice benefit of allowing a very simple triple buffering scheme for rendering. After every game update, I lock a mutex and copy the main game into a buffer game. Then, after every render, if the buffer game has changed, I lock the mutex and copy the buffer into the render game.
On the other hand, this prevents the use of dynamically allocated memory at runtime. If you memcpy a pointer (or an STL container that implicitly contains a pointer), multiple Game instances would then share the same memory. It also prevents the storage of pointers to anything except read-only, static memory. All references to other elements of game state need to be retrieved from IDs when they're being used. Obviously we can pass pointers/references between functions to avoid having to re-find things. But we can't save those pointers/references in memory, as that would also lead to multiple games sharing writable data.
Entity Component System
I've been a big fan of Data-Oriented Design ever since I saw Mike Acton's 2014 CppCon talk linked by Jonathan Blow. Having read Data-Oriented Design by Richard Fabian, I decided to try out an Entity Component System for Kinematic. I got rid of my old object hierarchy, and converted entities into components in various arrays that share an instance of my ID class. That class is pretty simple. A uint32 that contains a value that is ++ed every time a new entity is created. And a pointer to a statically allocated string that describes what the ID is for (for debugging purposes). Typically that debug string is the name of the Definition data that backs the entity up, or a static const char* from the code that created it.
How do I store the components in my weird memory setup? The last member of the Game class is an instance of my Comps class. The last member of that, in turn, is a big ol block of memory into which arrays of components can be stored. I put that member last so that when I memcpy the Game class, I can omit whatever portion of the memory isn't in use. Prior to the block of memory, I have an integer for each Comp class that says the offset from that integer to the start of the Comp array. Thus, when I memcpy to another Game, I can still find any given set of Comps, as the integer acts as a relative pointer. My natvis file gives me a human-readable interface to be able to easily inspect the contents of each array. (Seriously, if you use Visual Studio, go learn about natvis files)
If you're paying close attention, you're probably wondering WTF is up with CompTypeList.h. I certainly was the first time I saw this pattern. (If you're not wondering WTF, do you know what this technique is called? I can't figure out how to google it)
Basically, there's a list of types that I want to have a bunch of basically identical code, but for each of the types. Thus I can use macros to define what to do with a given type, then include the list to have that code exist for each type. I include CompTypeList.h 10 times in various places in the engine. It's similar to templating, but affording you a much larger shotgun to theoretically not shoot yourself in the foot with. Fortunately both Visual Studio and VS Code are capable of auto-completing the generated functions.
I'd love to hear what you're interested in knowing more about. I'll be talking more about the game in coming months, as it becomes less janky and more fleshed out. But I'm happy to talk at length about the engine, my tools, workflows, etc. in the meantime.
Join the discussion on Reddit!