Dev Blog 3 – more build system fun
Now that I have the startup flow working and the first pass on the UI / front end system for the game, I’m back to looking a bit at the build system again.
Note that even though this is a 1-person project, I have a fully automated build pipeline created for the project. I don’t want to have to worry about creating builds to get onto my phone (or potential testers devices) – so it’s important to get a pipeline up and running as soon as possible.
Let’s take a look at the build pipeline that I have created, and the various build projects that it involves.
- Gitlab – I use Gitlab for all of my personal source repositories for several reasons. First – it has 10Gb free private repo’s – MUCH larger than something like Bitbucket, and infinitely better than Github. It also has LFS support, which let’s Git play nicer with big (larger than 10Mb) files – this solves a huge potential issue when using Git for #gamedev. If things get serious enough, they also have a free open-source implementation that I can set up locally if I wanted (or on a server somewhere) should I want to have a personal / private git server.
- uTomate – incredibly useful automation utility for Unity 3d. Lets me ‘script’ my entire build pipeline using a visual flowcharting system. It’s what I used at Namco Bandai to set up the Unity build pipeline – and was very useful. Extending it with custom methods is very simple and having everything in a nice visual flowchart makes things nice and clean.
- TeamCity – My preferred continuous integration system. Jenkins is also popular (mainly because it’s free), but I find TeamCity so much nicer to use. Note that for Unity builds, I do NOT use the ‘Unity Runner’ plugin that ships with TeamCity. Every time I’ve tried to get it working has been a failure – and setting up Unity as a simple command line batch build is easy enough to do. At Namco Bandai & at Roadhouse we use(d) Jenkins for our CI Server, but I’ve always found it a bit flaky. I also really like the built-in Nuget server that TeamCity provides, which comes in handy for more complicated build pipelines. If you have more than 20 build configurations, TeamCity costs money, but for a small 1-person shop like me, it’s effectively free – which is always nice ;}
- HockeyApp – You can think of HockeyApp as a private Google Play (or IOS) store that allows you to distribute test builds to your team and/or testers. You can register any number of test users into Hockey App and they can download & install the builds for testing purposes. There is a free version available that allows you to get up to try it out. It’s well worth investigation – I use it for my single-person projects to get builds from my build server to the devices I’m testing on.
Why Do I Need a Build Pipeline?
Some of you may be asking – why do I need a build pipeline at all?
There are several reasons – and most are fairly important.
1) It forces you to test your game on Device regularly
Unity is an awesome toolset. Being able to launch the game easily is absolutely crucial for rapid iteration and development. However, many developers fall into the trap of either ONLY testing their changes in the Unity Editor or not testing changes on a real target device often enough.
In my case, I’m working on a mobile game – until the game actually works properly ON A MOBILE DEVICE – then I don’t have anything. If there are issues building to the target platform – then I want to know what those issues are AS SOON AS POSSIBLE. If there are issues running or otherwise playing the game on the target platform – I want to know as soon as possible.
I tend to work on my project every evening after work (and after the kids & wife are in bed). By making sure I commit the evening’s changes to Git that evening, I know that TeamCity will run and upload a new build to HockeyApp for me to test on the bus the next morning on the way to work. It’s an awesome system that works very well. Eventually, I’ll (hopefully) have testers that I can use HockeyApp to distribute new builds regularly to them as well.
2) Builds should be reproducible
One of the biggest challenges when creating software is the ‘it works on my machine’ syndrome. Every programmer, ever has run into this problem. When it comes to a project that is expected to be (in theory) up and running for an undefined period live in the market (like a mobile game), you have to treat the code and project like a service from day one. The sooner you can get reproducible, testable builds into your users hands – the better.
By forcing myself to use a build system to create the builds that I test, I can be extra sure of a few things:
- I actually checked everything into the source control system (git)
- Application settings were handled properly (setting dev / prod / testing modes / debug flags etc) – can (and should be) all be automated and handled by the build system to configure the game for the target environment. This includes things like Facebook API Keys, GooglePlay Services API Keys, Analytics keys and so on. Anything that needs to be modified or changed when switching from a development / sandbox testing environment to a live / production environment – should ALL be automated through your build system.
- I actually make a build of the game regularly. I’ve seen situations where a team goes days or weeks (months? [shudder]) between the points of having a testable build that can be run on a device. This is horrifying. At the point when a build fails and can not be run on a device – you should stop everything and FIX THE PROBLEM.
3) You test the whole pipeline / system
As I mentioned above, Unity is great – it lets you get things up and running quickly and easily. However as your project gets more complicated and has more moving parts – it can be easy to fall into the trap of not actually testing all of the components of your game in a real-world production environment often enough.
What do I mean?
- Asset bundles – are you using them? How are you building them? Where are they being deployed to? How do you switch between different versions of your app? Android and IOS require different bundles (as a simple example). Incompatibilities between Unity versions may require you to rebuild the bundles and redeploy to your users. What about cache handling? Cache busting? Versioning of the assets? All of these use cases need to be tested and handled properly well in advance of launch – so you might as well make them a regular part of your build / testing system. If you wait until the end of your project, you are asking for a whole lot of problems.
- Server configuration – if your game requires a server, you probably (hopefully) have a development, staging and live production environments of the server. How are you switching between them in your client? How do you configure switching between these environments?
For the PixelCommando client, I currently have the following target builds and which platform they are targeting:
- Android Client
- Android Asset Bundles
- IOS Client
- IOS Asset Bundles
- Windows PC Asset Bundles
Wait! you say – Windows PC Asset Bundles? What’s up with that? I thought this was a mobile game?
And you would be correct – except that I’m running the Unity editor itself on Windows – and I have the asset bundle pipeline fully integrated into my local development pipeline. So when I modify assets (add new ones, update or modify existing ones), I need to build & deploy the asset bundles for my device target (Android mostly) as well as bundles for the editor as well.
The above are the build projects for the Client – the Server builds are slightly more complicated – and something that I haven’t quite worked out all of the details for yet. I’ll go into those at a later date.
The asset bundle builds run basically the same pipeline as the main Client builds – with one major exception – they will be automatically uploaded to the end delivery location once they are build successfully.
In fact, this is the step of the pipeline that I’m working on currently – handling where the Asset Bundles are uploaded to (Amazon S3 bucket? FTP File server? etc) and how to handle downloading of the bundles to the client. I’ll let you know more when I sort this out!
Til next time!