Acorn Heroes

In my previous articles, we’ve looked at how we can set up an automatic build, with a focus on catching issues with bad check-ins immediately.  Another useful aspect of an automated build is that it can remove the chance of human error whenever we have a task that consists of a lot of manual steps.

One such example is generating a build for ad-hoc distribution to beta testers.  We could create a build by hand, but if we forget to increment the version number in our project, then our testers won’t be able to update from a previous build unless they delete the old build by hand off their devices and iTunes – definitely a pain for people you don’t want to inconvenience.

Also, imagine the scenario in a couple of weeks when a tester notifies you of a serious bug they’ve found.  Chances are the code you have now is quite different to the build they have.  Can you get back to that point in the code easily?

Fortunately Apple provide us with a handy tool called agvtool that can handle these situations for us.  It’s a command line tool, so it integrates nicely into Hudson.  But first let’s look at agvtool on its own, and see what we can do with it.

agvtool

In order for agvtool to do its magic, we have a few things to set up in our project first.  Most of the steps here are described in detail by Jamie Montgomerie on his blog.  Briefly though, we will use two different version numbers – a release version and a build number.  The release version is our “marketing” number – what our clients see (“Now updated to version 1.2!”).  Our build number is for internal use, and will connect a given instance of our App to a specific tagged version of our code in source control. To test this, I created a simple OpenGL project in Xcode and added it to SVN as a starting point.  The only modification I’ve made to it is to add a label on top of the GL view.  We will use this later to display our build number for testers.  Note that SVN is not necessary for using agvtool, but it will give us a few nice options down the track.

So, following Jamie’s instructions, we:

1. Open our project’s PROJECT_NAME-Info.plist file and set the Bundle version to 1.  This represents our build number.
2. Ctrl (or right) click on the plist and choose Add Row.  From the options presented, select Bundle versions string, short and set its initial value to (say) 0.1.  This is our release version.
3. Ctrl (or right) click on the project in the Groups & Files window, and select Get Info.  Make sure you’ve selected All Configurations and find the Versioning section, near the bottom.  Set the Current Project Version to match you build number – 1 in our case.  Set the Versioning system to apple-generic.

OK, not too painful and we’ll never need to touch these bits again!  Let’s test our a few things.  Open up a terminal window and cd to your project directory.  We can use agvtool to tell us both the current build and release version numbers (agvtool calls them version and marketing version respectively):

You should see the build number (agvtool what-version) and release version (agvtool what-marketing-version) that you entered into the project settings.  Now, using agvtool we can also update our build and release version numbers.  Note that doing this modifies your project, so ensure it’s either not open in XCode or everything has been saved – otherwise you may lose changes to your project such as settings or newly added files. So, to increment our build number we use:

agvtool bump -all

On the slightly rarer occasion that we want to adjust our release version, this can be done as follows:

agvtool new-marketing-version 0.21b

You can use the agvtool what- command to verify what’s going on here.  Note that our build number is automatically incremented by one using the bump command.  Our release version is in fact a string, and we can supply whatever we want there, in this case 0.21b.

Assuming you’re paranoid like me, now is a good time to check this into source control as we have all the basics covered.

Open the project in XCode and build it.  XCode will now automatically generate a .c file as part of the build.  In my case, these were found at:

build/AutoVersionTest.build/Debug-iphonesimulator/AutoVersionTest.build/DerivedSources/AutoVersionTest_vers.c

You don’t need to even open this file, or include them in your project (that happens automatically).  This file declares two variables, a version string and number that correspond to our build number.  We can put them to good use.  In my sample project, I’ve declared the build number as an external variable in my App delegate:

extern const double AutoVersionTestVersionNumber;

And then in my application: didFinishLaunchingWithOptions method I’ve used the build number on the label I added earlier:

versionLabel.text = [NSString stringWithFormat:@"Build: %1.0f",AutoVersionTestVersionNumber, nil];

Now when testers run my App they’ll see something like this:

Now, a tester can easily identify the build they’re running when giving me feedback.  By the way – if anyone know an easy way to get at our release version number in code, please let me know!  What we need now is a simple way to tie that back to a specific version of our code…

Tagging in SVN

Note, this also works with CVS as well, however while there are good arguments for not moving to a more modern system like Git or Mercurial, there’s no good argument for sticking with CVS.

agvtool has the ability to update our version information and then commit the changes, simply by adding the usesvn tag like so:

agvtool -usesvn bump -all

Nice.  But it gets better.  We can then tag these changes to make getting back to them simple:

agvtool -usesvn tag -baseurlfortag http://URL/to/svn/AutoVersionTest/tags

The -baseurlfortag option specifies where the tag should go.  After this operation, you’re SVN repository should look something like this:

Now, when you receive feedback from a tester it’s a simple matter to check out the tagged code and you can track down issues with the same code base your tester was using.  Note that the folder specified by the -baseurlfortag option must already exist – agvtool will not create it for you.

And Hudson?

Hopefully it’s not too much of a stretch from here to see how we can use this in Hudson (or any other automated build system).  First off, we should create a project in Hudson that will check out our code and build it.  If you already have an existing Hudson project that builds periodically or with each new check-in you can simply clone it (Hudson is good that way).  Configure the project to only build manually by unchecking all the build triggers:

And add in new build step(s), calling agvtool to bump our build number and (if using SVN) tag the resulting build.  Note that if this project were set up to trigger a new build when check-ins are detected we have a potential race condition, with agvtool checking in changes which would trigger a new build which would make agvtool check in changes….

So, with these changes in place we can now reliably build a version of our code for testers, tag it and know that we can come back to the code for that build at any time.  And it’s all done automatically.  Also our version numbers will always increase, insuring that our testers don’t have trouble updating to a newer build.

And Another Thing

OK, I haven’t played with this one extensively yet, but will add it here for your consideration.  Hudson allows us to add parameters to a build.  When we start a build, it will ask us to supply parameters that affect the build.  As an example we can add a BuildNumber parameter that will decide which tagged version of our code we want.  Configuring it in Hudson will look something like this:

So what’s the use of this?  Well the parameter becomes an environment variable when Hudson is building the project.  So we can slip the environment variable into our source control link.  This way, when we kick off a build we can specify the tag to use, making it easy to recreate our App as it was at any stage in its development.

Well that’s it for this week.  If you have any questions or feedback please leave a comment below.

This post is part of iDevBlogADay, a group of indie iPhone development blogs featuring two posts per day. You can keep up with iDevBlogADay through the web site, RSS feed, or Twitter.

After last week’s introduction, it’s time to get on with making this thing.  After looking at both CruiseControl and Hudson, I went with Hudson – mainly because it’s very simple to set up and start using straight away.  So without further fuss, let’s download Hudson from here.  I created a build folder ~/AutomatedBuilds (there’s nothing special about this location, put it wherever suits you) and put the downloaded file hudson.war.zip in it.

Open a terminal window and cd to the build folder.  The online documentation says that you should unzip hudson and run it using the command:

java -jar hudson.war

This works fine on Windows, however under OS X, unzipping the file and trying to run it causes the following error:

If anyone can explain the issue, I’d appreciate the insight.  However it turns out we can just run Hudson as-is out of the zip file:

Ok, so what’s happened?  Hudson is now running as a Java application in the terminal.  As well as running the build, it also provides a web interface for us to work with.  Open a browser and point it at http://localhost:8080/ and you should see something like:

Before we set up our first job, we should grab a couple of handy plugins first.  Click on the Manage Hudson link on the left, then Manage Plugins and finally the Available tab.  You should see a long, slightly exciting (or intimidating, depending on your outlook on life) list of plugins.  Have a look through the list – there’s lots here to play with – but for now select the Twitter and Mercurial plugins (assuming I’ve convinced you to try Mercurial), and click on the Install button.

You should see the plugins being downloaded and installed, along with a message to restart Hudson.  When Hudson is running, there’s a Prepare for Shutdown option on the Manage Hudson page.  For now though, it’s enough to go to the Terminal window you’re running Hudson in and kill it with Ctrl+C (I’m a bit old fashioned that way).

Run Hudson again from the terminal window, and we’re ready to set up our first automated build.  For a start, let’s create a build that monitors our source control repository and kicks off a build when it sees any changes.

On the main Hudson dashboard, click New Job.  Give the job a name, choose the “free-style software project” option and click OK.

On the resulting page you can configure a bunch of options.  Feel free to go nuts here.  Each option has a ‘?’ icon that provides useful help.  Here’s the settings I used:

Under Source Code Management, I chose Mercurial and pointed it at my Fogbugz Repository.  The Fogbugz repository requires the user to be authenticated, so we do this by including the user name and password in the URL in the form:

https://USER_NAME:PASSWORD@PATH_TO_REPOSITORY

Note that, if your user name has a ‘@’ symbol in it, replace the ‘@’ with ‘%40′ to avoid the URL being incorrectly parsed by Hudson or Fogbugz (see the screen shot below).

Hudson also supports many other types of source control, pick the one that you use.  Another small note here – when setting up an automated build for the first time, it’s a good idea to use a small test project first, to allow faster testing of your set up.  My actual production code can take many minutes to build (not my fault! OpenFeint is slow to build), so instead, for this article, I’ve created AwesomeGame from one of the standard XCode templates.  Once I have a running build system, I’ll then go back and add in my production code.

Next up, we want to check if we need to trigger a build by polling SCM for changes.  We schedule Hudson to check SCM every minute using CRON-like syntax :

Then, we add the actual build step, executing a shell command to run XCode from the command line using xcodebuild.  Have a look at the xcodebuild man page for all the various options.  For now, we want to build our AwesomeGame target in both Debug and Release modes.  I’ve done this below using two build steps:

Finally, let’s tell Hudson how to notify us when the build is done.  I selected email and Twitter notifications.  If you’re on a team, notice the option to notify the individual who committed the changes that broke the build – without having to notify the entire team.  As well as the person responsible for the break it’s good practice to have a nominated ‘build monkey’ whose job it is to check all builds – so make sure they’re getting all the build results as well.   A quick note on Twitter here – don’t use your own Twitter account unless you want to spam all your followers!  Instead, create a bogus build account for Hudson to use, and follow it from your main account.

Assuming we’re all happy with the settings (we can always come back and tweak them), click Save.

Hudson now takes us to the page for our project.  On the left is a Build Now option – let’s try it out…  After a few seconds, you should see a progress bar in the bottom left, telling you that Hudson is building your project.  If nothing happens, check the top right of the page for an Enable auto-refresh option.

Once finished, we’ll see a blue ball that indicates a successful build (your code in source control does build, right?).  If instead you see a red icon, then the build has broken, and we need to fix something.  Either way, click on the most recent build link to see details of the build:

On this page, we can see the status of this particular build (remember blue ball means success), the changes in source control from the last build (none in this case, as we manually triggered the build) and the console output, where we can see the details of the code being checked out from source control and built using XCode.  If your build has failed, this is a good place to look for the problem.

Phew!  The system works!  Checking Twitter, sure enough there’s a success message there:

But hang on, where’s my email?  The problem here is that we need to configure Hudson with an smtp server so that it can deliver mail to us.  Go back to the Manage Hudson page, and click on Configure System.  Near the bottom you can configure the global settings for email and Twitter.  If you’re unsure what smtp server to use, check the settings for your email program – you’re looking for the name of your outgoing smtp server.

So enter in your smtp details, and the system admin email address (that’s you).  Click on the Test button and make sure that you get the test email message.

Going back to the main Hudson page, you should notice that our project is listed, and on the right hand side is a Build Now icon (so you don’t have to go into the project to kick off a build).  Click it to start another build, and then click on your project to follow the build’s progress…  Still no email.  Bugger.  Going to the project’s Configure screen, I look up the help for email, which sheds some light on the issue:

This is sensible – no point in spamming out emails when builds are all going well.

So by now we know that we can check out our code and build it automatically.  But where is the build?  Hudson creates a working folder in your root directory.  Have a look in ~/.hudson, and you’ll see all of Hudson’s internals.  In particular, the folder ~/.hudson/jobs/Awesome Game contains all of our build history and a workspace folder in which the code is checked out and built.  So if I want to get the built app, I can grab it from:

~/.hudson/jobs/Amazing%20Game/workspace/build/Release-iphoneos/

Later we’ll look at packaging up the build and putting it somewhere handy, but for now, let’s make sure that it’ll pick up on changes in source control.  Check out a working copy (if you don’t have one) of your project and put in a small change that will cause the build to break.  This ought to do it:

Commit the change (or commit and push if you’re using a remote Mercurial server like I am) and wait for a minute.  You may notice a message mentioning a ‘quiet period’ before the build kicks off properly.  This quiet period is Hudson waiting after it detects a change in source control to make sure all changes have been committed – this is particularly useful with CVS, where every file is committed separately.

Ah, BOOM!  Our build has failed nicely.  And I got an email this time, which is nice  A broken build is  something that shouldn’t be left broken, so let’s fix it and check the build comes back happily.  Fix the code, push, commit, wait… success!  Twitter and email confirm everything’s OK.

We’ve covered quite a bit so far so it’s probably a good time to take a break and see what we’ve achieved, and how that matches our goals from last week.

What works?

• Monitor source control (Mercurial and Subversion in my case), kicking off a build whenever changes are checked in.
• We’ll be able to manually kick off a build as needed.
• The build will grab all our source code from source control and build it in all relevant configurations – debug, release, simulator, device and so on.
• When the build is finished, it’ll notify people of the success / failure.
• There’ll be a handy location (web page most likely) where we can monitor / control the whole thing.

What do we still need to do?

• We’ll schedule a nightly build as well, just in case anything outside of source control has changed. This is fairly simple – an exercise for the reader!
• Copy a successfully built app to somewhere useful, preferably packaging it up with a provisioning profile in a zip archive, ready for emailing.
• Automatically update version numbers on our ‘stable’ build with each build.
• Bundle up the resulting builds with a provisioning profile and copy them to a shared folder.
• Start up automatically under when your build machine is booted.
• Paint a unicorn.

OK, plenty to work on, I’ll be back next week with some more detail, or if you’re keen have a crack yourself and let me know how you get on!

Disclaimer: I’ll let you in on a secret now, to avoid disappointment later on.  By the end of this article you won’t have a fully working automated build  The reason is that I was half way through writing up this article using CruiseControl when I started to look at Hudson, which seems like it might be promising.  So this week is an introduction to the idea of continuous integration and automated builds and how they can save you time and grief.  Next week I’ll aim to have a useful build system up and running along with instructions on how to set it up…

What?  Why?

An automated build is a system that can build your software from source to finished product without human intervention.  At it’s simplest, this can be an XCode project or a makefile – after all, you don’t compile each file and link them together by hand, do you?  The key idea is that we can remove human error from the process, making it more efficient.  The ideal is to have a system whereby you can build all versions of your software with a single click (and also score a couple of points on the Joel test).

Consider the following situation.  You want a fresh build of the code to show off to your boss, “Reckless” Jim, to keep him off your back.  When you wake up, you grab your iPhone 4 (lucky bastard), open Safari and click a link on a web page before you get in the shower (remember to put the iPhone down first).  While you’re halfway through your hot water supply (and the Sound of Music soundtrack), the following is happening:

Cool, huh?

Or how about this:

Neat, eh?

This second example is what’s known as continuous integration – every time you check in new code to source control, the whole build is exercised to make sure nothing’s broken.  Why?  Often a broken build is a simple 30 second fix at the time, but 2 weeks later it might require hours of sorting through commit logs to see where the problem is.

The solution

So, what does it take to set up a system like this?  Not as much as you might think.  There are some neat tools out there to get you up and running fairly quickly.  The problem is that I didn’t have time to try out both of the likely candidates for this week’s deadline.  I’ve used CruiseControl before, and Hudson looks promising.

So here’s the goal for next week.  I’ll show you how to set up a system that will:

In the mean time here’s a couple of sample chapters from Pragmatic Project Automation: How to Build, Deploy, and Monitor Java Applications that covers the basics of setting up CruiseControl.

So, now to decide between Hudson and CruiseControl.  If you have any thoughts on either of these systems (or another I haven’t heard of), please let me know in the comments below…

Now that the initial excitement of getting an App on the store is starting to fade, it seems like a good time to look back over Goo!‘s development history and figure out what worked and what didn’t.

The Good

Sales:

It may seem odd to put this under the ‘good’ category, as our sales figures are still in double digits (although we’re hoping for a roll over into three digits before long).  However the purpose of Goo! was to be a test case, dipping our toes in the AppStore’s waters.  As such, it’s a success.  There’s a wonderful buzz from selling your own software that I had never experienced before.  Despite developing software for over ten years now, it’s always been for someone else.  The thought that other people, around the world have seen my software and wanted to buy it is thrilling.  And the feeling you get when one of your friends greets you by waving their iPhone in your face with your App on it is pretty special – thanks Chris, you’re an awesome friend.

Cocos 2d:

After faffing about with raw OpenGL for a while, we eventually switched to using Cocos2d as a platform.  Although both Sam and I are familiar with writing OpenGL engines, it just soaked up our time – something we have little enough of.  Cocos2d worked a treat and the change over took only an afternoon’s effort.

Twitter:

Twitter has proved to be a real find for me.  The iPhone developers I follow are a friendly, helpful bunch.  I’ve learnt plenty from these guys, found lots of great articles on the Internet and when we released Goo!, lots of people re-tweeted the announcement or promptly bought it on the spot.  Thanks guys, it means a lot!

Working together:

Sam and I have been working together on and off for over ten years now.  Ideas pass back and forward freely, and there’s enough mutual respect that we can quickly throw away bad ideas, or modify them to become good ideas with no ego or resentment getting in the way.

Submission process:

We hit a lucky patch with the App Store – just before Christmas people started reporting quick turn around with App submission.  Sure enough, Goo! went through the approval process smoothly in three days – a very pleasant surprise.  In fact, after hearing horror stories from developers over the last year, it is nice to see that Apple is improving things (or were we just lucky?).

The Bad

Too long:

Yeah, this is the biggie.  Nine months is a ridiculous length of time.  No excuses here.  Too often real life got in the way.  It’s just a fact of the lives we live with full time jobs and families.  Also…

Lack of a core concept:

In it’s day, Goo! has been a time tracker, the beginnings of a series of articles, a set of particle system toys and finally an implementation of the Game of Life.  Perhaps now you understand why it took so long.  Sam and I wanted a simple project to get started and we exercised some poor judgement along the way.  A lack of a clear vision for Goo! hampered us a lot.

Lack of a forward delete key:

Seriously, this is the single biggest pain I have working on a laptop.  And yes, I know about fn-delete, but it’s just not the same.

So, in summary it’s been an enjoyable process.  Goo!, while simple, is an App that we’re happy to have on the store.  The idea of selling our work on the App Store now feels much more achievable and we’e both brimming with new ideas, at least one of which is showing a lot of promise – and as far as I know is a completely new idea.  Of course the challenge is in turning a good idea into a successful project – watch this space.

A question popped up recently on the TouchArcade forums, about how many testers is the ‘best’ number to have.  At first it seems like more is better, but in fact that’s not always the case.

In terms of numbers, you’ll get diminishing returns on adding more testers. Say each tester finds on average 10 bugs / issues. With one tester, you’ve found 10 bugs. With two testers it may be, say, 15 bugs due to overlaps in what they find (i.e. they both report some of the same bugs). Three testers may find a total of 18 bugs and so on.

Very quickly you reach a point where your chances of getting something new from an additional tester become slim.  But it gets worse than that – you’re actually polluting your pool of fresh testers who have never played your game before.  These people are valuable, as they give you an insight into those vital, impression forming, first few moments a customer has with your product.

So small groups can be surprisingly effective. Start with a small group of, say, 5 testers. Get their feedback, make adjustments and then release an updated test version to a new set of five testers, plus the original group of testers.  The ‘fresh meat’ will help you avoid tunnel vision, giving you the 50,000 ft view, while the repeat testers can give detailed feedback on particular features.

Repeat until you’re happy, or at least until you’re satisfied your product has reached the level of quality you’re aiming for.

Risk Driven Development?  No, don’t leave, I’m not completely crazy.  Just hear me out…  It’ll make sense before too long.

Every project carries with it risks, for example:

• Can we fit the all the art assets and binaries into a 10Mb download?
• Is it possible for our sound track to change based on what’s happening in the game?
• Will people empathise with our main character?
• Is the game we’re making any fun?

If these risks aren’t addressed, there is a very real chance that one of them will bite you in the arse.  The longer a problem like this goes unnoticed, the harder it is to fix.  So we have to find these problems as soon as we can, and deal to them.

As one of my friends put it once, “I like to do my worrying early”.

It’s something that you can do at every stage of devlopment.  Step away from the keyboard, put the mouse down and take a look at the big picture.

Are there parts of the code that you’ve been avoiding because they scare you a bit?  When was the last time you ran the game on another machine or another device?  When was the last time you put a build in front of anyone else?  If the answer to any of these questions is “It’s not ready for that yet”, you may be suffering from tunnel vision and heading for trouble.

Hence “Risk Driven Development”.  Identify the greatest risks affecting your project and tackle them straight away.

• Rapid prototyping is a great way to prove your basic game design is sound.
• Generate rough assets for an entire level, or estimate what you’ll need, and do a back of an envelope calculation to see if you can fit in those 32 levels.
• Build a quick test harness for interactively blending sound tracks into each other to ensure it can be done.
• Make concept art for the main character, show it to people and listen to their feedback.

Be prepared to make design changes based on the results of these tests.

Do it now before your game is too mediocre to save.

A craftsman's tool

I’ve spent a large chunk of today splitting wood to stack away for winter fires.  This may seem like a simple, manual process, but when you see what an experienced person can do with an axe, you quickly begin to appreciate the skill involved.  

Now I’m no expert, but I can appreciate the art of picking the right place to attack with the axe.  Get it wrong, and you feel impotent as the axe bounces off the log a if it was made of rubber.  Get it right, and you feel like a God among men.  Or something.

In the previous post I mentioned that to make progress on our iPhone projects in spare time between day jobs and families, we need to work smarter, not harder.  It’s much like splitting wood.  Picking what and where to attack are vital skills to possess.

Don’t bite off more than you can chew.  I don’t think there’s a developer alive who doesn’t have dreams of crafting the ultimate RPG, the perfect 4X game or recreate Elite on modern hardware with an infinite universe of unbounded opportunity.  But there’s a reason that these games have huge budgets and large teams working on them.  To be effective as a one or two man team, you need to pick your project’s wisely – aim for something achievable or you risk giving up in despair.  This needn’t be a limitation, it’s a chance to think outside the box.

Be ruthless, trim away the fat.  Keep the feature list tight.  This doesn’t mean selling your game short.  It means picking a reasonable feature set that you can polish to a high level, rather than doing twice as many things to a rough and buggy level.  Some of the best games have been built on this principle.  Take Diablo for example.  You can write the details of the gameplay on half a sheet of paper.  Tetris.  Could it be any simpler?  Never confuse simple with bad.

Get out of your ivory tower.  The only way to truly know the worth of your idea is to put it in front of genuine users.  Do it as soon as you can.  And do it often.  A few moments with a friend over coffee might save you trying out a bad idea.  As a developer it’s easy to get tunnel vision, and not see the bigger picture.  Let a fresh pair of eyes, and a potential user of your software give you insight into the good and the bad of your project.

Min / Max.  It may seem mercenary, and uncreative, but pick the simplest things that add maximum value to the project.  Sure, you can add that really hard to implement feature if you need to, but make sure it’s worth it.  Is it a unique selling point?  Something that no one else has done?  If not, is it worth the effort?

Lists. Whether it’s a project plan, or a to do list for the day.  Keep lists, they’re perhaps the single best way to stay focused.  And that’s ultimately the most important thing.

A sharp axe is good, but putting it in the right place is the real craft.