Shoot The Rocks: tips and tricks for dealing with black holes in my version of Asteroids

There are a few things about the modern variant of Shoot The Rocks that aren’t really reflective of the original arcade games on which it’s based.

Shoot The Rocks is perhaps most similar to Asteroids Deluxe, which is a fairly basic evolution of the original Asteroids. We still don’t have the satellites in Shoot The Rocks yet: I do intend to add them, it’s just a case of finding the time.

The greater use of colour is reminiscent of Space Duel, and the (soon to be implemented) availability of power-ups is somewhat reminiscent of Blasteroids - although my power-ups are very much focussed on fairly simple weapons upgrades only.

I suppose I do like to think that Shoot The Rocks has its own character though, particularly when the modern variant is played. The graphics, whilst similar and very much tipping a hat to the original vector games, are different from the original, as are the game dynamics and some of the features.

For example, something that doesn’t appear in any of the arcade games or their ports, as far as I’m aware, are black holes.

A black hole - admittedly more of a blue hole - shown near the player's ship during a game of Shoot The Rocks. Note the asteroids in close proximity

Note the black hole (yes, admittedly it’s blue, but I felt like it needed to be visible to keep the game fair!) below the player’s ship. Note also that it’s drawn a number of asteroids towards it. It took me several attempts to get this screenshot as my ship kept getting destroyed by asteroids that were being sucked in towards the black hole (see tips below).

These are very much based on the star that appeared in the grandsire of all shooters, Spacewar!, in that they implement a gravity well that influences the movement of all other objects in the game.

They add another dimension to the game, which you can either use to your advantage, or which can be your undoing, depending upon how you deal with them.

So here are some tips and tricks:

  • Your ship is fitted with intertial stabilisers: this is why it will eventually come to a halt when no thrust is applied. It also means that, unlike most other game objects, your ship has some capacity to resist the pull of black holes. Eventually, when that pull becomes too much, your ship will get sucked towards the black hole. You’ll notice that flying saucers are also less affected by black holes than, for example, asteroids, because they too are fitted with inertial stabilisers.

  • Contact with black holes is not in itself fatal, however if you get too close your ship can either be captured (albeit temporarily), or more likely accelerated around the black hole and slingshotted off unpredictably. This can leave you in the direct path of other hazards, such as asteroids or flying saucers, and lead very quickly to your demise.

  • It’s advisable to give black holes a wide birth, both because of their tendency to cause your ship to accelerate unpredictably, and also because they draw other objects - particularly inanimate objects such as asteroids - towards themselves. This means that you can find yourself in the middle of a vortex of coalescing rocks from which you won’t be able to escape, except by using your always risky hyperspace capability.

  • If a black hole is around, having fewer other objects on the screen is generally better due to the tendency for the black hole to slingshot these objects around in unpredictable ways. I.e., try to avoid a situation in which you have shot all the larger asteroids and turned them into a much larger number of smaller asteroids, otherwise you’re probably going to be quickly annihilated in a collision. Similarly, if you only have a few smaller asteroids left at the end of a level, perhaps draw out completion of that level for as long as possible, or avoid shooting too many larger asteroids when the next level begins.

  • If you see lots of asteroids being drawn towards the black hole at once this can be a good opportunity to quickly clean up by shooting rapidly directly at the black hole. Hopefully your shots and the asteroids will be drawn together in the black hole’s gravity well leading to mass destruction. Do keep yourself at a safe distance so you have some chance of avoiding any errant rocks that are thrown out of the gravity well towards you.

Hopefully these tips will help you effectively take advantage of the presence of black holes, without falling foul of their dangers.
Play Shoot The Rocks now to try them out.

Making gravity fun: adding black holes to Shoot The Rocks, my version of Asteroids

Well, it’s been a while (!) since my last blog post here but, you know, life intervenes. Still, I’ve started working on the site and improving the games again.

I had this idea that I wanted to inject a little chaos and variety into Shoot The Rocks by adding black holes with associated gravity effects. This is something the original Asteroids didn’t have, and I don’t believe they’re in Asteroids Deluxe either. Still, I thought they might make for a fun addition.

If you want to jump straight in an see what effect this has all had on the game, just click here to play.

I want the classic version of the game to be quite close to the original arcade game so black holes only appear in the modern game variant. (I’ll probably add satellites to both classic and modern variants though because Shoot The Rocks is more like Asteroids Deluxe than the original Asteroids!)

Whilst black holes didn’t take ages to implement - really only a couple of afternoons - the road wasn’t totally straight. Much of the time was spent tweaking behaviour until it felt right: it’s more important for any game mechanic I add to be fun than it is for it to be realistic.

Gravity is obviously a real phenomenon so I did start off with a fairly realistic model, in the form of Newton’s Law of Universal Gravitation:

\[F = G\frac{m_1m_2}{r^2}\]


  • \(F\) is the force in Newtons
  • \(G\) is the gravitational constant
  • \(m_1\) is the mass of our black hole
  • \(m_2\) is the mass of whatever other game object we’re interested in
  • \(r\) is the distance between the black hole and the other object (Strictly, I believe, it’s the radius of the circle whose origin is the centre of mass of the black hole and whose edge would intersect the centre of mass of the other object, or vice versa. Details, details, etc.)

In simple terms, the gravitational force between two objects:

  • Is proportional to both their masses (actually to the product of their masses)
  • Gets weaker the further apart the objects are: i.e., it’s inversely proportional to the square of the distance between them (known as an inverse square law)

Granted, I could have gone more realistic, but worrying about relativity or quantum gravity in a game based on something as inherently unrealistic as an homage to Atari’s Asteroids is clearly ludicrous, and Newton’s law has the advantage of being simple to get to grips with.

Also, I kind of knew from the outset that any changes to increase the fun factor would most likely to simplifications and corners cut, so it makes sense to start off with something simple to begin with. We need to get to the point where gravity effects feel like they fit in with the rest of the game which, to begin with, has very simple mechanics.

The main things to grasp are that the heavier objects are, the stronger the gravitational force between them, and the closer together they are the stronger the force is. The reverse is also true.

Hopefully that’s fairly intuitive and, as you’ll see, the specifics of the equation aren’t really that important from a playability and fun perspective.

Something else to mention is that I want gravity to affect most, if not all, objects on screen when a black hole is present:

  • The player’s ship
  • Asteroids
  • Bullets
  • Flying saucers (and other enemy craft that we might eventually implement)
  • Particles
  • Any power-up collectables we implement

Let’s get down to business and talk about how I made this work.

Simplification, simplification, simplification

Firstly, we’re going to assume that the mass of every object other than the black hole is 1. This simplifies Newton’s law to:

\[F = G\frac{m_1}{r^2}\]

Secondly, we’re going to assume that the gravitational effect of these objects on the black holes, and on eachother, is zero. I.e., black holes are the only objects that exert any gravitational influence. This massively reduces the number of calculations we need to perform.

Thirdly, as I mentioned, G is the gravitional constant. This is a very small number indeed: approximately \(6.674\times10^{-11} m{^3}\cdot kg^{-1}\cdot s^{-2}\). It’s kind of aggro so we’re just going to throw it away and pretend it never existed, which yields:

\[F = \frac{m_1}{r^2}\]

We’re also going to bin off the concept of force, and substitute acceleration straight in there.

\[A = \frac{m_1}{r^2}\]

And this is really where I started with my implementation. I also precalculated everything and chucked it in a lookup table because, of course, we need to separate acceleration along the x and y axes, which requires some trigonometry.

Still, what I found sticking strictly to the inverse square law from Newton is that the gravitational influence of the black holes fell off too quickly. I could, of course, increase the mass of the black holes, but this led to absolutely out of control acceleration of objects very close to them, which also wasn’t what I wanted.

This meant I tried the following:

\[A = \frac{m_1}{r}\]

I.e., rather than an inverse square law, I simplified to a basic inverse law. Unfortunately this now meant that the gravitational effects were felt too strongly on more distant objects, which in practice was equally undesirable, and felt kind of weird.

In the end, after a bit more fiddling around, I settled on the following:

\[A = \frac{m_1}{r^{1.5}}\]

I.e., an inverse square law is an instance of a more general inverse power law, and what I’ve done here is simply to reduce the power to a level where for our purposes it gives a nice balance of stronger affect at close range versus still having some effect at greater distances.

And that means we now have some sort of pseudo-gravity effect that injects some chaos and fun, whilst also working well with the three sizes of black hole I’ve implemented (small, medium, and large). Large black holes are now just about powerful enough to capture flying saucers, but not powerful enough to permanently capture the player’s ship, which is really what I wanted.

They also exert a pleasing, but not out of control ridiculous influence on all other game objects.

As I write this I’m sat approximately 12 miles from Trinity College in Cambridge where Newton was a fellow. Were he still alive, and aware of what I’ve done here (especially given the computing power available even within the average smartphone), I imagine he might have a few stern words to say to me the next time I venture into town.

Still, if you’re of a more forgiving bent and want to have some fun by playing the game and seeing how this has all worked out then click here to play.


Perceived performance part 2: load assets asynchronously or on-demand

When I started my side-project, for such it is, I had a variety of motivations. One of them was that I was a bit fed up of all these low-rent Flash versions of classic arcade games that I’d loved as a kid tarnishing my nostalgia with their long load times, pages covered with ads (yes, there are ads on this site, because I want it to pay for itself, but I try to keep things tasteful), and generally sub-par experience.

I even felt pretty smug and superior about the idea of building games with HTML5, CSS, and JavaScript that would load instantly, or pretty close to, compared with their bloated Flash equivalents with their interminable loading progress bars. Then, of course, reality came calling, complete with its usual payload of disappointment - something I posted about in a comment on Hacker News. There I identified some fairly obvious reasons why games can take a while to load that go some way beyond “Flash is an awful nineties/noughties horrendo-tech abortion that should be blown into charred corpse fragments with dynamite”:

  • Loading games assets takes time, particularly on slower connections (like mobile)

    • Graphics (background images, sprites, textures, logos, etc.)
    • Sound effects
    • Music
    • Fonts
    • Maps and other metadata (game objects, etc.)
  • Creating procedurally generated content can take a while

Now this is frustrating, because it turns out these are big problems for me in that I have a combination of downloaded assets and procedurally generated content. So how do I get around this?

It’s really a combination of two techniques:

  1. Asynchronous loading of non-essential large assets after DOM content has loaded, and the page has rendered, which happens immediately after.

  2. Immediate feedback to the user that something is happening upon page rendering.

Neither of these results in anything being loaded any quicker, but it does get the loading "out of the way", and at least shows the user that something is going on whilst the procedurally generated content is being created. I’ll talk about (1) in detail below, and address (2) in a follow-up post.

Now both Star Castle and Asteroids are vector based arcade games so in my versions, Star Citadel or Shoot The Rocks, we don’t need to worry about downloading any bitmaps or textures for either. The graphics for all the game objects - spaceships, asteroids, flying saucers, mines, etc. - are specified using a simple vector-based notification as JavaScript objects, so they’re just part of the source code. These specifications are just based on designs I knocked up quickly in OmniGraffle.

Using JavaScript objects is not really super-efficient in terms of space but uglify does a good-enough job of compressing them, and they’re not very large or complex anyway. The player’s ship in Star Castle is 8 blue lines, for crying out loud!

If I did have a lot of bitmaps, textures, pixel art, 3D models, or whatever, I’d probably have to load a lot of them upfront because they’d be needed to actually play the game, and there’s not much avoiding that. The one thing I could do, if I were using different graphics for different levels, for example, is load only a minimal set of graphics up front, and then load the rest on demand later on.

I also don’t have to worry about maps, which are another potentially sizeable asset. Again though, I could load them on demand for each level or area as required rather than all in one go.

This deferred loading is a big advantage over Flash games which, at least in most of the cases I see, tend to load most of their assets in one go upfront. I’m not sure if this is a necessity or just a case of bad architecture. Given my somewhat snarky "low-rent" comment earlier I’d guess at bad architectural choice. I don’t know Flash at all, but it’s quite hard to imagine that Adobe wouldn’t have included some equivalent of JavaScript’s XHR in it somewhere, to allow content to be downloaded in the background.

What I do have in terms of large assets are sound effects and music. These are obviously compressed, and I use MP3s because they have the best across-the-board browser compatibility of any compressed audio format. They also work with both the HTML5 <audo> element, required for IE11 compatibility, and the Web Audio API, which is well supported across other browsers. I’d prefer to use Ogg Vorbis because it’s more efficient but the browser support just isn’t there unfortunately.

Sound effects don’t take up much room, currently weighing in at less than 200K for both games, although that’s set to increase because I haven’t created all the sound effects I need yet.

Sadly music is a different story. Both games use three tracks: one for the title/main menu screen, another whilst playing the game, and a final track for the game over screen. These weigh in at 6-7MB total for each game on desktop, or 4-6MB for the lower quality mono versions I serve to mobile devices and tablets.

BEST PRACTICE: To improve REAL performance of your website across a range of devices you should consider serving more optimised assets to mobile users.

This is obviously problematic, particularly over lower bandwidth mobile connections. Still, whilst sound effects and music add to the experience of playing a game, they’re not actually essential: you can play the game without them. Therefore I load sound effects and music asynchronously in order of priority as follows:

  1. Sound effects.

  2. In-game music.

  3. All other music.

The priority is just about the order the requests are submitted in and is most beneficial over slower connections, and those using HTTP 1.1 as opposed to HTTP2. Fortunately nowadays I only have to worry about serving HTTP 1.1 to IE11 on Windows 8.1 and earlier, and Opera Mini (which I don’t care about anyway). I decided to drop support for IE10 because, although games are/were playable, I have no reliable way to test against IE10 (and Microsoft themselves dropped support for it some time ago).

The prioritisation is somewhat obvious: sound effects are clearly the most important for the game-playing experience so it makes sense to load them first, but why load the in-game music before the title screen/main menu music?

Simply because on iOS devices web audio is disabled until it gets triggered by a user interaction, like tapping a button. I have all my buttons set up to enable audio, in addition to whatever other action they trigger. Since the first thing a player is likely to do is tap the Play button, I load the in-game music before any other music even though the title screen appears first.

This all leads to what I’d suggest is another best practice:

BEST PRACTICE: Any asset that isn’t absolutely essential to use your website should be loaded later, or not loaded at all.

This shot of Chrome Dev Tools’ Network tab shows how this plays out in practice:

Loading Star Citadel visualised in Chrome Dev Tools. Note how all the DOM content loads and the page renders before we start loading sound effects.

I’ve used Star Citadel as the example here, simply because Shoot The Rocks does a lot of image creation, and these images show up as resources in the Network tab even though they’re generated locally.

Now this is over a fairly slow WiFi connection in the Swiss Alps, which has lower bandwidth (and is somewhat laggier) than my broadband connection back home. But you can see that the DOM content is loaded, and the page rendered, in under 600ms (at home this would be less than 200ms). You can see that we don’t start loading sound effects (the bottom five rows) until after the render.

I mentioned in my previous post that I’d highlight areas where improvement is still needed, and this is one of them.

In all games it’s possible to enable or disable music and sound effects, and this can be done independently, as you can see from this screenshot of the main menu in Shoot The Rocks:

Shoot The Rocks main menu showing sound and music configuration, with a few leftover asteroids from the previous game still floating around

Sound configuration in Shoot The Rocks - note the two toggle buttons at the bottom of the menu. In this case sound effects are enabled and music is disabled. (You can clearly tell I’ve been playing a game when I should have been writing this post because there are asteroids flying around behind the menu. Busted!)

Now this is grand but I really should load sound effects and music only when the corresponding setting is enabled, whereas at the moment I just load them anyway, which isn’t so great. This is particularly important for mobile users, who might be on capped or metered data plans.

Moving on, let’s take a look at some code to see how I’ve achieved this deferred loading.

Firstly, sound effects and music are just configured directly in the source code as a bunch of descriptors, which look like this:

Each game has its own version of this configuration.

You see the config object has two array members, sounds and music. Each of these is a collection of descriptors, most of which are hopefully pretty self-explanatory. Note that music can (but does not have to) have two sources, one of which is a lower quality file for mobile devices. Sound effects have so far been small enough that I haven’t bothered with lower quality versions, although I might support them in future.

Apart from the overall prioritisation I’ve described above, XHRs to load sounds are fired off in the order they appear in this lists, so if some sounds are more important than others they should just be listed first.

(Btw, if you’re wondering what pinjector is, it’s a very simple, and rather cruddy injector I wrote that behaves somewhat like the Angular 1.x injector. The reason for its existence is simple: when I started writing Star Citadel I was, for literally no good reason, using Angular. I quickly realised this was madness, so ditched Angular but couldn’t be bothered to rewrite all the service boilerplate. This is a decision I now regret. If you really want the source code for it you can find it here.)

And, using these descriptors, we load the sounds with something like this:

This is perhaps a bit more code than we need for this example (I literally copied and pasted it verbatim from the source) but the key point to take away is that each descriptor is populated with a gain module, which is connected to the sound destination, and then a buffer containing the sound itself, which is loaded asynchronously via XHR.

We can then play sound effects using these functions:

Note how, if sound effects are disabled, the effect doesn’t exist, or the effect doesn’t contain a buffer (i.e., the sound effect hasn’t yet been loaded) we simply return without doing anything.

I posted a shot from the Network tab in Chrome Dev Tools showing how this all behaves, but what’s quite cool is that you can test out what happens for yourself. It really becomes much clearer if you apply throttling to simulate a slow connection, because then you can see that the games still work without sound effects and music being completely loaded. To do this, open up Chrome Dev Tools and so the following:

  1. Switch to the Network tab.

  2. In the Throttling dropdown towards the right of the toolbar select GPRS (500ms, 50kb/s).

  3. Point your browser window at one of the following URLs:

  4. When the game’s menu eventually appears (which will take a while - possibly 10 seconds or more whilst the game’s JavaScript loads), click the Play button. (Btw, you may notice some JS errors in Adsense code whilst you do this; it seems that ads don’t necessarily play well with really slow connections.)

Assuming you have both sound effects and music enabled you’ll probably find that the game plays fine, but there’s no music to start off with (and it takes quite some time before you hear any). It should also be obvious that some of the sounds are missing. As you carry on playing you’ll start to hear more and more of the different sound effects as they gradually load.

So the end result is that, as much as the sound effects and music taken together are quite large assets, deferring their loading allows for a much more satisfactory experience for the user, because they can start playing the game straight away. This is despite the fact that complete loading of the game and all assets is measurably no faster than if I loaded them all upfront.

In my next post I’ll talk about how to get that key above the fold content loaded and rendered as quickly as possible so that, even though there’s still quite a lot of work to do before the page is fully loaded and interactive, the user has the impression that content has loaded and that something is at least happening.

Read the rest of this series on perceived performance:

PREVIOUS: Perceived performance part 1: what is perceived performance?

THIS POST: Perceived performance part 2: load assets asynchronously or on-demand

COMING SOON: Perceived performance part 3: load key content first and give the user immediate feedback

Perceived performance part 1: what is perceived performance?

I have this talk I first gave late last year called Client-side performance for back-end developers, of which those slides are just the first iteration. A notable topic I haven’t included, is the importance of perceived performance. Yet it’s something I’m very conscious of, and I’ve been working continuously to improve it for My basic question is always, “How does it feel to use this?” And this is a principle that cuts across the whole site, especially the games. I want it to feel great: I want it to feel slick, fast, responsive on all devices and browsers.

Running across this piece by Brian Jackson (formerly of keycdn) the other day brought the topic of perceived performance back to the forefront of my mind, and reminded me was something I should write about. So in this series of posts I want to cover what perceived performance is, and how I’ve improved it on

OK, so let’s start off with perhaps the most important question: what is perceived performance?

Well, one way to look at it is it’s the answer to the question, “How fast does this feel?” So, it’s not about how fast some process really is, but about how fast a user perceives it as being. And so it can sometimes be surprising when real, objective, measurable performance and perceived performance don’t always correlate. Now with that being said, it should be obvious that in many cases improving real performance can often help with the perception of performance too, but sometimes you can improve perceived performance significantly without improving real performance in any way (and sometimes whilst making it slightly worse).

You might legitimately wonder why this matters but, when it comes to the web, the fact is that slower websites lead to dissatisfied visitors who don’t hang around, and fewer conversions (however you define conversion for your site). And never mind how fast it is or isn’t, if your website even feels slow, you’re going to lose out. People will vote with their attention by directing it elsewhere.

The key point I’d like to make - and I hope this isn’t too controversial - is that your users’ perception of your web app’s performance is actually more important than its real, measured performance. Like I say, perceived performance is key to a great user experience.

How have I approached this for Well the first thing to say is that I have, and continue to, expend effort on making everything actually, measurably, really faster, and these are things that I’ll talk about in other posts. Beyond that, and it’s really something you do when you’re out of options for making something measurably faster, I’ve used a few tricks and techniques to improve perception:

  1. Load assets resources out of band/asynchronously or on demand.

  2. Load key above the fold content as quickly as possible, and give the user immediate feedback that something is happening.

  3. Gracefully degrade the experience on slower devices to keep performance up.

  4. Load and render key above the fold content first, delay everything else, even if this means the page load takes multiple seconds overall.

I’m going to talk about each of these techniques in more detail in the follow-up posts, so you can see exactly what I’ve done. I’ll also highlight areas where improvement is still needed (I’m nothing if not honest!).

For now though, I’m just going to leave you with this take home…

PRINCIPLE: Improving real performance will generally improve users’ perception of performance. Where real performance improvements are not possible, selectively loading and displaying content, or otherwise indicating that activity is taking place, can improve their perception of performance even at a small cost to real performance.

Read the rest of this series on perceived performance:

THIS POST: Perceived performance part 1: what is perceived performance?

NEXT: Perceived performance part 2: load assets asynchronously or on-demand

COMING SOON: Perceived performance part 3: load key content first and give the user immediate feedback

Is that Star Castle in The Expanse on Netflix?

A few days ago I started watching the excellent new series, The Expanse, on Netflix. They claim it as a Netflix Original, but both IMDB and Wikipedia claim it’s an Alcon Entertainment and Sean Daniel Company production, originally for SyFy, so I don’t know.

Anyway, I don’t want to say too much about it for fear of revealing spoilers. Suffice to say back in the day I was a big fan of series like Cowboy Bebop, Firefly (and, of course, Serenity), and Battlestar Galactica, and The Expanse goes some way toward scratching that itch in a way that hasn’t really happened since BSG ended. If you’re a fan of sci-fi or space opera with a bit of an edge to it I’d highly recommend you check it out.

Anyway, I’ve been watching this in the evenings for a few days, and towards the end of episode 8 I spotted something interesting on a holo-display. Here’s the shot with enough of the dialogue to set the scene:

Do we think this officer is actually playing Star Castle in The Expanse and not, as she claims, analysing a space battle involving the Donnager? Hmm.

Oh, really?

Well, you can call it whatever you like but it looks to me like your boss just busted you enjoying a good old fashioned game of Star Castle when you should have been working.

Still, it was probably worth it, don’t you think? And if you fancy a game yourself just click here.

How good is Cloudflare's Always Online™ technology?

If your site goes down, exactly how good is Cloudflare’s Always Online™ technology at bailing you out?

Rather good. But…

We’ll get to the “but” in a minute. Let’s deal with the “rather good” first. Here’s‘s homepage when the site is down:

The homepage when the site is down. Note that Cloudflare is still serving it up, which is great.

By down, in this case I mean the service that runs the site is down, although the instance it normally runs on is up.

Now apart from that banner, which is basically an ad that says, “Cloudflare is awesome and you should use it!”, everything looks normal. I could probably get rid of that by giving Cloudflare more money, and maybe one day I will. Nevertheless, my dodgy web design skills aside (and this page is going to get a makeover), everything that should be on the homepage has loaded and is working fine because Cloudflare has been able to serve it up from its cache.

So can I play the games?

Yes, I can. My Asteroids remake, Shoot the Rocks, works fine. Here you can see it loading as if nothing is amiss:

Shoot the Rocks loading like nothing's wrong - awesome.

The main menu looks a bit gimpy because the banner messes with the layout a bit, which might be more problematic on iOS than on desktop or Android, but everything still works:

The Shoot the Rocks menu is slightly messed up but still works.

And the game itself runs flawlessly, and weirdly without the banner:

Shoot the Rocks running flawlessly, thanks to Cloudflare.

Well, bless Cloudflare is all I can say to that.

There was that “but” though, so we should take a look at that because it doesn’t always work out so well. Here’s what happened when I tried to run Star Citadel, my take on Star Castle:

Argh! Star Citadel doesn't work - I just get this nasty error.

That’s a less happy, although completely understandable, outcome: Cloudflare simply didn’t have the page in its cache. If your site was up for a while before it went down, or you get a decent amount of traffic around the world, hopefully this doesn’t happen.

I say all around the world because, being a CDN, Cloudflare can cache your site on edge servers all over the globe but won’t cache it anywhere until somebody accesses it. This means that pages will be cached on servers located geographically nearer to people who use them (in general), but not on servers located geographically further away.

Thus in places where your site gets used a lot, hopefully people get to carry on using it even whilst your back end services or infrastructure are burning, but people in other areas won’t get the same experience, and will instead see an error.

Again, if I paid more I think could probably replace the above with some sort of branded custom error page, so choose an appropriate plan for your use case.

What will and won’t work offline?

So this all works because of the way I’ve architected the site. Almost everything you see when you visit a page or play a game on is static content served up by my web server or, more likely, Cloudflare. As little as possible happens on the server, which is why Cloudflare can bail me out.

If all my content were dynamically generated server side it would be a different story: I might still see a static version of the pages as last seen by Cloudflare, but nothing would really work.

In reality most sites sit somewhere in between these two extremes, and even for, there are things that wouldn’t work.

For example, the as yet publicly unavailable hiscore service where you can record your performance in each game for posterity and bragging rights to your friends? Totally not going to work because there’s back end infrastructure in there:

  • A service to record high scores - in fact ideally I’d record scores from all games and build up a gigantic league table for each game, as played in modern and classic modes, split by device, and so on,

  • A data store to put all of this in: right now I’m thinking SQL Azure because we’re talking about a big table of data, for which SQL is absolutely perfect (this definitely is not a document shaped problem at any rate),

  • A service to expose that hiscore data, or a league table if you like (or possibly we’re talking about two different views on the same data), to clients, which might be the games themselves, or other stats pages on the site, possibly in the future even dedicated mobile apps.

If the two services are deployed separately to the main website they might still be available but, if not, or if there’s a problem with the hosting infrastructure not in a million years will the above work.

That doesn’t mean you can’t record a player’s score in the meantime. There is, of course, LocalStorage or IndexedDB, so if you’ve built your site for an offline first experience, your user’s might not even notice that there’s a problem, particularly if hiscore data was loaded before the site/service went down. You can just record the player’s score and submit it later when the site is back online.

There are, of course, caveats. Any data you put into LocalStorage as plain text is incredibly easy to tamper with using tools such as the HTML5 Storage Manager All In One Chrome extension. You can use a library like lz-string to compress any data you add to LocalStorage to render it no longer human-readable, but somebody who’s committed can always copy and paste it out, decompress, edit, recompress, and then paste it back in to LocalStorage. How much you’re worried about that is something you should consider.

You can also encrypt the data using, for example, this JavaScript AES implementation. This will certainly discourage the casual hacker, but remember that the merits of cryptography in JavaScript are extremely questionable (see also here for a slightly more recent perspective). JS encryption certainly isn’t going to stop somebody who’s committed to tampering with the data.

As far as I know there’s no (supported) mechanism for accessing IndexedDB except by the site that wrote the data but, remember, this is data stored on the client so you may want to consider it suspect anyway.

LocalStorage and IndexedDB work well when whatever you store in them will eventually undergo strong validation on the server-side. It does not work so well when this isn’t the case - perhaps because the data is collected or generated in such a way that validation is very hard - and you can’t trust your users (because they want a ludicrously high score for bragging rights, for example).

Anyway, to drag this back to the point: you can offer a decent offline experience, and this will prove especially useful in scenarios where strong validation later on, when your web client can once again connect to your services, is suitable.

In some situations you’re just going to have to tell the user that certain functionality isn’t available at the moment but it will be back online soon. For example, I’d like to offer some sort of player history and stats service. I’d really like to offer people the ability to compare with their friends, and perhaps compete directly with them. Clearly, this won’t work offline, but graceful failure is an option I’ll happily embrace.

So, the verdict on Cloudflare Always Online™? Pretty good - just be aware of the limitations:

  • Consider where your users are: will they be able to access the pages they need? Can you use a service such as pingdom to ensure Cloudflare caches the necessary pages? (Actually, thinking about it, maybe this is something Cloudflare already offer in their Business or Enterprise plans.)

  • Consider how you can build an offline first experience for functionality with online requirements: can you hold data for later? Or perhaps limit functionality? Or can you at least make it unavailable gracefully by telling the user what’s going on, rather than just breaking?

And that’s it. As always, hope this was useful, and please do leave questions or comments below.

Thanks for reading!

How to run a Node.js app as a service on Ubuntu

One of the things I’ve needed to do with for a while is start running node as a service.

It’s easy enough to run background tasks directly from bash with a command like:

sudo nohup node index.js &> /dev/null &

And if you run that command and log out of your ssh session, node will carry on running just fine in the background. But what happens if node falls over? For this has never happened, but that’s likely because the site hasn’t been under much stress up to now.

(By the way, if you want a detailed discussion of running bash commands in the background it’s definitely worth checking out Running Bash Commands in the Background the Right Way.)

Realistically at some point the site is going to go down, and whilst Cloudflare will to some extent bail me out, I’d like it to come back up again without a manual intervention, especially if it’s due to something as simple as the node process crapping out.

Here you can see the effect of Cloudflare’s assistance. It’s not bad but that banner (basically an advert) at the top isn’t ideal. I suspect I could get rid of that by paying for a more expensive plan.

Here's Cloudflare dragging my backside out of the fire when I'm testing what happens when node goes down. It's not ideal, but at least you can still play the games.

The great thing is that you can still play the games so if you need a fix of Asteroids or Star Castle you can go ahead and fire up Shoot the Rocks or Star Citadel and they’ll (probably) still run. (I’ll blog on this topic in due course as well.)

So how do we make sure it runs as a service and recovers without intervention?

I’m running the site on an instance spun up from a bog standard Ubuntu AMI in EC2, and deploying is just a case of cloning my git repo into a new instance and running a script. Nothing fancy either: just a shell script that installs a few things, runs gulp, and copies some files around. (That’s right: no Docker. I’ll post about why later.)

These days I’m much more familiar with Windows than Linux, so had to do a bit of digging around on how to run a process as a service, but it turns out to be quite straightforward, albeit that there are a couple of gotchas, and there are two different ways of doing it, depending upon which version of Ubuntu you’re using.

Pre Ubuntu 15.04 - upstart

Due to the inevitable lag as Google’s ranking algorithm catches up with today’s technological landscape the material you’ll most likely run across first suggests going down the upstart route, which involves writing a .conf file into /etc/init. However, whilst that’s fine if you’re running an older version of Ubuntu - and as of this writing I believe Ubuntu 14 is still widespread - upstart was deprecated in 15.04 in favour of systemd and if, like me, you’re running 16.x, well, you’re out of luck. If you’ve been keeping more up to date with Ubuntu developments than I have you probably already know about this.

In theory you can still use apt-get to install upstart on later versions of Ubuntu but this isn’t something I’d recommend due to reports of serious problems after doing so.

Still, if you are in the old world you can use upstart, and I’d recommend this gist from Jobsamuel Núñez as a guide.

I followed these instructions and the only thing I did differently was to redirect to /dev/null because all my logging is sent to loggly because I don’t want to manually manage log files. You can see this under the script section here:

Note the respawn settings. This tells upstart to try and respawn the process up to 99 times at 5 second intervals. If the service can’t be restarted after 99 tries, upstart will give up.

Ubuntu 15.04 and later - systemd

This is no more difficult that using upstart. In fact, in some ways it’s a bit easier. Full instructions at Just skip over the node server section at the top and start reading the systemd section. I had only two issues.

The first was simple: I needed to prefix all the systemctl commands with sudo, otherwise it would ask me for a password, which I don’t have because I log in with ssh. Here’s an example:

sudo systemctl enable arcadely.service

The second was that when I first started the service, whilst all my 301 permanent redirects were working fine, none of the site content would load - everything was coming back with 404 - Not Found. This turned out to be because the working directory of the service process is set to / by default, and thus when express.js tried to serve up content relative to this folder it inevitably failed. This is easily fixed by setting the WorkingDirectory property in the descriptor, as you can see here:

Note also the restart settings - this is what tells systemd to bring the service back up again if it fails.

The following bash fragment shows how I’m installing the service.

You can see that after I’ve installed it once, I don’t bother installing it again (which is just a file copy), but I do bounce it so that node picks up any changes that have just been deployed. I could use nodemon for this, and not bother with the bounce, but I’ve found that sometimes nodemon can sometimes get stuck and require manual intervention if the app it’s monitoring crashes due to restarting in the midst of a deployment, which can easily happen if the deployment takes longer than usual, for some reason. I do still use nodemon during development, in concert with a gulp watch task.

I might change this so that it copies the service descriptor every time, in case there are changes. This is likely to be infrequent but it means I never have to think about it again, which is a bonus.

The only thing is, when the service descriptor changes you generally need to do a reload of the systemctl daemon:

sudo systemctl daemon-reload

Don’t worry - if this is necessary you’ll be prompted. There’s no guesswork. In any case, I’ll probably automate this away by adding it to the script.

Anyway, I’m now in a position where I can reliably deploy to a vanilla Ubuntu AMI with all dependencies in just a couple of minutes. This includes time to compile Python 2.7.13, which is the lengthiest operation.

I hope the above tips come in useful!

EDIT - 24 Jan 2017: systemd local root exploit in v228

There’s been some noise today about a local root exploit in systemd v228. I first picked this up on HackerNews, and you can find the original report here.

This has been fixed in later releases (and note the controversy about the stealthy way that happened), but you can check if your systems are vulnerable using one of the following commands to check your systemd version. Which command you use depends on which distro you’re using:

  • init --version

  • systemd --version (Ubuntu, at least v16.04; Debian Jessie; Mint 18)

  • systemctl --version (Fedora)

For example, here’s the output for’s production instance:

[email protected]:~$ systemd --version
systemd 229

If in doubt just try them all until you find the one that works.

Welcome to the blog

Welcome to the blog, where we’ll be talking about games, game dev, and web performance.

All our games are built with HTML5, JavaScript, and CSS. That’s right: no Flash. Which means they work on mobile and tablet, as well as desktop and laptop.

So far we only have a couple of games:

We hope you enjoy playing them!

For anyone who’s interested, our behind the scenes tech stack is express.js and node.js running on Bitnami, currently in Amazon Web Services’ with Cloudflare as CDN. The blog is generated by Hexo using the excellent alpha-dust theme from Jonathan Klughertz.

We recommend you subscribe to our Atom feed for posts and updates on new game releases.