Perceived performance part 2: load assets asynchronously or on-demand

When I started my side-project, for such it is, I had a variety of motivations. One of them was that I was a bit fed up of all these low-rent Flash versions of classic arcade games that I’d loved as a kid tarnishing my nostalgia with their long load times, pages covered with ads (yes, there are ads on this site, because I want it to pay for itself, but I try to keep things tasteful), and generally sub-par experience.

I even felt pretty smug and superior about the idea of building games with HTML5, CSS, and JavaScript that would load instantly, or pretty close to, compared with their bloated Flash equivalents with their interminable loading progress bars. Then, of course, reality came calling, complete with its usual payload of disappointment - something I posted about in a comment on Hacker News. There I identified some fairly obvious reasons why games can take a while to load that go some way beyond “Flash is an awful nineties/noughties horrendo-tech abortion that should be blown into charred corpse fragments with dynamite”:

  • Loading games assets takes time, particularly on slower connections (like mobile)

    • Graphics (background images, sprites, textures, logos, etc.)
    • Sound effects
    • Music
    • Fonts
    • Maps and other metadata (game objects, etc.)
  • Creating procedurally generated content can take a while

Now this is frustrating, because it turns out these are big problems for me in that I have a combination of downloaded assets and procedurally generated content. So how do I get around this?

It’s really a combination of two techniques:

  1. Asynchronous loading of non-essential large assets after DOM content has loaded, and the page has rendered, which happens immediately after.

  2. Immediate feedback to the user that something is happening upon page rendering.

Neither of these results in anything being loaded any quicker, but it does get the loading "out of the way", and at least shows the user that something is going on whilst the procedurally generated content is being created. I’ll talk about (1) in detail below, and address (2) in a follow-up post.

Now both Star Castle and Asteroids are vector based arcade games so in my versions, Star Citadel or Shoot The Rocks, we don’t need to worry about downloading any bitmaps or textures for either. The graphics for all the game objects - spaceships, asteroids, flying saucers, mines, etc. - are specified using a simple vector-based notification as JavaScript objects, so they’re just part of the source code. These specifications are just based on designs I knocked up quickly in OmniGraffle.

Using JavaScript objects is not really super-efficient in terms of space but uglify does a good-enough job of compressing them, and they’re not very large or complex anyway. The player’s ship in Star Castle is 8 blue lines, for crying out loud!

If I did have a lot of bitmaps, textures, pixel art, 3D models, or whatever, I’d probably have to load a lot of them upfront because they’d be needed to actually play the game, and there’s not much avoiding that. The one thing I could do, if I were using different graphics for different levels, for example, is load only a minimal set of graphics up front, and then load the rest on demand later on.

I also don’t have to worry about maps, which are another potentially sizeable asset. Again though, I could load them on demand for each level or area as required rather than all in one go.

This deferred loading is a big advantage over Flash games which, at least in most of the cases I see, tend to load most of their assets in one go upfront. I’m not sure if this is a necessity or just a case of bad architecture. Given my somewhat snarky "low-rent" comment earlier I’d guess at bad architectural choice. I don’t know Flash at all, but it’s quite hard to imagine that Adobe wouldn’t have included some equivalent of JavaScript’s XHR in it somewhere, to allow content to be downloaded in the background.

What I do have in terms of large assets are sound effects and music. These are obviously compressed, and I use MP3s because they have the best across-the-board browser compatibility of any compressed audio format. They also work with both the HTML5 <audo> element, required for IE11 compatibility, and the Web Audio API, which is well supported across other browsers. I’d prefer to use Ogg Vorbis because it’s more efficient but the browser support just isn’t there unfortunately.

Sound effects don’t take up much room, currently weighing in at less than 200K for both games, although that’s set to increase because I haven’t created all the sound effects I need yet.

Sadly music is a different story. Both games use three tracks: one for the title/main menu screen, another whilst playing the game, and a final track for the game over screen. These weigh in at 6-7MB total for each game on desktop, or 4-6MB for the lower quality mono versions I serve to mobile devices and tablets.

BEST PRACTICE: To improve REAL performance of your website across a range of devices you should consider serving more optimised assets to mobile users.

This is obviously problematic, particularly over lower bandwidth mobile connections. Still, whilst sound effects and music add to the experience of playing a game, they’re not actually essential: you can play the game without them. Therefore I load sound effects and music asynchronously in order of priority as follows:

  1. Sound effects.

  2. In-game music.

  3. All other music.

The priority is just about the order the requests are submitted in and is most beneficial over slower connections, and those using HTTP 1.1 as opposed to HTTP2. Fortunately nowadays I only have to worry about serving HTTP 1.1 to IE11 on Windows 8.1 and earlier, and Opera Mini (which I don’t care about anyway). I decided to drop support for IE10 because, although games are/were playable, I have no reliable way to test against IE10 (and Microsoft themselves dropped support for it some time ago).

The prioritisation is somewhat obvious: sound effects are clearly the most important for the game-playing experience so it makes sense to load them first, but why load the in-game music before the title screen/main menu music?

Simply because on iOS devices web audio is disabled until it gets triggered by a user interaction, like tapping a button. I have all my buttons set up to enable audio, in addition to whatever other action they trigger. Since the first thing a player is likely to do is tap the Play button, I load the in-game music before any other music even though the title screen appears first.

This all leads to what I’d suggest is another best practice:

BEST PRACTICE: Any asset that isn’t absolutely essential to use your website should be loaded later, or not loaded at all.

This shot of Chrome Dev Tools’ Network tab shows how this plays out in practice:

Loading Star Citadel visualised in Chrome Dev Tools. Note how all the DOM content loads and the page renders before we start loading sound effects.

I’ve used Star Citadel as the example here, simply because Shoot The Rocks does a lot of image creation, and these images show up as resources in the Network tab even though they’re generated locally.

Now this is over a fairly slow WiFi connection in the Swiss Alps, which has lower bandwidth (and is somewhat laggier) than my broadband connection back home. But you can see that the DOM content is loaded, and the page rendered, in under 600ms (at home this would be less than 200ms). You can see that we don’t start loading sound effects (the bottom five rows) until after the render.

I mentioned in my previous post that I’d highlight areas where improvement is still needed, and this is one of them.

In all games it’s possible to enable or disable music and sound effects, and this can be done independently, as you can see from this screenshot of the main menu in Shoot The Rocks:

Shoot The Rocks main menu showing sound and music configuration, with a few leftover asteroids from the previous game still floating around

Sound configuration in Shoot The Rocks - note the two toggle buttons at the bottom of the menu. In this case sound effects are enabled and music is disabled. (You can clearly tell I’ve been playing a game when I should have been writing this post because there are asteroids flying around behind the menu. Busted!)

Now this is grand but I really should load sound effects and music only when the corresponding setting is enabled, whereas at the moment I just load them anyway, which isn’t so great. This is particularly important for mobile users, who might be on capped or metered data plans.

Moving on, let’s take a look at some code to see how I’ve achieved this deferred loading.

Firstly, sound effects and music are just configured directly in the source code as a bunch of descriptors, which look like this:

Each game has its own version of this configuration.

You see the config object has two array members, sounds and music. Each of these is a collection of descriptors, most of which are hopefully pretty self-explanatory. Note that music can (but does not have to) have two sources, one of which is a lower quality file for mobile devices. Sound effects have so far been small enough that I haven’t bothered with lower quality versions, although I might support them in future.

Apart from the overall prioritisation I’ve described above, XHRs to load sounds are fired off in the order they appear in this lists, so if some sounds are more important than others they should just be listed first.

(Btw, if you’re wondering what pinjector is, it’s a very simple, and rather cruddy injector I wrote that behaves somewhat like the Angular 1.x injector. The reason for its existence is simple: when I started writing Star Citadel I was, for literally no good reason, using Angular. I quickly realised this was madness, so ditched Angular but couldn’t be bothered to rewrite all the service boilerplate. This is a decision I now regret. If you really want the source code for it you can find it

And, using these descriptors, we load the sounds with something like this:

This is perhaps a bit more code than we need for this example (I literally copied and pasted it verbatim from the source) but the key point to take away is that each descriptor is populated with a gain module, which is connected to the sound destination, and then a buffer containing the sound itself, which is loaded asynchronously via XHR.

We can then play sound effects using these functions:

Note how, if sound effects are disabled, the effect doesn’t exist, or the effect doesn’t contain a buffer (i.e., the sound effect hasn’t yet been loaded) we simply return without doing anything.

I posted a shot from the Network tab in Chrome Dev Tools showing how this all behaves, but what’s quite cool is that you can test out what happens for yourself. It really becomes much clearer if you apply throttling to simulate a slow connection, because then you can see that the games still work without sound effects and music being completely loaded. To do this, open up Chrome Dev Tools and so the following:

  1. Switch to the Network tab.

  2. In the Throttling dropdown towards the right of the toolbar select GPRS (500ms, 50kb/s).

  3. Point your browser window at one of the following URLs:

  4. When the game’s menu eventually appears (which will take a while - possibly 10 seconds or more whilst the game’s JavaScript loads), click the Play button. (Btw, you may notice some JS errors in Adsense code whilst you do this; it seems that ads don’t necessarily play well with really slow connections.)

Assuming you have both sound effects and music enabled you’ll probably find that the game plays fine, but there’s no music to start off with (and it takes quite some time before you hear any). It should also be obvious that some of the sounds are missing. As you carry on playing you’ll start to hear more and more of the different sound effects as they gradually load.

So the end result is that, as much as the sound effects and music taken together are quite large assets, deferring their loading allows for a much more satisfactory experience for the user, because they can start playing the game straight away. This is despite the fact that complete loading of the game and all assets is measurably no faster than if I loaded them all upfront.

In my next post I’ll talk about how to get that key above the fold content loaded and rendered as quickly as possible so that, even though there’s still quite a lot of work to do before the page is fully loaded and interactive, the user has the impression that content has loaded and that something is at least happening.

Read the rest of this series on perceived performance:

PREVIOUS: Perceived performance part 1: what is perceived performance?

THIS POST: Perceived performance part 2: load assets asynchronously or on-demand

NEXT: Perceived performance part 3: load key content first and give the user immediate feedback