Audio implementation for iOS devices.

Audio is the unsung hero of most productions. And that truth remains when it comes to developing audio for mobile devices. Gamasutra has a great article discussing the challenges sound designers face when it comes to mobile.  This article deals specifically with iOS devices.  It is well worth reading.  Here are the highlights:

Asset Size – With Apple’s over the air size limit of 20MB, even if you are lucky to get half of that, you are left with only 10MB.  This should not be an excuse to produce shabby audio, but a reason to rise to the challenge and impress.  You can still create great sound design with some creative looping and compression.

One way to create some variety in your soundtrack would be to create an overall bed of ambient music.  It could have the basic structure; drums, bass, rhythm and the basic chordal structure.  Melodic instances; guitar licks, piano, sax (that’s right saxophone) could all be broken into smaller, bpm matched, chunks and called back programmatically or randomly, thus giving some randomness and variance that our consumers so desperately crave.

Environment – How and where consumers listen to your apps vary.  No matter where, the audio still has to transfer from the device to the users ear.  Elements like an eerie sound track can easily get lost in bus station or restaurant.  Moving towards sounds that are transient and short, but not annoying, can help the user still feel engaged in noisy spaces.  But the challenge then is to make sure those sounds translate well when the user is wearing headphones.

Creating audio for mobile devices forces us as sound designers to do things that will make our music producing counterparts cringe with disapproval.  You know those 15 cent ear-buds that came with the set of Bic razors your wife just purchased?  Better bust those out and listen to your game/app with those, because that is what most of the consumers will be using.  If you are lucky they might still have the $5 ear-buds that came with their iPhone.  But don’t count on it.

Compressed Audio Files – iOS only allows playback of one compressed audio file at a time.  So, if you want to have a soundtrack playing underneath all of your great sound design then it has to be uncompressed.  If your developers want a new music track and ambience per level, then you are strapped with a whole new set of challenges.  A one-minute-long PCM file at 24Khz is 5.8 MB. So the music and ambience alone amounts to 58 MB for 5, 1 minute loops for 5 different levels.

Now it is true that you can have more than one compressed file playback at a time if you use the software decoder, but this could be a hard sell to your developers especially if they are trying to maximize processing power.  They need to keep their framerates rock solid.  The idea of giving you some of their coveted processing power so your “click” sound can play during runtime is sure to send wafts of laughter through the assemblage of empty Mountain Dew cans and Cool Ranch Dorito bags  that festoon the dev cube…. oh wait that’s the recording studio.

So all in all, iOS (mobile devices in general) provides some new hoops for sound designers to jump through, but with a little creativity and some planning, they can be the things that make us all better.

2 Comments

  1. EXCELLENT article!!
    thank you

  2. We’ve recently started using ima4-encoded CAF files for sound effects on iOs. ima4 gives you lossy 4:1 compression over traditional PCM-based audio, and is not processor-intensive to decompress. Actually, it may decompress at load time, I’ve never been able to get a firm answer on that, so it may still use the same amount of RAM, but it definitely saves size on the actual app and download size.

    Also, assuming your audio engine supports it, I generally try to use he-aac, rather than mp3, for iOs music loops. It loops better and gives you noticeably better sound and much lower bitrates, and still uses the hardware decompression on the device.

Leave a Reply

Your email address will not be published. Required fields are marked *