iPhone OS Testing Best Practices

Question

Not so long ago iPhone development was quite simple, only a few OS versions and even less devices.

Now however, there are 2 major OS versions and 5 different devices to

Answer 1

In my experience, you won't have much compatibility trouble between iPod/iPhone. There are other gotchas to be aware of:

• The devices run at different speeds. iPhone v1 and iPhone 3G run at 412MHz; iPod Touch runs at 532MHz, and the new 3GS runs at 600MHz. This can have a big impact on performance and even functionality if you're getting fancy.
• There's a huge performance difference between EDGE/3G/WiFi networks; often the differences are counter-intuitive. EDGE can often have better latency (time to first byte) than 3G, while 3G has 10x better bandwidth. You'll want to test your app under all three conditions.
• Are you using Core Location? iPhone v1 and iPod Touch do not have GPS.
• Are you using the camera? The iPod Touch does not have a camera.
• Is your app compatible with jailbroken phones? A lot of people have done it, and if your app crashes on them, they will blame you, not the Dev Team hackers, and this will be reflected in your App Store ratings. Note especially that background apps can use up memory that you might have thought would be available exclusively for your app. Leave yourself some overhead.

So, what do you need to buy? If you're a serious developer, yes, you should have all 5 devices available. But do you need to test every build on all 5 devices? Does every one of your developers need 5 devices each? No.

One developer can probably test everything that matters with one iPhone 3G and an iPod Touch. Toss in a 3GS and your coverage is probably nearly perfect. (Note that development on 3GS is much nicer just because the CPU is faster, so your apps deploy more quickly.)

At work we have one device per developer, but they're a mix of 3G and iPod devices (and, today, one 3GS).