iPhone testing and bugs

Some 1-1,5 years back, when the iPhone 3G had made such a success in Sweden, to develop apps for it got to be a hot potato in the software development business. Everyone wanted an iPhone app for their product/company, and we were not late to adopt this need and make it happen. Now we are pretty good at developing cocoa apps.

From the testing perspective, I started to look at if testing on iPhone was different from any other software testing. I started off asking a senior app developer about his view on this, and got this answer in an email:

Iphone has very many complete components, which means that the developer doesn’t have to invent the wheel all the time. These complete components are well tested by Apple.

Focus on iPhone app testing should be on the levels of
1. Application logic
2. Verifying that the Apple interface guidelines is followed.
3. Input validation

As a tester, this “easy to develop, easy to test” mentality should in my perspective trigger all bells and whistles as warning signs.

To cut to the point, having this background and tested some of the apps developed by my friends, I want to say that testing iphone apps is nothing different nor creates different kind of bugs than when testing other software. Some of the different type of bugs that can be shown through screenshots is shown here. The number of times I have gotten apps to crash is facts that I actually don’t know, since it happens all the time.

For clarification, the environment is a regular iPhone 3G with the (at the time of any bug/screenshot) latest iOS3 software. I haven’t upgraded to iOS4 yet because of the risk of the phone getting slower than it already is. My dad complained about this just couple of weeks ago after upgrade. Neither have I jailbroken the phone, and I am not a very big consumer of apps, which means that I have a very limited amount of apps on my phone.

The reasons that I point out for the bugs are just guesses from my point, and should be taken like guesses as well =)

Application UI issues

Application UI issues include Tweetdeck where sending a tweet progress bar is shown on top of the tweet search column header. Another UI issue which I have seen in many other apps include that the default(?) error popup message has been left as default without describing the usage error that the user in this case forgot to fill out some mandatory fields.

Application logic

I am actually not sure if these logic issues should be on client or server side, but my guess is that its client side. The question here is if its a coincidence that two different apps encounter null as profile name when looking at the profile pages. A wild guess is that they both use the same kind of ”complete components” in development.

Server issues

Very often you need to serve your client app with data from a server. Having incorrect or inconsistent data creates these kind of issues. Like when I use this app for showing when the tram leaves my location. You might expect that the data source is the same for both iphone client and monitor at the station. But comparing them showed clearly that this was not the case.

Showing a wordpress site with an embedded picture within the tweetdeck UI looks more challenging than you might think of. Maybe CSS issues on the server side?

Native apps

If we take a look beyond the 3rd party apps from app store, I also want to highlight some native apps. I have actually no clue why my alarm time disappears sometimes, but it does quite often. It could be noted that it has nothing to do with the modules that carry out screenshots. Gut feeling says that it has something to do with having many alarms in the list.

Also App store reveals that there are humans behind all this. Not thinking of the text length when listing the ”Top paid iPhone apps” is of course not a very serious bug, but there is a lot of trust to be lost from sceptical customers that are just about to open their wallet for getting their first paid app.

So with a background of a software tester, I would like to state that testing iPhone/iPad software is no different than testing other software. Take into account that my most recent experiences come from testing large enterprise systems integrated with each others. Software is software, and it is programmed by humans.