Even if I have been a little low on posts on technical parts of testing and test automation, this is something of an interest to me as well. For example, I have been doing some ruby/watir/cucumber gui automation testing. I really like cucumber and the intentions it has on communication between stakeholders.
This is actually also one of the intentions that my dear friend and colleague Renas Reda has with his new framework Robotium.
Robotium is a test framework created to make automatic black-box testing of Android applications significantly faster and easier than what is possible with Android instrumentation tests out-of-the-box.
With the support of Robotium, test case developers can write system and acceptance test scenarios, spanning multiple Android activities.
Inspired by cucumber, there are great plans to integrate with it or at least create the same kind of BDD syntax in Robotium. And I really like that idea.
Try it out, its open source at google code.
How often do you find a really serious bug the first thing you do on monday mornings? Or any mornings for that matter, when you just started to look at it.
How often is it the other way around, that the first thing you do is trying to remember what you did on friday, to cause that serious issue you found at 4.30 before you were leaving for home and for the weekend?
For me, it is always the latter. Not only fridays, but any day of the week, the most common time when I find bugs is really late in the afternoon, preferably if you are having an appointment or anything right after work. =) But why is that?
I have three possible explanations, that I have been thinking about:
1. You could be testing a really hard-to-set-up system, that takes half of the day setting up before it is possible to test even a little bit. This is the reason for me every now and then. Recently we have been setting up load tests during night, which have failed a couple of times, so the mornings have been devoted to investigate logs for failure explanations and then it has taken some time to reset databases and getting up and running again. Usually with a bug fix or configuration tweak that also takes some time to verify.
2. Exploring the product and its features and areas all day has made you get more sensitive to finding the weaknesses, and poking the system in these spots make the really tricky bugs appear. These might be the most severe afternoon bugs, or the opposite, the ones that are too complex to be a threat to the product and just are hard to fix, with no value added if fixed.
3. You are unintentionally defocusing from the tasks at hand since it is getting late in the day and the lack of energy is carrying your senses in completely other directions from what you have been doing all day. The defocusing strategy is actually something that is brought up in rapid testing contexes for finding other types of bugs than you normally do. Lack of energy just makes you get in this mode automatically during afternoons. These bugs are more too often hard to recreate for investigation, since your lack of energy might miss out on observing the reason behind the bug.
Have you experienced the afternoon bug heuristic? What have been causing you to find them? and specifically, why do you find them at that time?
I have written before about what I think of pair testing with developers. This post is going to give you some more examples of the benefits.
Pair testing is like pair programming, in the way that you get each other to concentrate on the task at hand. Otherwise it is easy to step into sidetracks and staying there for too long. Sidetracks are possible also in a pair, but the time it takes to get back on track is shorter.
This time, I am actually performing the tests together with both of the developers of the system. It is another system from last time, but one of the developers is the same.
Just like the other system, I have learned alot about how the developers think and master the tools useful for this testing, giving me all sorts of perspectives on the product. They are both different in how they think of things, but their knowledge of the system is the same, but anyway they have different ways of expressing how the system works. This is a very exciting setting for me as tester, to gather and filter relevant information to the risks that we are facing.
Another aspect of this product, is that we have a very narrow deadline for production. This has lead to an enormous interest with the product owner, which is spending sometimes hours in a day following our progress of tuning the product for release. This gives also exciting discussions, since he has an even broader view of the users and functionality. He is technical, but sometimes I still have to translate the meaning of words between developers and him, in any direction. This is why I think of communication as a very important aspect in testing. More about that later.
Getting to the point, sitting next to developers when solving a bug, gave me the chance of getting a glance at the workflow in the product code. Usually I try to avoid this deep digging, but since he was holding the weel, I could just watch the exploration of the code. If I dont follow, I can just ask about explanation to see if there are risks to be considered. Yesterday, this actually revealed a bug. When trying to find another bug, I was observant enough about a discrepancy in another part of the code. Highlighting this just made my partner express the words I really like to hear from any developer: “Thats a bug!”
This was followed by an explanation about it, and the most important thing, it was a bug that would have been really hard to find if it had been ruining things later on.
The thing in this case, would be that if this bug had been seen by myself reviewing the code, I would not have been sure about it, and the feedback loop to any developer would have been a very long one, for this type of issue. Typo in a string, would that ruin your day? Well, in this case, the string value was very important, but as a tester do you think about that at all times?
We saved alot of money on the hours we would have been throwing away of finding this bug later on, and I think that is worth the money spent on two persons testing together, may it have been a developer and a tester.