Context driven testing in passport control

Some weeks ago when I was traveling to CAST in Seattle, I had to go through the well-known border control of the US. All in all, I have to say that I was a little disappointed since I did not at all experience it as very complex or special as I had imagined. But of course there was more control than when traveling between European countries, and I was surprised about the well-informed officer talking to every single arrival in the line for passport control. While waiting, I observed how he asked very thoughtful and deep, yet quite regular and open questions about arrivals and their stay in the US. Getting closer to him made it possible to over hear some of the conversations. A family in front of me almost got into trouble when the father said the children were american citizens but they had some foreign passports.

The girl in front of me had an interesting passport with a lot of Visas in it and a complex citizenship history, so the officer questioned what she was doing in US. She was an astronomer and they got into talking about her theories. The officer had apparenty read something about these things and she responded as if it was relevant as well. Then it actually turned out that what the officer had mentioned was very relevant to what her research was about, and they had a deep conversation about black holes and the universe. I was very impressed by how much this man knew about astronomy, but did not think I would need to explain myself and my work as much as her. What are the odds that he had read or knew something about software testing.

The officer asked me the usual questions, where I was going, the conference name and venue, what my company was called and so on. Then upon seeing my Chinese Visa from 2007 he started a conversation in Chinese, with quite good pronunciation. I could answer him and ask him some question which he really could understand. That was cool but still a little creepy. Then he started to ask about the conference, and I could never have imagined this in beforehand.

Officer: So what is the conference about?

Me: It is about Software testing.

Officer: But isn’t that area quite big, can you be more specific?

Me: Well, the theme is context-driven testing.

Officer: And what does that mean?

Me: It means that instead of trying to apply all or any specific testing practices to your testing at hand no matter what the product, you just try to use some that is needed for the specific context. (a little startled by the coming depth of this conversation)

Officer: So, it is really about knowing and understanding what you are doing when testing software?

I was really impressed with the quickness and accuracy of that conclusion, especially in the setting that this guy is a police officer inspecting passports at the airport. Making the leap from my quite startled and very brief answer to that question towards a thoughtful shorter version of it that was just spot on. And really, his answer reflects quite good in James Bachs and Cem Kaners definition principle “6. Good software testing is a challenging intellectual process”.

When thinking about this discussion for a couple of days, it came to me how right he really was.

I see context-driven testing as when you apply the best and most suitable approaches to testing according to your current understanding of the environment and prerequisites. This means that when I have some understanding of a product and its features, I can apply some known heuristics and suitable oracles in my testing.

Example: If I have knowledge about the logging mechanism of a product, I might use those features as part of my approach when revealing information about product status and its environment. There might be oracles in the form of “error codes” that are logged in certain scenarios.

If I at some point realize that my understanding of the current area of testing is limited, I have to hunt for more information to be able to get a better understanding of those areas.

Example: The logging mechanism seems to fail randomly during some of my testing of a certain feature. Doing some experiments and exploration through the logging documentation I finally realize that under certain conditions there is need to use other parts of the logging tool that is not used anywhere in the product. 

Through my expanded understanding of the logging I am able to continue my testing in a good way. If I would have continued testing without changing to better logging, the rest of my testing would really have been a waste of time. I need to understand what I am testing!

So in conclusion, it really comes down to the point that the passport officer said, “So it is when you need to understand what you are doing”, which was a really impressive observation of my startled reasoning in the passport control at Sea-Tac airport.

#contextdriven #softwaretesting #testing #qa #cast2011