Home > agile, testing > Agile Testing – Context driven testing perspective

Agile Testing – Context driven testing perspective

The context driven school of testing is a good representation of my personal normal state of mind when it comes to my profession. I really like to hang out with like-minded people that are outspoken about belonging to this community of testers. I also recognize many of the people within this community to be interested in and are practicing good Agile testing. When I started to write this post I also realized I gave it a shot 3 years ago, when I still hadn’t wrapped my head around context-driven. But although my knowledge and experience has changed a lot during that time I still think the post is valid in some sense.

Before continuing, read my previous posts in this series:
Bridging communities
Agile Testing – Traditional testing perspective
Agile Testing – Agile perspective
Agile Testing – Programmer perspective
Agile Testing – Project management perspective

Context driven testers

Similar to the Agile movement, the context-driven school has a set of principles (http://context-driven-testing.com/), followed by explanations to them for clarification.  Even this concept might give a never ending loop when it comes to the “there is no best practice principle”, when it is written in the definition which could be seen as the best practice of context driven (topic for another time). What the principles point out is the fact that for context-driven testers nothing is EVER written in stone, but can always be questioned to reveal more information. A good tester should ask as many questions as possible to reveal not only parts of the context but enough of it to be able to act on the information. It is also an outspoken goal to do the best possible testing with the context given and be able to be proud of that work. These things actually overlap some with the software craftsmanship movement (also another topic).

Although testing and software quality is the main interest of the community, many of the context driven testers are also highly interested in other areas like systems thinking, epistemology, philosophy and the like. And surprise, these topics also have their own specialized communities.

It is very important for context driven people that quality decisions are made by “management” or “product owners”. The decision is at least never on the lone testers table. “I can tell you whats wrong, but now its hands off for me, since I am not being responsible at all for the reactions of the information in my test results.” This is one of many examples how even though people are criticizing the throwing-software-over-the-wall approach, context driven community still talks about and practices testing as though it is a sport of its own within software development projects.

Agile testing?

There are plenty of things from context driven that relate to how good Agile works. The example I had in my previous post, where my colleague was asked to name the “explore > learn > adapt and then explore again” and he called it Agile, while I was in fact describing exploratory testing shows that to some extent. There is no standard answer to any question in this community, but I am quite sure that Exploratory testing is a good starting point when context driven testers are asked about what Agile testing is.

When it comes to test automation, its really just testing when created, and then its called checking. This is something that I know very many people outside of the context-driven community is having a hard time understanding or grasping. And when I have explained it to developers for example, they don’t really see the point in me trying to rename something they have been calling tests for many years. I know the value of having a good check suite that might fail on a commit, this means I can draw the conclusions myself that the newest software is not worthy my testing yet. There is good value in that, although are called unit tests by my developers.

A couple of years ago tried to explain the value of Specification by example/BDD/ATDD to some of the most prominent people within the context driven community, and I must say that these supportive ways of discovering gaps in communication before code is written did not really seem interesting at all. This is an example of proactive testing activity that I think is very important for successful Agile projects. However this and other pro-active test activities do not seem to appeal to the most extreme context driven people.

Context driven testing is a very tester/testing centric view of software development. As much as the context driven school has developed the craft of testing it still has the “reacting-after-the-fact” mentality. Doing “the best we can with what we get” (cited context-driven-testing.com) is a mentality I can’t do in projects. If something is wrong with how the project is run, I will make a change happen. It is everyone’s job to improve the situation of the project. A big part of this is test activities that support the whole team to being able to create better software in the first place.

Thoughts

If

  • Exploratory testing is the first and only thing that comes into mind when asked about Agile testing
  • You think quality decisions must /never ever/ be the testers responsibility
  • You do not acknowledge quality to be a responsibility of the whole team
  • No test results are ever good enough if they only consist of red/green or pass/fail information
  • Your view of testing is highly based on re-active activities and thinking

Is it possible to perform good Agile testing? Will you be a support or a burden to any Agile team you are on?

  1. January 16, 2013 at 11:40

    “This is one of many examples how even though people are criticizing the throwing-software-over-the-wall approach, context driven community still talks about and practices testing as though it is a sport of its own within software development projects.”

    This statement seems pretty out of step with anyone I’ve spoken to who identifies with the CDT community and as well, I think you confusing treating the act of testing as a specific skill set and discipline with trying to isolate it within a project.

    “reacting-after-the-fact” “many examples”?? Those are some generalizations that would don’t ring true with my experience with the CDT folks…just saying!

    • Sigge
      January 16, 2013 at 21:45

      Hi Keith, and thank you for taking time to react on my post.
      First of all, my series of posts along this topic is intended to be provocative and I am very well aware of exaggerations that don’t really hold true all the way. I pointed this out in previous posts, but realised that it might not be needed. And it might give the food for thoughts that I am after for my concluding posts.

      Being provocative does not mean I want to bend the facts, which is why I really want to base it on experiences and discussions I have actually had with people. Your first reaction was a tweet asking me if I had talked to CDT people before this post, and I can tell you that I consider myself part of the CDT community. I have not got this specific post reviewed, but only when I had the outline for the series layed out. And since I did not get it reviewed, I was expecting feedback like yours.

      So, for the things you are commenting on:
      I actually had discussions with people at CAST when I was presenting how I coach developers to perform exploratory testing, about their view of testing as a separate discipline. How developers maybe should not be involved in the magic activity of exploring the software.

      Also, the context driven principles do not consider but even explicitly state that “Testing groups exist to provide testing-related services. They do not run the development project; they serve the project.” It might be my English as second language here, but if I want to drive an Agile software development project with information gained in testing, I am being more than a servant.

      If you feel I am being completely out of line with facts around CDT then please tell me, since I want to learn and improve. Since CDT is not the main topic for the series, I think some generalisations and exaggerations are ok as long as I don’t offend anyone. I know a lot of context driven people as well that don’t fit into the descriptions, but I will get back to that in a later post.

      • January 16, 2013 at 21:54

        RE: “Testing groups exist to provide testing-related services. They do not run the development project; they serve the project.”

        This is not one of the context-driven principles. This is an illustration of the principles in action. IF there is a “testing group”, it does not run the project — it serves the project.

        This example does not preclude people doing testing (whether confirmatory or investigative) from being part of an integrated development team.

      • Sigge
        January 16, 2013 at 22:04

        Hi Ben,
        sorry for being sloppy about that being one of the principles. Of course it is the first “Illustration of principles in action”. And no, this does not say that testers cannot be integrated in development teams. Actually, without me stating it explicitly, my context relates to this. I (the test team of one), usually work integrated with the development team. And that is fine with CDT. However, I as a person that cares about testing and perform it to find valuable information, am still not supposed to drive the project according to CDT. As an integrated part of the team where the valuable information comes from, it is quite hard NOT to drive development.

        And this conclusion is drawn before mentioning the SBE/ATDD/BDD activities that testers can use to drive projects.

        Did I understand your comment correctly Ben?

  2. January 16, 2013 at 22:11

    RE: “It is very important for context driven people that quality decisions are made by ‘management’ or ‘product owners’. The decision is at least never on the lone testers table.”

    This depends on context. Sometimes the tester is part of the team making decisions. Sometimes the tester’s role is to provide others with information to help make such decisions. This view that it depends on context is contrary to the once-common (and still common in some circles) view that it is the duty of testers to be quality police — to be quality gates.

    That said, there are appropriate contexts for being quality police. I’ve worked in one of those: as a 3rd party validation/certification authority. However, even in this case, providing information to help people pass the tests was central to our work as testers.

    As a CDT person, I don’t think of testing as a sport of its own. Instead, It is a specialty within the sport of software development. Whether testing is done by those who specialize in programming, those who specialize in testing, or by generalists, all are playing the same sport for the same team.

    • Sigge
      January 16, 2013 at 22:59

      You are of course absolutely correct that also this relates to context. As stated in the comment to Keith, I am exaggerating from some conversations that I have had with CDT people over the last year.

      So what I am actually saying is that when it comes to certain types of arguments with some people, this might be the perceived and understood standing point. While that standing point might be true for that person, it can easily be misinterpreted as the common standpoint for CDT, if the listening person might not be engaged in the CDT community.

  3. January 16, 2013 at 22:17

    RE: “Doing ‘the best we can with what we get’ (cited context-driven-testing.com) is a mentality I can’t do in projects. If something is wrong with how the project is run, I will make a change happen.”

    Our ability to change or influence change is part of “the best we can with what we get”. If something is wrong and you can make change happen, that is part of the best you can with what you get. You got the ability to make change. However, as there is in any social group, there are likely some change you can’t make happen in a timely manner. Our scope and ability to make things better depends on context.

    • Sigge
      January 16, 2013 at 23:07

      Oh yes, the improve what you can is an important part. However, also here I would like to point out the weakness with the statement that is taken from the closing commentary. There is a strong focus on the accepting-the-fact-of-our-situation, which for me is a very dangerous path to walk. Anything else than the tester being involved in creating the situation from the start is bad.

      • January 18, 2013 at 12:36

        Sigge :
        However, also here I would like to point out the weakness with the statement that is taken from the closing commentary. There is a strong focus on the accepting-the-fact-of-our-situation, which for me is a very dangerous path to walk.

        I think you’re misunderstanding what is being proposed by the commentary. I come to totally different conclusions. Look at the next sentence in that commentary: “Rather than trying to apply “best practices,” we accept that very different practices (even different definitions of common testing terms) will work best under different circumstances.”

        If “what we get” is a context where (e.g.) session-based testing wouldn’t be the most effective practice at a given time, then we don’t say that the context must change to fit our favorite practice or notion of how things “must be”. We instead look at the context and figure out what tools to use to solve the problem here and now. The commentary is warning against context-imperialism. It’s attempting to illustrate what context-driven means.

        Another way to look at it would be to keep in mind what things were like when the principles page was first created (in 2001 I think, and according to the site it hasn’t changed since). Not a lot of cross-functional development teams in those days I’d imagine (can’t really say, I was still at the university in those days). My guess is that the majority of test teams back then where given incomplete requirements documents or specifications and asked to “verify” them. In such a general context (speculation, granted), I would think the commentary to say: “Listen here tester… I sympathize with your situation. There’s not much to go on here. But, there’s more to testing than verifying requirements. Do the best you can with what you’ve got instead of saying that you can’t test because you’ve got crappy requirements!” (incidentally, that includes leaving your desk to go search for more requirements information than you have, as well as becoming more involved in the requirements gathering and development “process”.)

  4. January 16, 2013 at 22:28

    I (the test team of one), usually work integrated with the development team. And that is fine with CDT. However, I as a person that cares about testing and perform it to find valuable information, am still not supposed to drive the project according to CDT. As an integrated part of the team where the valuable information comes from, it is quite hard NOT to drive development.

    If you are a member of a self-organizing agile team (the tribe of how), then you have as much right and responsibility to drive how things get done as every other member of the team. Now, as to driving the project or product: that usually isn’t something done by the members of the team — that is usually the role of a business/product person(s). While they do not drive the project, the members of the team can have great influence on those who do — use it.

    • Sigge
      January 16, 2013 at 23:10

      Also this argument can be turned around completely in different contexts. This is also taken from certain conversations with people to point out the extremes.

      It is the extremes of context driven testing community that I am trying to emphasize.

      Thank you for driving the topic in a nice direction!

      • January 18, 2013 at 11:59

        Sigge :
        It is the extremes of context driven testing community that I am trying to emphasize.

        The emphasis of that emphasis is what I miss in your post, Sigge. It’s a good way to start a discussion (obviously, just look at the amount of comments), but I got very confused when I read it. I’m reading it as your view of what context-driven testers in general (or even the basic principles) are saying/thinking, not as some extremists or outliers’ views.

        I’m sure there are examples of such extreme views, but none of the items in your summary (as they are stated) represents views that I think the majority of context-driven testers I know would relate to. In fact, I think it would be to misunderstand or at least seriously simplify testing (in general) for anybody to make statements like the ones in your list.

        So yeah, if it’s all an example of some extreme views out there, fine, but I misunderstood it as a misrepresentation of the context-driven (majority) view. Just saying.

  5. January 17, 2013 at 17:30

    Interesting post and comments. I think agile testing and CDT are completely compatible, and complementary, but I have experienced some of those extremes that you mention (from both groups). One of the leading members of CDT community said that ATDD is one of those ‘requirements gathering’ things (not an exact quote), and implied it had nothing to do with testing. Testing was after the fact.

    Now, if he said software testing (and he may have) – he could be right. We can actually only test software after it has been built. But, I personally think (my agile testing side coming out), that testing is more – we can test ideas, assumptions, before the software is built. That is about preventing defects, not discovering them. I don’t consider this driving the project, but collaborating to find the best way to get it done.

    • Sigge
      January 17, 2013 at 22:12

      Thank you Janet for also telling about the experience I tried to describe about ATDD, since I think the arguments are very much alike from our stories.

      About such things as driving projects, there are of course extremes regarding this as well. While driving project with information from testing might for some people be called a “lean startup” (I dont know enough about this though), the collaboration for finding information together in the team to make wise decisions together is a softer approach to it. Yet it is proactive ways of testing activities we are talking about.

  6. January 21, 2013 at 11:54

    Hi Janet,

    I’ve got some thoughts on what you wrote:

    “..But, I personally think (my agile testing side coming out), that testing is more – we can test ideas, assumptions, before the software is built. ”

    To me this is not agile. I was software-testing waterfall eons ago and the testers were involved in requirements review. There was many a whiteboard session with a developer were he went a shade paler as he realised through my questions he had overlooked something !

    What I struggle about with ATDD is that people seem to adopt it with the philosophy “Explicit == complete”.

    There are always implicit assumptions in acceptance criteria regardless of how much we try to make things explicit. Thats why we need checking AND testing when we deliver products.

    • January 22, 2013 at 08:52

      Anne-Marie, Testing requirements is not necessarily new, but I know it was (is) not common in all organizations. Agile enables the experience. It is not limited to reviewing documents, but active participation in exploring the requirements. I am glad you were able to participate eons ago. My experience was quite different.

      I have come across people that think ‘explicit = complete’ as well. It is much like some of those other extreme ideas that Sigge was talking about. Ideally, our automation with ATDD does the checking part, but I agree we absolutely need the additional testing to find out what we didn’t think about.

  1. January 16, 2013 at 01:42
  2. January 17, 2013 at 06:09
  3. January 21, 2013 at 10:50
  4. February 1, 2013 at 00:12

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: