Elaborating on DET – ETDD evolving?
I got some positive response about DET that I wrote on my blog and in my CAST session proposal, so I thought I would elaborate a little on where I think this could be going. I will probably cover more hands on aspects in the coming weeks, but I really want to explain a vision I have around it first.
How about including the business stakeholders?
In my current project, I started to involve our main business stakeholders in our test sessions from the start when I got involved. And they have gotten really excited about attending once a week. So during the past weeks when we have had the focus on our mobile client we had groups of three testing together, server dev, mobile dev and stakeholder. So we are having deep product knowledge on both backend and frontend together with the business knowledge testing together.
Mission is important!
Groups get a prepared session notes template with a mission statement and the prerequisites for it. We also discuss around the mission to make it clear what is important for that group to explore during that session. Usually it has been about some area that needs specific bug-catching-sort of testing. By stating mission explicitly there is at least some chance that it will get somewhat covered. I have seen that if this facilitation and coaching is not done, business people are quite good at distracting their groups, just as developers that get excited about a specific feature they are testing and forget all about the rest of the mission.
Output of sessions
Apart from the shared understanding of the quality status of the product, the most obvious output of test sessions are bug reports of course. Depending on the setting, there are also other things that are found and discussed after the session. In the current project I found it valuable to put demand on the notes taken mapping to the specific issue types in our bug reporting tool Jira. This little change made some discussions even more valuable since we could have a more mature discussion about Bugs, Improvements, User Stories, and New features. I actually think we have great stakeholders in that perspective, since there is an understanding about not everything being bugs.
What have we acheived?
Everyone gets something out of it. Business people get to interact with the product together with the developers in a structured fashion. Developers get input on what the business people feel about the product. Feedback is direct and face to face, discussing the solutions and how stakeholders actually want it with a standpoint in what is in front of them. Everyone involved gets the feeling of what the quality of the product is at the moment. By creating this common understanding together with our business stakeholders we are also managing the expectations of quality to be shown at the demo.
The next step?
While we are having fewer and fewer bugs found during our sessions, I am considering ways of changing the types of test missions to fit into the status of the project. What value does it add to continue having test sessions regularly together in the team? Of course, we will still get the common understanding on product quality status and collaborative environment. But lets shift the focus forward. Why dont we let the whole team drive development further through carrying out exploratory testing sessions?
Introducing Exploratory Test Driven Development – ETDD
While some sessions still need to focus on testing the current product, consider shifting the mission focus sometimes. Let the mission at hand to be about finding as many future possibilities as possible and to recognize where the system is going in the future. As I pointed out above, business people in particular really like elaborating on current product in a wide fashion. Explicitly setting a mission like this also enables developers to relax about the current status and envision extensions of the product we are building.
What about Specification by example and the output?
Purpose of testing is to provide information for decision makers. This approach will certainly be hands on testing informing decisions in a direct way. But this time it is not forming decision about the past development, what you have now. Testing will inform decisions about the future, thus will be driving development on an even higher level than with specification by example. Gojko Adzic describes specification workshops in Bridging the communication gap, where the point is to go from user stories to more defined examples. He also speaks about design workshops that deal with user stories in the long perspective. Test sessions will not create usable examples, but rather be a way to elicit new user stories to build examples about later on. In reference to where Gojko describes the scope being derived from business goals, having business stakeholders involved in the feedback activity of a test session will make the scope derivation easier to grasp with the current product at hand. Testing will also give some input on current prioritization, in discovering which already known user stories need that soon need to be refined with examples.
I made a simple sketch of it through modifying this really nice picture of ATDD from Janet Gregory.
ETDD
Leave a Reply
Speaker at Lets Test Oz 2014
Speaker at CAST 2012
Speaker at Agile 2012
Speaker at Agile Testing Days 2012
Program committee member at Öredev
Happytesting 
Several years ago, Tom Poppendieck pointed out to me that the exploratory testing in Quadrant 3, business facing tests that critique the product, feed back to Quadrants 1 and 2, business and tech facing tests that support the team. We may discover missed or missing requirements, or find that a feature doesn’t really solve the problem, or we may uncover missing tests. So new stories and tests arise out of exploratory testing.
I love the idea of involving both programmers and customers more in this process. Personally I haven’t had much luck getting our customers to pair with us for exploratory testing, but I’ll keep trying!
Thanks for the information about having and sticking to a mission. This is something I often forget to do.
I think the explanation from Tom suits very well into this. In the current organizational work I am doing, the quadrants are getting more and more involved in our discussions, and I must say I am disappointed in not seeing its place in this post. Thank you for pointing that out!
For getting customer involvement in test sessions I have had some thoughts regarding another project, where the customer is not quite as happy about it. My suggestion there is to make it an extending part of the demo or next iteration planning workshops. Since we want the output to deliver into supporting the team, we just dont need to call it a test session, naming can be a winner here. =)
Thank you!
I don’t agree with Tom here. The testing activities on the right side (product critique) eventually lead to new user stories being written for future iterations. Some of their bugs also feed back to missing checks on the left side of the quadrants, too.
In general I think this reveals process issues. How come we identify missing requirements while doing exploratory testing? What can we do about it? What can we do to avoid forgetting about those missing tests in the future? Sounds like mid-iteration Retrospective time for me. Something I clearly see as one outcome happening in the debrief of an Exploratory Testing session.
I think it is interesting that you see the approach revealing overall process issues. However, I will have to think about that for a moment before I fully agree with you.
On the other hand, I think it is natural that any test activity might reveal missing requirements. However, the level of these can differ quite much. It may be that the examples defined in beginning of iteration were incomplete, but it may also be that people get these really great ideas while actually exploring the product, which might become new user stories for future iterations as you said.
I blogged about the essence of collective test chartering a while ago: http://www.shino.de/2011/01/05/collaborative-test-chartering/
I think I didn’t think about project stakeholders being included there. This is an important aspect. I think we should strive for planning the iteration together in the team. This holds for specification by example, technical implementation details, and exploratory testing charters.
Thanks for this entry.
Thanks for the comment and the link Markus. I see it as natural as planning the scope to actually define and plan for quality scope as well in the team. That way you are on the same level of understanding and expectations of the quality when delivered. This is also why I want the stakeholders to take part of actual testing as well.
I think it’s important to realize that any such diagram is only one way to approach the work. This work is done by people, and there are infinite possibilities.
E.g., any of the nodes could feed back to the generation of new stories. We learn all along the way, not just at exploratory testing.
E.g., the jump from Create A User Story to Write Customer Tests is difficult to do in one step. Rarely will a business person who proposes a story then express the acceptance criteria of that story as a test. Even more rarely will that business person have the testing skills to express that as a GOOD test.
I talk a bit about the process of getting from story to test in http://manage.techwell.com/articles/weekly/three-amigos but I think I’ll track back to Janet’s article to discuss that.
Hi George, thank you for your comment.
Models are always models, and models are most probably faulty when compared to reality. I reckon that any of the nodes could feed back to new stories, but what I really wanted to highlight in my adaptation of the diagram, was that when the exploratory test sessions have an outspoken mission of revealing future stories, this is very well possible and can give interesting results.
The jump from user story to customer tests or examples is very well described by Gojko Adzic in his book “Specification by example”. I also like your the three amigos that I have heard many different analogies for. I also think that is a big part of DET, to have the different viewpoints when testing together.
Interesting idea… not sure about the ‘X-Driven-Development’ thing, there are too many of them floating around anyway. I can certainly see that a group of people working together on exploratory testing can identify usability issues and smaller improvement, but on the projects I work on it wouldn’t really lead to significant feature extensions of major improvements – mostly to small changes.
Thank you for the comment Gojko. About ETDD, I really know what you mean and somehow I just wanted to use it as a standpoint.
Regarding the improvement suggestions we are able to identify during test sessions, I dont agree with you. I really depends on what you might be focusing on, which is stated in the mission. It is also affected by which people you have around for testing. Business stakeholders in my experience are really good at thinking big around problems that might be minor issues to fix during this sprint or the next. Those issues trigger discussions that tend to lead towards other types of thinking. I have also seen people really taking action against known limitations when they are shown in first person while testing.
Thank you.
That’s wonderful Idea.
I worked with two kinds of customer one who is not interested in features much but in bugs and other who is very much under pressure and no time for sessions.
I appreciate that you involved your customer in Exploratory Testing 🙂
The outcome of your approach is to come up with new enhancements ? I mean what was your mission statement? What test technique you used?
How about changing your test technique may result in finding bugs if your mission is to find bugs 🙂
Cheers!
Savi