How about including the business stakeholders?

In my current project, I started to involve our main business stakeholders in our test sessions from the start when I got involved. And they have gotten really excited about attending once a week. So during the past weeks when we have had the focus on our mobile client we had groups of three testing together, server dev, mobile dev and stakeholder. So we are having deep product knowledge on both backend and frontend together with the business knowledge testing together.

Mission is important!

Groups get a prepared session notes template with a mission statement and the prerequisites for it. We also discuss around the mission to make it clear what is important for that group to explore during that session. Usually it has been about some area that needs specific bug-catching-sort of testing. By stating mission explicitly there is at least some chance that it will get somewhat covered. I have seen that if this facilitation and coaching is not done, business people are quite good at distracting their groups, just as developers that get excited about a specific feature they are testing and forget all about the rest of the mission.

Output of sessions

Apart from the shared understanding of the quality status of the product, the most obvious output of test sessions are bug reports of course. Depending on the setting, there are also other things that are found and discussed after the session. In the current project I found it valuable to put demand on the notes taken mapping to the specific issue types in our bug reporting tool Jira. This little change made some discussions even more valuable since we could have a more mature discussion about Bugs, Improvements, User Stories, and New features. I actually think we have great stakeholders in that perspective, since there is an understanding about not everything being bugs.

What have we acheived?

Everyone gets something out of it. Business people get to interact with the product together with the developers in a structured fashion. Developers get input on what the business people feel about the product. Feedback is direct and face to face, discussing the solutions and how stakeholders actually want it with a standpoint in what is in front of them. Everyone involved gets the feeling of what the quality of the product is at the moment. By creating this common understanding together with our business stakeholders we are also managing the expectations of quality to be shown at the demo.

The next step?

While we are having fewer and fewer bugs found during our sessions, I am considering ways of changing the types of test missions to fit into the status of the project. What value does it add to continue having test sessions regularly together in the team? Of course, we will still get the common understanding on product quality status and collaborative environment. But lets shift the focus forward. Why dont we let the whole team drive development further through carrying out exploratory testing sessions?

Introducing Exploratory Test Driven Development – ETDD

While some sessions still need to focus on testing the current product, consider shifting the mission focus sometimes. Let the mission at hand to be about finding as many future possibilities as possible and to recognize where the system is going in the future. As I pointed out above, business people in particular really like elaborating on current product in a wide fashion. Explicitly setting a mission like this also enables developers to relax about the current status and envision extensions of the product we are building.

What about Specification by example and the output?

Purpose of testing is to provide information for decision makers. This approach will certainly be hands on testing informing decisions in a direct way. But this time it is not forming decision about the past development, what you have now. Testing will inform decisions about the future, thus will be driving development on an even higher level than with specification by example. Gojko Adzic describes specification workshops in Bridging the communication gap, where the point is to go from user stories to more defined examples. He also speaks about design workshops that deal with user stories in the long perspective. Test sessions will not create usable examples, but rather be a way to elicit new user stories to build examples about later on. In reference to where Gojko describes the scope being derived from business goals, having business stakeholders involved in the feedback activity of a test session will make the scope derivation easier to grasp with the current product at hand. Testing will also give some input on current prioritization, in discovering which already known user stories need that soon need to be refined with examples.

I made a simple sketch of it through modifying this really nice picture of ATDD from Janet Gregory.

ETDD

My grandma is not very much of a technician. Yet she uses her computer and Internet every day, all day long. To do what? Listen to the Icelandic web radio of course, sharing her digital photos through email and play hearts, spider and winmine amongst other things. Any of these tasks must be enabled in the least amount of scripted steps to achieve as there is no room for exploration when grandma uses her computer. I have to do the exploration.

It has been time to upgrade her hardware for some time, and this needed to happen now. So we got her a new computer, a laptop with Win 7. But that was just the problem, since she has been used to Win XP for so long, still everything needs to stay the same. Of course there are differences, and that is what I have told her. Yet I thought I would be on the safe side on the overall experience. But hey, the devil is in the details as always when it comes to software and humans.

Volume and balance controls

As grandma is always listening to the radio, she has two speakers with really long cords that reach all over the house. Sometimes she wants to just listen to the one in her room, so she uses speaker balance in windows taskbar for this. However, the balance bar in the volume control was removed in Win 7, now it takes 8 clicks to get to this basic functionality (duh?). After some research I found out I am not the only one with this issue, although some support people manage to not understand the problems at all. But to enable the one-click easy adjustment I had to install a 3rd party workaround for it, SimpleSndVol. And that was after I had been looking for it way too long. (It even took me 15 minutes to find it again on google when writing this, since I forgot the name of it.). The tool is not pretty, but it does the job.

No balance in volume control

Balance, but not pretty

Importing images

That is a whole story of itself. How hard can it be to make good photo importing software? Apparently its really hard, since the pre-sets when attaching her Sony camera gives us three different programs to use. And they are just about useless all of them. One of them (from Sony) actually decided to pop up every time after we used it once, and the only way to get rid of it was actually removing it in Remove/Add programs.

At least the Win XP one did what you expected it to do, and nothing more of those fancy blingbling (dont want the name changing in this one) that is just annoying. With these new options it was just plain impossible to do what grandma wanted it to do. That is, just import the pictures as is into a My Pictures/<year> folder. While this might not be a very standard solution it works for her to look at all pictures in the order of a year at a time. And I really wanted to avoid her using just copy paste from camera to filesystem in the Explorer.

I ended up using the most advanced tool, that actually had the options of imports that we wanted. However, there was one catch with the screen resolution that I will get back to later.

Delete photos in My Pictures folder

While trying out the different importing tools, I created a bunch of copies of photos that I needed to trash. This is where I found a real bug in the Explorer UI.

See the pictures below to follow.

1. I am using copies of the example pictures, and I have 5 of each type.
2. I want to remove all 5 of one kind by highlighting them and press delete.
3. “Do you want to move these 5 items to the trash?” > Yes
4. Observation shows that only one of the five images have been removed. Some questions arise:
   1. How does that happen?
   2. What happens if I delete the remaining four?
   3. Was the fifth picture actually moved to trash or is it just hidden or moved somewhere else?

5 Hydrangeas

Pictures highlighted and moved to trash

Still there? Thought I moved you to trash

Some investigation showed that the pictures were actually moved to trash, but that the GUI of Explorer had not been updated accordingly. It was not possible to delete the pictures again, but they disappeared if switching to another folder and then back. I can reproduce this any time on grandmas machine but not on other i tested on with different version (enterprise)

Disappeared when going to another folder and back

Radio non-stop

So this might be the worst bug for my grandma. The radio is streamed every day, the whole day. So the show must go on, right? Well it doesnt. While on Win XP and IE, the radio never went dead while listening to show after show. But apparently with Win 7, newest flashplayer and firefox, the sound is cut every now and then. Grandma needs to get up from her chair to walk in to the computer to wiggle the controls and press play on the flashplayer again.

Since this is an issue I have not been able to see for myself, I made the experiment to make her switch to Crome when listening to the radio, and after that she at least hasn’t said anything more about it. It might have solved her problem, although I am not so sure.

If anyone has a suggestion on this, please help me out here.

Update: My brother actually came up with an idea and explored around it that might work. Involving media player and saved playlists to stream through. Lets see how it goes.

Spider, number of moves not shown

One of the favourite games is Spider Solitaire. A neat feature that gives some status of the game is the “number of moves” shown in the lower-right corner. However, this was not shown correctly, and the actual number was not shown. My first thought was to check the English version of it, since “Förflyttningar” is a longer word than “Moves”. But the lack of an English version made me give up quite quickly. It was a minor bug that was easy to live with.

Look into the lower right corner

Then I realized that this problem was somewhat related to the problem with the photo importing software. Why is that?

Resolution and visual accessibility – Root causes!

Grandma needs all the help she could get with seeing whats displayed on the screen. So the first thing I did was of course fix the resolution for her. But in the middle of all the fiddling with the different software described above It was shown that the lower resolution did just not work.

The importing software was just a 90s looking GUI smashed with controls. It also had a quite stupid layout, using a portrait view of controls that just didnt want to fit on the screen on a lower resolution. In fact the most important control of them all, the “Get photos” was hidden under the task bar.

See the "Get photos"

So I needed grandma to use a higher resolution if she ever wanted to import using this tool. What about the rest of the visual experience? I found a different control for just elements on the screen that can be shown with a bigger scale without tampering with the resolution. Voilá!

125% on some elements

And guess what, through this feature I also found the root cause of the Spider problem. The moves are actually shown when using a lower resolution. But it is NOT if using the screen element scale higher than 100%, even if I have a high resolution as well. Unfortunately we will have to keep elements visible in favor of the number of moves in Spider.

See the moves count?

Disclaimer

Just to make sure, I am a full-time mac user since many years. The testing I have done is mostly on anything BUT Windows Vista and 7 (WinXP and a variety of mobile and web platforms). So I took the tour with grandma to explore the little things in this OS.

While humans are still using the software we produce, it is still in need of testing by humans.

Hope you enjoyed my story.

There is a common practice in our company to perform Developers Exploratory Testing sessions, explained by my colleague Davor here. The cool thing is that this way of performing higher level testing has actually become accepted by our developers, and they really enjoy it.

In my current work of developing our organization wide practices for quality, I have made a deep dive into how DET is carried out on a regular basis. What I have seen is that DET is accepted and acknowledged as a valuable practice, however it is not really carried out in its full potential. There are many details and aspects of it to work on, especially regarding reporting and follow-up.

The other day I was asked to help one of our teams with a DET session.As they are familiar with the approach, I wanted to expand their view on its potential through light weight note taking and showed them an example of a very basic session template I like to use. I also explained the intentions of having bugs and issues separated as well as how I do with test notes and the summary. We decided to use a whiteboard instead and put stickies with issues and bugs on there for visualization. We spoke about the mission and decided on two different areas to focus on. The app is an iOS app where a new component is used in the app core functionality for iOS5.

The four of us paired into two teams with some different iOS devices and tested for about 45 minutes. We found quite a few issues and bugs that were put on the whiteboard, but not so much notes about which areas we covered. Of course, by looking at the issues you might get a hint of what areas have been covered, but how much? We also quite quickly noted that we had to tag each issue with the test environment (iOS version and device), since they differed quite a lot. Most of the things found were considered bugs to fix, which is also not always the case in all projects or settings. It usually depends on the customer and the relationship to them, as well as having a decision maker on this present during testing. In this project, the scrum master/tech lead which was present knows the customer well enough to make those judgements.

But what about the learnings?

After the session debrief I got into a meta-debrief, discussing the outcome compared to my introduction about reporting. At first there was not very much understanding of why having test notes is valuable, but this shifted a little during our discussion. “While our smaller projects change that rapidly, even storing test notes might be waste” is a motivation I will take with me. I explained a common scenario of being asked what was tested and with what configuration. It could also be valuable for future test sessions to know what parts of a functionality was covered and when this was tested before. I also like to emphasize the possibility to explain current functionality which might not be explicitly stated in requirements.

The team was happy with the experience and I got some more input on how to improve the value of our DET sessions. I am not going to abandon the reporting, but I need to find a way of combining the fun part of interacting between pairs through the collaborative setting of the whiteboard. This actually helped getting attention on the issues in the debrief.

And then the developers actually got their own ideas of what a session note taking tool should look like to suit their needs, this is how they sketched it out after the session. I would explore some other possibilities before building our own for example RapidReporter or SessionWeb, but it is really cool that the meta-discussion could trigger the further thinking about the problem. Other aspects of the problem with collective knowledge transfers I wrote about here.

Our company has moved more from delivering individual consultant services to taking whole in-house product commitments delivered as a service to our customers. During the last months we have also been in a process of re-evaluating the core company values. Through this values work it has been clear that we want to feel proud about the quality delivered to our customers.

The transition in itself has evolved in a good agile manner, where a bigger quality initiative was a natural step forward. My first task was to investigate the current (not explicitly stated) quality values and propose an organization wide quality vision which has been lacking. This post will explain what I have done so far and the further work on it. The further work consists of defining models and activities to use in an implementation of our values.

Since quality covers many aspects of software development, it was clear at an early stage that our present status was not always that the product itself was of bad quality. Sometimes the development team think the quality is bad but the customer is happy, and sometimes the other way around. It actually goes down to managing the expectations of all stakeholders (team, customers, end users etc) and maximize the perceived quality accordingly.

Our company context

All projects themselves have their specific contexts, but there are aspects of our deliveries that are the same because of the company values and technical similarities like platforms used. When defining quality models from our values, it will be important to include these for narrowing down the overhead of re-specifying common context.

Organizational values

The ongoing initiative of re-evaluating and specifying the company values needed to be a part of the quality values for them to be aligned with the foundation of the company. There was also good timing of specifying quality values, since the organization has been active in the overall values evaluation.

Delivery organization initiative

There is ongoing work with aligning our delivery work into a streamlined delivery organization. This initiative has a high stake in my work, since I have to make something for them to pull into the overall project practices with workshops and customer collaboration. There is also work within this structuring and handling future support commitments which will be a stakeholder to the quality delivered.

Agile methods

While not committing to any single method of solving any project, the agile values are used as a foundation in all out projects that usually have a Scrum or Kanban like approach, which ever fits the current setup.

Small projects

The size of our projects reach between 3-6 people over a time of 1-4 months for most of them. This directly affects the possibility of having dedicated testers on team, which is usually only possible for the higher range of theses sizes.

Big customer variation

We work with a wide variety of customers concerning types of businesses and experience. Some are experienced with buying software development services and some not. Some have rigorous processes to follow to reach their internal quality standards whereas some just expect software to be bug-free without having to dedicate time themselves.

Products

I am not going as far as saying that all our projects are on mobile platforms, but many of them are. Usually with some kind of backend systems and integrations.

My story

This great task of mine started already before I had time to work on it. I know my company organization and I talk to my colleagues on a regular basis about the internal projects. So the thoughts about improvement ideas were all over my brain already. I started off structuring my work in a mind map that quickly became a tool for showing my progress as well as gathering some of the data. My initial intention was to publish this to show my progress, but it more became a tool for myself containing all my thoughts about big and small things I discovered. Since the interviewing of people became some sort of an audit, I felt that it was not in a state of showing off to everyone.

I realized that my work would require a lot of communication with all my stakeholders, people in all parts and functions of my organization. I had my product owner (my boss) send out an email to everyone describing my assignment with a clear message with his own intentions in it. This qualified me to more easier engage into conversations and schedule meetings with whoever in the organization about my work without having to explain myself all the time. I interviewed people on all levels, CEO, sales, team leads, project leads, testers and developers which gave me a very broad but still clear picture of our status regarding quality. At the same time I got the chance to tell them about my work and my current status. I also got permission to engage in four different projects to ask about practices, customer collaboration, project setup and even participate in their activities.

There are plenty of quality models with different focus out there, and a part of my work also became to explore and look at existing models to see if they would fit into our context. An example of exploration was to post on twitter asking for similar work, and I got a quick answer from Lisa Crispin describing their project quality vision.

@lisacrispin:

@siggeb ours (my edit: quality vision) has been ‘the best possible quality product we can deliver’ for 8 yrs, maybe sounds too vague but it has worked

With all the gathered data I started to sketch out some bullets in the form of statements of a quality ideology. The important thing to keep in mind was the level of abstraction, below company values but above project context specifics. With a high level description, the ideology over some iterations with different people became a list of 6 shorter sentences with elaborations around them appended below. Of course wording can be a problem, which is why I needed to get feedback on this on a frequent basis. So as I continued to improve these I scheduled and had discussions about the bullets with several people, also here covering the many different disciplines in the company. I also had a coaching session with Anne-Marie Charret as well as discussed the work at SWET3. I think it is important to note that by this time I have had deeper discussions or interviews about my work with more than 20 people within the company, which has about 160 in total.

After getting some more guidance by my product owner, and together with him talking to our designer it stood clear that what we really wanted at the moment was on the higher level of the scale between company values and project context. Together with the designer and some iterations more through the organization I cut the texts down to easily-grasped sentences which still contain the essence of what we need to say.

So there I ended up with a first delivery, our company quality values (version 1.0) that will be put up on the walls of the office and web. By having a common vision on delivering quality software, it will be easier to enable the goals and activities needed for it in our projects.

Jayway quality values

Using our knowledge and collective skills we deliver a level of quality that we are proud of.

We require customer involvement
Customer and development team define core quality criteria together and through this share the commitment to the end product.

We create and withhold a sustainable “Definition of done”
On all levels, there is a need to create a common understanding of the quality scope.

We allocate time for testing
Proven agile testing practices are used to achieve good test coverage. Test results reveal the quality and the status of the work in progress.

We take responsibility for the end user
We will help our customers make decisions that do not compromise end user experience or value.

We deliver professionally crafted software
To meet maintenance demands, we visualize the status of agreed upon internal product quality.

Probably, and hopefully this version will evolve when taking some further steps into realizing it.

Further work

It is important to know that behind the derived quality values there is a whole bunch of background information on all levels. This needs to be structured into context independent strategies, project quality models and concrete activities that fit our projects.

The context independent strategies have already been very much touched upon since that is pretty much were I was before cutting into the real values. They include company context specifics as well as present models like the agile testing quadrants, heuristic based test strategy model and software quality characteristics that I have found useful in further work.

On a more hands-on level, I can for example start with improving the way our project teams perform and follow-up their DET sessions, as well as raising the quality awareness of all the people involved just by pointing at our derived values and explain that thoughts behind them. That is the next step to visualize.

A nice write-up by Rikard Edgren with picture of attendees and the abstracts can be found here.

This last weekend I attended SWET3 (Swedish Workshop on Exploratory Testing), which is a peer conference with a clear focus on aspects within context driven testing more than just exploratory testing in itself. This is my second time SWETing as I was also at SWET2, and I really enjoy the format as it is personal environment with a bunch of really experienced people. This also gives higher dimensions on the discussions about testing than you normally have for example in customer assignments.

The theme of the weekend was “Teaching testing”, with all the different angles that can give. My own topic that I submitted as an abstract was about my lectures for university and higher vocational students. The embryo for the session is in the form of this mind map. Note that I did not present or discuss this.

My abstracted topic

As I am working on getting more used to working with mind maps, I wanted to share the session discussions from the conference as mind maps as well. Since SWET2 I have actually spoken to quite a few people on using mind maps for test progress communication as shown by Christin Wiedemann when she demonstrates xBTM.

The first session at SWET3 was Rikard Edgren that actually has a similar context, but where he is actually teaching a whole course on the higher vocational education in Karlstad together with Henrik Emilsson. With a high degree of practice and interaction with open source software for teaching purposes I am after the session actually quite sure the testers that come out of that program become very good at testing.

Testing is never better than the communication of the results of it

Rikard Edgren

Second session was Johan Jonasson about the BBST courses and how the situation of teaching online works. However hard it is to teach and learn online, I actually think after the session that a bigger challenge is the cultural differences between the attendees and instructors. Here are my notes.

…while the video material could be hard and daunting to take in, the actual take-aways are the interactions and experience based assignments.

Johan Jonasson

Before dinner it was time for lightning talks. I spoke about my current work in “Building an organization wide heuristics based quality model”. Notice that the name is in working, and I am writing a blog post on that work as we speak. Anyway, I will share my talk as mind map as well.

My lightning talk - Organization wide heuristics based quality model

The third session was Simon Morley speaking on mindset changes. There was a lot of speaking about gut feelings, and the need for a better word for it to be sustainable in a professional and academic environment. I found this when searching for it, but no better wording. Michael Bolton suggested “HEURISTICS” and sighed on twitter, while Zeger van Hese suggested ØRISTICS as the scandinavian version of it. =) Although I am not sure that was the context of gut feelings we were discussing.

If projects lead are involved in root cause analysis, it usually ends up in updating process model documents

Simon Morley

Thank you all for a good weekend of testing thinking and discussions.

Tags: #softwaretesting #testing #swet #swet3 #exploratory testing

The girl in front of me had an interesting passport with a lot of Visas in it and a complex citizenship history, so the officer questioned what she was doing in US. She was an astronomer and they got into talking about her theories. The officer had apparenty read something about these things and she responded as if it was relevant as well. Then it actually turned out that what the officer had mentioned was very relevant to what her research was about, and they had a deep conversation about black holes and the universe. I was very impressed by how much this man knew about astronomy, but did not think I would need to explain myself and my work as much as her. What are the odds that he had read or knew something about software testing.

The officer asked me the usual questions, where I was going, the conference name and venue, what my company was called and so on. Then upon seeing my Chinese Visa from 2007 he started a conversation in Chinese, with quite good pronunciation. I could answer him and ask him some question which he really could understand. That was cool but still a little creepy. Then he started to ask about the conference, and I could never have imagined this in beforehand.

Officer: So what is the conference about?

Me: It is about Software testing.

Officer: But isn’t that area quite big, can you be more specific?

Me: Well, the theme is context-driven testing.

Officer: And what does that mean?

Me: It means that instead of trying to apply all or any specific testing practices to your testing at hand no matter what the product, you just try to use some that is needed for the specific context. (a little startled by the coming depth of this conversation)

Officer: So, it is really about knowing and understanding what you are doing when testing software?

I was really impressed with the quickness and accuracy of that conclusion, especially in the setting that this guy is a police officer inspecting passports at the airport. Making the leap from my quite startled and very brief answer to that question towards a thoughtful shorter version of it that was just spot on. And really, his answer reflects quite good in James Bachs and Cem Kaners definition principle ”6. Good software testing is a challenging intellectual process”.

When thinking about this discussion for a couple of days, it came to me how right he really was.

I see context-driven testing as when you apply the best and most suitable approaches to testing according to your current understanding of the environment and prerequisites. This means that when I have some understanding of a product and its features, I can apply some known heuristics and suitable oracles in my testing.

Example: If I have knowledge about the logging mechanism of a product, I might use those features as part of my approach when revealing information about product status and its environment. There might be oracles in the form of “error codes” that are logged in certain scenarios.

If I at some point realize that my understanding of the current area of testing is limited, I have to hunt for more information to be able to get a better understanding of those areas.

Example: The logging mechanism seems to fail randomly during some of my testing of a certain feature. Doing some experiments and exploration through the logging documentation I finally realize that under certain conditions there is need to use other parts of the logging tool that is not used anywhere in the product. 

Through my expanded understanding of the logging I am able to continue my testing in a good way. If I would have continued testing without changing to better logging, the rest of my testing would really have been a waste of time. I need to understand what I am testing!

So in conclusion, it really comes down to the point that the passport officer said, “So it is when you need to understand what you are doing”, which was a really impressive observation of my startled reasoning in the passport control at Sea-Tac airport.

#contextdriven #softwaretesting #testing #qa #cast2011

Then I would like to thank for the fun competition James set up, it was really a learning experience and in retrospect I would maybe have put even more effort in the learning parts throughout the exercise. This, and my ability to concentrate may also have been impaired because of the time of day (after 6 pm after long conference day) and that I was still jet-lagged. But enough whining now, here is the story and my learnings that hopefully will help me make better decisions in the future.

First out, I was going to enter the competition myself, but then I got to know Jay and Sandra and we formed Happy purples (even written in purple on the registration flip chart). It felt quite good to be a team instead of working solo on this. Especially since we had the Miagi-do all-star team on the table next to us, and they were plenty.The start of the competition could have been better. While we knew we were already crippled with having two computers in a team of three (with two iPads as complement), we thought we would be ok since we were told we would get a web-link to the software to test. Here was a first set-back when we realized that it was a Windows application to test on our two macs and iPads. Dual-booted Win XP worked for one of the laptops while my own Parallells Win 7 instance had not been running for a while and did not really want to cooperate.

It is important to set the stage in a group before starting solving a problem like this competition. I think our group would have benefited from at least some structure in our work, and I think during the heat of action we underestimated these needs. It is especially important to have some kinds of statement of work when being a group of more than two people. This was shown over and over in all activities we performed, yet we did not cope with it at all, i guess because of the stress and not realizing the need for it in the short term of the competition. We even observed the structured way the Miagi-do team set up their SBTM on flip charts.

About the worst bug report award

The two bugs mentioned by James are of course questionable. The tooltip inconsistency was actually filed with an assumption of consistency, but later we got to know from the developer that those type of bugs were not important. So, getting feedback on bugs from developer did actually change the type of bugs we filed later on, we improved. But it was still not possible to edit or delete already filed bugs.
On the “slow download of application under test”-bug, it is related to a discussion I had with Michael Bolton on my blog a while back. I would consider it a testability issue to not get the application delivered in a timely fashion. Especially if we were going to get more deliveries of it during the competition time frame. In retrospect, it might seem more rational NOT to make a bug report on this, but only include it in our final test report, as it delayed our testing quite a bit.

This post was also posted on our team blog.

Tonight I was reminded on my hectic spring in some tweets with Michael Bolton. Ended up being pointed to his series about the whole team approach written in January. I stumbled upon a comment from myself in Zegers post so I am sure I have read these things, but have no memory from it. Michaels posts are marked as read in my feed reader as well.

Now it is really time for some well-needed vacation, which I will spend at the World Scout Jamboree, that actually is in Sweden for the first time. Actually it will be located only one hour drive from my home, and there will be about 40000 scouts there. It is a pity, I will leave the jamboree a day earlier, just to make it for CAST. I hope to have switched to vacation mode by then, but will have to sharpen up for the conference again.

And CAST, soo much looking forward to it that I actually have almost neglected the scout camp in terms of looking forward to an event. The people I will meet are the ones I read blog posts and tweets from. I decided not to go for an emerging topic of my own, just to be able to concentrate on conferring and learning. And I really hope to meet and talk to everyone there. I am quite bad at remembering faces, so please tell me your twitter id. =)

If you are going to be there, I am going to stay some days afterwards, up for a hangout and some test talk. Or as sightseeing company.

I also hear that there are seats left

How do my notes look?

I have already described how I went about using cloud docs for notes here. In my current project I am not allowed this freedom of putting information outside the organizations infrastructure. But the difference is also that my test subjects are more easily available from my own laptop. Also, in the current setting where everything that is not my own workspace are in bigger management tools (missions, scripted tests, defects etc.), I just use simple text files for my own notes. They are always tagged with date and domain wording in the file name, and stored in a simple folder structure of notes. Similarly I keep my defect reports in text files in another folder together with logs, screenshots and similar data.

With this simplicity, I am able to search within my files for the domain wording I am looking for. And as I stated, anything outside of my personal working environment is supposed to be stored in our tools.

What value do my notes deliver?

What I have done, is actually make note taking the natural way of recording what I do and how I think about it while doing it. The notes are always a good start when creating defect reports or give status updates. The times that I look back at the notes, is usually because of an issue that comes back after fixing, or when I get questions like “didn’t you test that?”. Those two scenarios are the most common, although they don’t really happen very often. For solving those situations, my notes are usually enough as they are, with dates and usually describing the features tested with the domain specific words and contexts. But the bigger picture is missing, how should I go about getting payoff in the longer perspective? How do I make my notes more valuable for other scenarios? And even better, what values would collective note taking add?

Collective note taking

I would like to think outside my own context for a moment. Remove the test management tools around my own working environment and add four people to my personal working space. The project is another, its smaller and more light-weight. Development cycle is two weeks from specification to working software. The software is changing rapidly and there is no point in having any testing specified in predefined test scripts. We do not yet have any automated Specification by example/ATDD running, but we have our unit and integration checks running on our CI. We run sessions of exploratory testing twice a week according to Davor Crnomats DET, an agile adaptation of SBTM. Where do you think the most valuable insights on the software status can be found?

The most valuable resource here would be good note-taking during test sessions. I think it is really important to acknowledge that testers not only find bugs, they find risks, curios, artifacts, problems, models, frames etc as James Bach and Michael Bolton describes. More importantly, when testing software you find the status of the software by adding up all those things in a snapshot.

This is where I would like to go further with the notes gathered during test sessions and session reporting. Session reporting according to DET is mostly about summarizing the bugs and issues you have found and discuss in the team about them. According to SBTM, there are also some session metrics that may be interesting to collect, I wont go deeper into those. However, I would like to keep the notes created for knowledge about the status of the software as well, with the current behaviour of it described through the gathered knowledge base of the notes.

Clean information

With a starting point in my own notes and how they look at the moment, I would like to share some key points I believe are essential for being able to derive more value from the collected notes. Looking at these from a developers perspective might suggest these are good rules for programming as well, right?

1. Contextual project agreements on feature names
As with all wording used in projects, it is good to have common dictionary for key areas. This will enable easier searching within, and easier understanding of each others notes.

2. Common structures
As with the contextual agreements, having common structures on how the notes are written will also enable easier sharing.

3. Check in notes into a common repository
Everyone should be able to use the knowledge from past test sessions. Some would say that this is where a wiki suits better, but I believe in using text files for the simplicity and being able to store notes with code.

Gathering knowledge and behavioral status

When note taking gets more mature in the project team as well as running test sessions with more focus and discipline, this is where the extra value can be created. I think it is important to acknowledge a quote from a developer colleague (who probably cited someone else),

developers are lazy,

by the means that there should not be done any work that is not delivering value. This can be both good and bad in note taking. While information needs to be rich on context, it also needs to be concise. This is why I think it is important to really consider what extra value your own project needs from testing, and create that from the notes. Here are some of my takes on that.

1. Create Specification by example/ATDD/BDD scenarios
If every test session would result in at least one scenario that has not been specified before, it would enable visualizing status even more towards the team and stakeholders. Since the testing is about telling about the status of the software, it can be an important eye opener for any customer to show them how the software actually works in areas that have not been explicitly specified before. It will also show off a richness in the application that until then is unknown, because of those hidden features that were gathered from that library that you anyway used for other things.

These scenarios would of course be used when project gets mature enough to automate these, for getting a more powerful checking coverage for regressions, especially for those missions that were run to verify a bug fix.

2. Create FAQ or other documentation
If creating scenarios is too much, why not make it a part of every test mission to create a new section to the softwares’ FAQ or guidelines. This is also a way of showing the product status in areas that have not been explicitly specified. Also, if it is explicitly stated that some of the session time should go to refining the documentation in the areas tested, that documentation will get really valuable and correct information. You will of course also save the time of creating those parts, since you would anyway have spent time to explore this within the documentation tasks. Thus you would have less time doing the same things.

Summing up

My goal with these suggestions is to create more value from the gathered notes. I want to explore how far you can go with the co-creation and collective note ownership on a team and project level.

Somewhere along this way I thought of note taking tools to be the continuance, but I really want to explore any other ways before I start going in that direction. How far can you go without this aid?

How strict are the notes created until they are called documentation? I would say that notes are made on the fly in a certain way to add value directly. This is why I dont want to call them documentation, it is a heavy word.

I am quite fond of the creating scenarios as part of test session reporting and will try it out myself the first time I get the chance. Do you already have experience by this? Do you have other suggestions on how to get more value from your notes? Please tell me.

Curios

Already in 1883 George Abbott was thinking about the relevance and values of good collective notes when working many doctors with a patient, that carried the paper of notes around.

Such papers I have found to be invaluable for both private and hospital practice, as they involve but little real additional work, and enable me to recall as a glance the whole antecedents for a case.

Where are we now?

“You can never become a great tester if you are always optimistic!”

The sentence still rings in my ears, but I still think I am able to become just that some day, a great tester. Being a happy person has always been an advantage for me, especially at work I might say. I always meet people with a smile and happy attitude, and this usual reflects back on the way they treat me. This is why I really value getting a good start especially with the developers, whose code I will be testing.

Test results, especially those that are given as feedback to developers are rarely happy news. This is one reason I usually try to find positive things for feedback as well, but they should never hinder my bug advocacy.

As a side-note, I actually also used the /concept/ of /happy testing/ in my first big software project in university. This project class was all about experiencing the waterfall process model from a real perspective. I was test manager of a team of 4 testers in a project setting with 18 people. The most funny thing was that when all the specified tests had run, we did some exploratory testing, where we found a last bug. But since this kind of testing had not been specified in any predefined plan, we had to introduce the concept happy testing. =)

So, as long as I deliver my value as a tester professionally, I think it is really valuable having a positive attitude to my surroundings. It will make the surroundings more recipient to the information I have to give, may it be positive or negative.

Happy testing!

Update: Having a discussion about the post with James Bach, I need to clarify this. I think I am able to control the scope of my optimism, which means that the context of testing a product is not subject to the optimism. As James stated, optimism and critical thinking will take each other out because of their nature.

My reflection on this is that as long as I somewhat can control scope of optimism, the critical thinking I perform on the product is sufficient for good testing. I might mis out on some aspects, but I think that they are small enough to be marginalized as other things that I would miss for other reasons.