Collective note taking – More value from your test notes?

For quite some time now I have struggled with making note taking a natural part of my personal progress while testing. And well, I can say that it has really made impact on many other aspects of my work in other situations as well. I am actually quite proud to say that it has made great impact on how I perform in general, and how easy it is to make follow ups when done with anything. Now I would like to take this some steps further to explore how my notes can give more value to the whole team in a project setting, both mine and the teams’ collective notes.

How do my notes look?

I have already described how I went about using cloud docs for notes here. In my current project I am not allowed this freedom of putting information outside the organizations infrastructure. But the difference is also that my test subjects are more easily available from my own laptop. Also, in the current setting where everything that is not my own workspace are in bigger management tools (missions, scripted tests, defects etc.), I just use simple text files for my own notes. They are always tagged with date and domain wording in the file name, and stored in a simple folder structure of notes. Similarly I keep my defect reports in text files in another folder together with logs, screenshots and similar data.

With this simplicity, I am able to search within my files for the domain wording I am looking for. And as I stated, anything outside of my personal working environment is supposed to be stored in our tools.

What value do my notes deliver?

What I have done, is actually make note taking the natural way of recording what I do and how I think about it while doing it. The notes are always a good start when creating defect reports or give status updates. The times that I look back at the notes, is usually because of an issue that comes back after fixing, or when I get questions like “didn’t you test that?”. Those two scenarios are the most common, although they don’t really happen very often. For solving those situations, my notes are usually enough as they are, with dates and usually describing the features tested with the domain specific words and contexts. But the bigger picture is missing, how should I go about getting payoff in the longer perspective? How do I make my notes more valuable for other scenarios? And even better, what values would collective note taking add?

Collective note taking

I would like to think outside my own context for a moment. Remove the test management tools around my own working environment and add four people to my personal working space. The project is another, its smaller and more light-weight. Development cycle is two weeks from specification to working software. The software is changing rapidly and there is no point in having any testing specified in predefined test scripts. We do not yet have any automated Specification by example/ATDD running, but we have our unit and integration checks running on our CI. We run sessions of exploratory testing twice a week according to Davor Crnomats DET, an agile adaptation of SBTM. Where do you think the most valuable insights on the software status can be found?

The most valuable resource here would be good note-taking during test sessions. I think it is really important to acknowledge that testers not only find bugs, they find risks, curios, artifacts, problems, models, frames etc as James Bach and Michael Bolton describes. More importantly, when testing software you find the status of the software by adding up all those things in a snapshot.

This is where I would like to go further with the notes gathered during test sessions and session reporting. Session reporting according to DET is mostly about summarizing the bugs and issues you have found and discuss in the team about them. According to SBTM, there are also some session metrics that may be interesting to collect, I wont go deeper into those. However, I would like to keep the notes created for knowledge about the status of the software as well, with the current behaviour of it described through the gathered knowledge base of the notes.

Clean information

With a starting point in my own notes and how they look at the moment, I would like to share some key points I believe are essential for being able to derive more value from the collected notes. Looking at these from a developers perspective might suggest these are good rules for programming as well, right?

1. Contextual project agreements on feature names
As with all wording used in projects, it is good to have common dictionary for key areas. This will enable easier searching within, and easier understanding of each others notes.

2. Common structures
As with the contextual agreements, having common structures on how the notes are written will also enable easier sharing.

3. Check in notes into a common repository
Everyone should be able to use the knowledge from past test sessions. Some would say that this is where a wiki suits better, but I believe in using text files for the simplicity and being able to store notes with code.

Gathering knowledge and behavioral status

When note taking gets more mature in the project team as well as running test sessions with more focus and discipline, this is where the extra value can be created. I think it is important to acknowledge a quote from a developer colleague (who probably cited someone else),

developers are lazy,

by the means that there should not be done any work that is not delivering value. This can be both good and bad in note taking. While information needs to be rich on context, it also needs to be concise. This is why I think it is important to really consider what extra value your own project needs from testing, and create that from the notes. Here are some of my takes on that.

1. Create Specification by example/ATDD/BDD scenarios
If every test session would result in at least one scenario that has not been specified before, it would enable visualizing status even more towards the team and stakeholders. Since the testing is about telling about the status of the software, it can be an important eye opener for any customer to show them how the software actually works in areas that have not been explicitly specified before. It will also show off a richness in the application that until then is unknown, because of those hidden features that were gathered from that library that you anyway used for other things.

These scenarios would of course be used when project gets mature enough to automate these, for getting a more powerful checking coverage for regressions, especially for those missions that were run to verify a bug fix.

2. Create FAQ or other documentation
If creating scenarios is too much, why not make it a part of every test mission to create a new section to the softwares’ FAQ or guidelines. This is also a way of showing the product status in areas that have not been explicitly specified. Also, if it is explicitly stated that some of the session time should go to refining the documentation in the areas tested, that documentation will get really valuable and correct information. You will of course also save the time of creating those parts, since you would anyway have spent time to explore this within the documentation tasks. Thus you would have less time doing the same things.

Summing up

My goal with these suggestions is to create more value from the gathered notes. I want to explore how far you can go with the co-creation and collective note ownership on a team and project level.

Somewhere along this way I thought of note taking tools to be the continuance, but I really want to explore any other ways before I start going in that direction. How far can you go without this aid?

How strict are the notes created until they are called documentation? I would say that notes are made on the fly in a certain way to add value directly. This is why I dont want to call them documentation, it is a heavy word.

I am quite fond of the creating scenarios as part of test session reporting and will try it out myself the first time I get the chance. Do you already have experience by this? Do you have other suggestions on how to get more value from your notes? Please tell me.

Curios

Already in 1883 George Abbott was thinking about the relevance and values of good collective notes when working many doctors with a patient, that carried the paper of notes around.

Such papers I have found to be invaluable for both private and hospital practice, as they involve but little real additional work, and enable me to recall as a glance the whole antecedents for a case.

Where are we now?