Organization wide test strategy – Step1 – Deriving our quality values

This post is originally posted on our company blog, any feedback is greatly appreciated.

Our company has moved more from delivering individual consultant services to taking whole in-house product commitments delivered as a service to our customers. During the last months we have also been in a process of re-evaluating the core company values. Through this values work it has been clear that we want to feel proud about the quality delivered to our customers.

The transition in itself has evolved in a good agile manner, where a bigger quality initiative was a natural step forward. My first task was to investigate the current (not explicitly stated) quality values and propose an organization wide quality vision which has been lacking. This post will explain what I have done so far and the further work on it. The further work consists of defining models and activities to use in an implementation of our values.

Since quality covers many aspects of software development, it was clear at an early stage that our present status was not always that the product itself was of bad quality. Sometimes the development team think the quality is bad but the customer is happy, and sometimes the other way around. It actually goes down to managing the expectations of all stakeholders (team, customers, end users etc) and maximize the perceived quality accordingly.

Our company context

All projects themselves have their specific contexts, but there are aspects of our deliveries that are the same because of the company values and technical similarities like platforms used. When defining quality models from our values, it will be important to include these for narrowing down the overhead of re-specifying common context.

Organizational values

The ongoing initiative of re-evaluating and specifying the company values needed to be a part of the quality values for them to be aligned with the foundation of the company. There was also good timing of specifying quality values, since the organization has been active in the overall values evaluation.

Delivery organization initiative

There is ongoing work with aligning our delivery work into a streamlined delivery organization. This initiative has a high stake in my work, since I have to make something for them to pull into the overall project practices with workshops and customer collaboration. There is also work within this structuring and handling future support commitments which will be a stakeholder to the quality delivered.

Agile methods

While not committing to any single method of solving any project, the agile values are used as a foundation in all out projects that usually have a Scrum or Kanban like approach, which ever fits the current setup.

Small projects

The size of our projects reach between 3-6 people over a time of 1-4 months for most of them. This directly affects the possibility of having dedicated testers on team, which is usually only possible for the higher range of theses sizes.

Big customer variation

We work with a wide variety of customers concerning types of businesses and experience. Some are experienced with buying software development services and some not. Some have rigorous processes to follow to reach their internal quality standards whereas some just expect software to be bug-free without having to dedicate time themselves.

Products

I am not going as far as saying that all our projects are on mobile platforms, but many of them are. Usually with some kind of backend systems and integrations.

My story

This great task of mine started already before I had time to work on it. I know my company organization and I talk to my colleagues on a regular basis about the internal projects. So the thoughts about improvement ideas were all over my brain already. I started off structuring my work in a mind map that quickly became a tool for showing my progress as well as gathering some of the data. My initial intention was to publish this to show my progress, but it more became a tool for myself containing all my thoughts about big and small things I discovered. Since the interviewing of people became some sort of an audit, I felt that it was not in a state of showing off to everyone.

I realized that my work would require a lot of communication with all my stakeholders, people in all parts and functions of my organization. I had my product owner (my boss) send out an email to everyone describing my assignment with a clear message with his own intentions in it. This qualified me to more easier engage into conversations and schedule meetings with whoever in the organization about my work without having to explain myself all the time. I interviewed people on all levels, CEO, sales, team leads, project leads, testers and developers which gave me a very broad but still clear picture of our status regarding quality. At the same time I got the chance to tell them about my work and my current status. I also got permission to engage in four different projects to ask about practices, customer collaboration, project setup and even participate in their activities.

There are plenty of quality models with different focus out there, and a part of my work also became to explore and look at existing models to see if they would fit into our context. An example of exploration was to post on twitter asking for similar work, and I got a quick answer from Lisa Crispin describing their project quality vision.

@lisacrispin:

@siggeb ours (my edit: quality vision) has been ‘the best possible quality product we can deliver’ for 8 yrs, maybe sounds too vague but it has worked

With all the gathered data I started to sketch out some bullets in the form of statements of a quality ideology. The important thing to keep in mind was the level of abstraction, below company values but above project context specifics. With a high level description, the ideology over some iterations with different people became a list of 6 shorter sentences with elaborations around them appended below. Of course wording can be a problem, which is why I needed to get feedback on this on a frequent basis. So as I continued to improve these I scheduled and had discussions about the bullets with several people, also here covering the many different disciplines in the company. I also had a coaching session with Anne-Marie Charret as well as discussed the work at SWET3. I think it is important to note that by this time I have had deeper discussions or interviews about my work with more than 20 people within the company, which has about 160 in total.

After getting some more guidance by my product owner, and together with him talking to our designer it stood clear that what we really wanted at the moment was on the higher level of the scale between company values and project context. Together with the designer and some iterations more through the organization I cut the texts down to easily-grasped sentences which still contain the essence of what we need to say.

So there I ended up with a first delivery, our company quality values (version 1.0) that will be put up on the walls of the office and web. By having a common vision on delivering quality software, it will be easier to enable the goals and activities needed for it in our projects.

Jayway quality values

Using our knowledge and collective skills we deliver a level of quality that we are proud of.

We require customer involvement
Customer and development team define core quality criteria together and through this share the commitment to the end product.

We create and withhold a sustainable “Definition of done”
On all levels, there is a need to create a common understanding of the quality scope.

We allocate time for testing
Proven agile testing practices are used to achieve good test coverage. Test results reveal the quality and the status of the work in progress.

We take responsibility for the end user
We will help our customers make decisions that do not compromise end user experience or value.

We deliver professionally crafted software
To meet maintenance demands, we visualize the status of agreed upon internal product quality.

Probably, and hopefully this version will evolve when taking some further steps into realizing it.

Further work

It is important to know that behind the derived quality values there is a whole bunch of background information on all levels. This needs to be structured into context independent strategies, project quality models and concrete activities that fit our projects.

The context independent strategies have already been very much touched upon since that is pretty much were I was before cutting into the real values. They include company context specifics as well as present models like the agile testing quadrants, heuristic based test strategy model and software quality characteristics that I have found useful in further work.

On a more hands-on level, I can for example start with improving the way our project teams perform and follow-up their DET sessions, as well as raising the quality awareness of all the people involved just by pointing at our derived values and explain that thoughts behind them. That is the next step to visualize.