Archive

Posts Tagged ‘test’

SAST and Tester as business analyst

June 15, 2010 3 comments

I was at a SAST Öresund meeting last thursday. It was time again to hear a little more about the knowledge of agile within the testing community in the south of Sweden. I am sorry to say that other times I have gotten somewhat disappointed, but this time I felt that people are actually starting to get it.

The first presentation, I am not going to say so much about. It was about using Scrum in an outsourced test project. The number of questions that were thrown at the end of the session showed that there was a big number of uncertainty and inconsistency for the presentation to gain acceptance in the audience of testers. It also proves my point on the agile maturity at SAST. In my point, and probably more with me when thinking of all the questions, I am not so sure the project was very agile at all actually with its surrounding managemental and additional processes used to achieve the goals for the poor Scrum team.

Then there was Thomas Stjern from Inceptive, that presented the view on requirements management in the agile context. When listening to Thomas, it struck me that he was almost exactly describing some things that are in the book that I am reading at the moment, Bridging the communication gap by Gojko Adzic, about having User stories on a really high level. Then they become business rules for development and then Fitness tables for acceptance testing during development. I will cover more on this in future posts.

What was more exciting for me about the presentation, was the view of the roles of the requirements/business analyst and the tester. As Thomas referred to scrum teams of size 4-6 people, the role of business analyst and tester could obviously and preferably  be the same person. By doing that, you will never have knowledge/information drop when creating tests from requirements. Going back to Gojkos book, it makes even more sense to do this, as he talks about the requirements written as acceptance tests in the first place.

In some projects I have been in, the business analyst tasks have been carried out by the product owner or scrum master person. This has always created a gap in knowledge about the domain and the customers true wishes between this person and me as a tester. In my current project though, I have actually grown the amount of trust from my project stakeholders, that I was a part of the requirements discussions before any of the features in a the new release were started to be developed. Compared to the other projects I have been involved in at the same client, I have always gotten involved sometime during development before  a release. As this is my 6th product release of a product here, I have every time gotten closer and closer to the initial meeting of the release. Now I have caught up and I see this as a big difference to the first release I had. This time, I have the overall understanding of the design and am able to focus more on the actual testing and quality of the developed product, during development. In contrast to when I need to gather fragments of old discussions together and then analyse what is actually wanted from the product by the customer, and do this maybe after everything has been developed.

Somehow, this also relates to my post on Dare (not) putting your product in a testers hands?. Will you invite your tester to the next design/requirements/scope workshop or meeting? I can assure you that a good tester in that position will find the bugs in the discussions. See my post on that here

To ask the (right) questions

April 16, 2010 5 comments

Last week I had a really interesting and motivating dinner with James Bach, who told me some about his recent ideas and research in the testing domain. One of them was about how to visualize your product and its state of quality by letting it be represented by three different circles, the imagined product, the specified product and the actual product. We had some loose discussions about how these three circles were connected and what kind of ecosystem the were in, but that is another much greater story. However, the cool thing about this representation is that the last week I have been putting it into different contexts and found some interesting mappings.

This weekend I attended the Startup weekend in Lund, which gathered a lot of entrepreneurs combined with a mix of business and development people to a 54 hour work weekend. The goals were about creating the best business ideas from the business pitches made on the friday. The rest of the weekend was all about developing products, business plans and key selling points to be able to care for realization of the ideas. I was at this event as a part of my company Jayway being a sponsor. For my part, I was there as a mentor being able to put my professional perspective as software consultant and tester to the different teams. I tweeted about my participation as I was going to give the right questions. Of course I got a spot on reply from Olacsc about how I knew what was the right questions. It got me thinking and I found out that it was worth much more than a 140 char reply (this post).

With the circles in context from earlier in the week, it really got me thinking about some common contexts. Of course I had to adapt my questions as a mentor to the specific project at hand. All of the project ideas where different in both domain and their progress before the weekend, so any context could be taken into account. I am going to tell you about some of them.

We had the Momo project, where newcomers to Sweden were supposed to find the most relevant information as fast and smooth as possible, which evolved from being completely imagined as a product to becoming more and more specified and actual during the whole weekend. In the end, there was an actual business plan and specified sketch-up of the technical solution, whereas there still were some loose imagined parts of the complex solution still in the minds of the project group. On the first day, my questions were a lot about specifying the imagined product onto paper and in the discussions that led to consensus. There was a session where the initial design of the website was created. When I put some perspective on the design with some questions about different personas using the website, the effort was focused into a more specific setting.

We have the search engine group Geistr, where the product was almost finished. A real-time semantic search engine, that produced quality data from tweets and presenting them in an order of relevance and re-tweets. My questions here was some about the search itself and also about the business case. When asked about spam handling, I got the answer that this had not actually been thought of. So how do you test spam in a real-time semantic search engine? Well, a search for the word “sex” actually gave realistic and useful results. Another search for “Viagra” did however conclude that no one was actually tweeting anything useful about Viagra at the time.

I got into a real user role when questioning Stella Star, a platform for user-generated design that is put into production if enough people bound themselves to buy the finished product. How does a user really feel about crediting their credit card knowing that the product is not even realized yet? They got a headache thinking about this use case, and actually had a better solution with an alternative way when presenting their case.

There was the Locate Me group with an iPhone app idea. By actually performing a user testing session with a paper UI, I questioned the functionality of the app, making it visual that some extra functionality was needed and even some important aspects of the original app-idea.

As you can understand, the imagined product in the different teams could be quite diffuse, with not very clean borders, while others further in their progress were not as diffuse. Since many of the ideas were new to some team members, in some cases apart especially, since the idea originator had been thinking about for some time, I would say that the imagined product circle could have any shapes; small/big/diffuse etc.

If I would treat the specified product circle as I imagine it (*smile*), it is all about what has been agreed on and documented in some way. Written words and conversations where the consensus has been agreed on and narrowed down to a realistic view that is actually possible to document. It is important to know that something that is specified can have more dimensions on it that are actually imagined.

The actual product circle was usually very small or diffuse. An exception would be the search engine where the product in itself was quite clear, but if you widen the span of the product to include the business idea as well, the product circle is bigger and more diffuse.

For me, the testers’ role in any project is to question the product and provide useful information about its state to whoever wants or needs it. In the different projects this weekend, my questions to the products were distributed depending on the context and state of the product. Any questions asked will move the product circles in some way.

About the desired state, I would say that it is highly dependent on the context, but there are some I have some theories on it. When the actual product circle is covered by the specified and imagined circles in a satisfying way, the product could be shippable. If however there is too much gap between actual and imagined, the problem it is supposed to solve might not be solved.

In my perspective, any question that would drive the circles just a tiny bit closer to its perfect ending state, is a right question. I will not dig deeper into what is the perfect state, but it actually means that as long as the questions asked lead to moving the circles in the right directions in their contexts, no question is actually wrong. Especially if you consider that any questions asked will change the shape of the imagined product because of the learning done by both tester and project team. In that perspective, a good tester should never run out of questions.

About this post, I’ll have to add, I know that I am making a lot of assumptions and some parts of this is not really facts or something that is finished. But any questions will move these product circles and make mine and maybe James’ knowledge about this expand. So do question me, even if I might not have the answers right away.

Skånetrafiken 2 – Usability and user testing

February 19, 2010 2 comments

I have written before about the Skånetrafiken vending machines for the trains in my region. Skånetrafiken recently launched a completely new system for buying tickets and using a new type of RFID cards for payments, called Jojo cards. The neat thing with these, is that they dont require the card to be put into the machine, but instead just require cards to get close to the reader. This way you dont need to pick the card from your wallet when using it. Thats neat.

With this nice solution comes new machines for reading the cards, which are installed in all the buses, as well as some other type of machine for buying train tickets. I have seen the latter in two forms, and havent looked into the difference yet. What I do know, is that one accepts cash while the other is only for Jojo cards. I just had to try one of these out when I saw it at the station and I had some time spending waiting for the train.

Exploring the machine, I immediately discovered some flaws with user-friendliness and strange workflow. Touchscreen being too resistant to touching and too little information in each screen was things that I noticed.
One of the strange workflows parts is the fact that if you want to pay with Jojo card, you are first required to register it, choose your way of travelling, and then register card again. Somehow as a user this is a very annoying workflow having to raise your hand with the card twice, and I couldnt seem to find out why they wanted to do this. You could imagine that they want to see the amount of money on the card to see if you could afford the trip, but this was not the case since my card only had time period credits for unlimited use, but no money credits to buy special trips. When registering the card the second time, the machine just stated not enough money credit, and cancelled my search. This resulted in some frustration having spent time on the search, but abruptly cancelled by the machine. The possibility to use my credit card for the searched trip was something that I had to choose as the first option before I could even search and see the price for the trip.

I remembered reading in the local newspaper about Skånetrafiken stating that “Well, its always like that, people just need to get use to the system, they will soon enough.”

After playing with the machine some more, finding more of this type of “bugs”, a girl came up and wanted to buy a ticket so I moved away. Watching this girl get more and more frustrated with the machine for about a minute, she turned to me and asked me for help buying a ticket. She just wanted to buy a youth ticket to Helsingborg from the closest station Maria station where we were situated. The thing is that Helsingborg is one of the really big towns in my region, and should be one of those “quick buy options” (shortcuts), especially if you are travelling from Maria, which is a of part of Helsingborg town. When looking at the shortcuts on the home screen, I realized that all of them went to reasonably big towns in the region, but were all predefined for adult ticket prices. Then what should a person do to buy a youth ticket? You have to keep in mind that youth ticket prices apply to any person in the region under the age of 20, which applies to quite many people travelling by train.

The option “other towns” on the home screen shows about 20 more options of destinations in the region, but since the 6 biggest ones were on the frontpage, they are not shown in this screen either. The biggest difference from the home screen is that these destinations dont come with the predefines adult price. But since we wanted a youth ticket to one of the destinations on the home screen, we did not find our solution on the second page either.

Well, after that I had two options to try, the “Map” to use and find the destination there, which included zooming in multiple steps before the destination appeared on the map as a button.

So, how about some use cases with personas to discover these issues at an early stage of development? I mean, youth ticket is most probably a common used ticket.

More flaws on the user interface of the machines can be found at http://www.skanetrafikensuger.se/.  I recall many of them being issues that could have been avoided with proper testing actually. Since I dont know if they are going to remove it soon, here quoted in swedish:

Därför suger de nya biljettautomaterna

  • Touchskärm
  • Ingen maskin har knappar som låter “ding” när man trycker på dem. Vilket spånhuvud har skakat fram det gamla ding-ljudet från Windoze 3.1 och stoppat in i maskinen. Rätt ljud för audioell feedback är givetvis ett kort och koncist “dutt”.
  • Touchskärmen sitter så långt bakom glaset att det endast går att trycka på en “knapp” på skärmen om man är pygmé och tittar på skärmen rakt framifrån. Vi andra normallånga måste trycka 2cm under önskad “knapp”.
  • Vansinnigt segt GUI. Inte ens ett litet inbyggt system är så slött, detta körs under Windoze på (industri?)-PC!
  • Gränssnittet är en kognitiv katastrof. Inte ens en ingenjör kan göra ett sådant ointutivt gränssnitt.
  • Omöjligt att få ut tidsstämplat kvitto för resegaranti, måste därmed ha tillgång till kopiator för att ansöka om resegaranti.
  • Omöjligt att få ut ett nytt papperskvitto på köpt biljett. Användbart när automaten på bussen har slut på papper. Som en extra teaser ser man biljetten på skärmen.
  • Måste hålla upp JoJo-kortet två gånger mot läsaren vid köp. Var gör man föresten av kortet under köpet? Obegripligt!
  • Skärmsidan där man väljer antal vuxna/barn/hundar/katter vid biljettköp har två knappar med snarlikt namn, en “Nästa” och en “Fortsätt”. Obegripligt innan man tryckt fel en gång.
  • Inmatning från knappsatsen för kod till konto/kreditkort kort segar efter.
  • Kortläsaren för konto/kreditkort är värdelös. Går inte att hantera med stora fingrar eller vantar på händerna. Verkar dessutom rätt okänslig, ibland får man stoppa i och ur kortet mer än en gång. Ointutiv att man ska stoppa i och dra ur kortet.
  • Varför den svarta skärmsläckaren med den röda ringen?
  • Maskinen får spel och står och låter ding-ding-ding om man lämnar den i fred en stund.
  • Det ointuitiva och sega användargränssnittet resulterar i att biljettköp tar långt mycket längre tid än med de gamla automaterna. Dessa, ska tilläggas, var heller inte direkt användarvänliga.

Inte många rätt med andra ord… Faktum är att jag ansträngt mig för att komma på en enda bra grej med de nya automaterna men inte lyckats. Fast det spelar ju ingen roll, enligt Skånetrafiken är det ju kunderna som är ovana och inte begriper något.

Har du fler anledningar? Tveka inte att maila mig påph@skanetrafikensuger.se.

Exploratory Testing – Testers way to being agile?

December 16, 2009 7 comments

When I first skimmed this headline and then the post, I thought that this guy had the same thoughts on the subject as me. But reading it a couple of times made me realized that the post was not about the same as I was thinking about at the time. Although I acknowledge everything in it as good stuff, and very much on the path of DET (Developers Exploratory Testing) that my colleague Davor spoke about at Öredev in Malmö and ExpoQA in Madrid this year.

With my post, I have a slightly different perspective on it, which has been growing in my mind and discussed in some different settings with some friends.

The other day I asked a dear friend/colleague of mine what he thought of when I said the words:
“explore>learn>adapt and then explore again”
He directly responded with: “Thats agile!”

To be clear on this, he is a very knowledged and skilled developer and scrum master, but like many developers he still has the eyesheds of not looking too much at the testing perspective on things more often than sometimes.

So what is the difference then between being agile as a tester and doing exploratory testing? And why does my colleague associate the same words with being agile, and big parts of the testing community would say its ET. I dont see much of a difference actually. In James Bachs rapid software course this is the summary of what ET is: “by treating learning, test design and test execution as mutually supportive activities that run in parallell throughout the project. ” To me thats pretty much like the shorter and more concise wording I asked my friend.

In a project I am in now, I have to deliver a test specification as a product of the testing. So how do I do? Well, I explore the product in some way to learn how it works, design my tests and document in my specification and at the same time execute and adapt to my next step of exploration. I would say that in my case, I am doing ET, and those are the exact words I asked my friend about. The funny thing here is that the wording had to be neutral from the word test, for him to understand the meaning of them being agile, but for the testing context they are the same as ET, at least in my opinion.

I know that most people that think of ET, think of a scenario where the exploration is done manually through a gui, like on a web page. I acknowledge that as ET as well, but I think that this common understanding of ET comes from the simplicity of describing it like this. That kind of description I think is the most common among developers trying to understand ET.

The harder thing to understand about this, is to see the testing as the iterative feedback loop that it actually is, which responds to the feedbacks of developing with TDD or actually any agile process. The loops are just different in size and visibility. In my project some of the loops are 5 minutes and sometimes it takes a whole day. For exploring a web site, the feedback is mostly about seconds.

To conclude the post, why not just call exploratory testing being an agile process approach to testing, instead of an approach to testing within an agile process. The latter being the most common description I have seen when talking about agile testing. You see the difference?

Alot to do lately

October 26, 2009 Leave a comment

I have been busy with alot of things lately, keeping myself away from spare time blogging. So I just thought I would tell briefly about some of the events/working tasks that I have experienced, and would really enjoy some feedback on what is actually interesting enough for you, my readers to read a blog post about. So please comment on the things you want to know more about.

First out, I was on a nice competence weekend last weekend with Testway, were we gather all of us for friday morning – saturday lunch, learning new stuff and share knowledge. This time we had three main areas to talk about:

Lean test optimization
This is a product in the company portfolio, where two of us in a couple of smaller steps (workshop, analysis and tryouts), start with the current test process of a customer and make it more lean, removing obvious waste in a first iteration. Using baby steps of improvement in the current setting is easier than changing the whole process at once. The product consists of a complete first baby step during a week for the customer.

Cloud and testing
There is an obvious change within the industry towards using cloud solutions for hosting and running applications in the cloud. There are a couple of different settings to choose from depending on your needs. How can the whole concept be used for us as testers?

Selling consultancy services
Mainly about how we as a company are marketing ourselves and selling our services.

Apart from these things, we of course touched all the other amazing and exciting things to discuss during the weekend.

Monday I attended an internal workshop about The kanban game with Björn Granvik. This very interesting workshop Björn is going to have at Öredev together with Scott Belware. Starting with an introduction to Kanban and the concepts, we continued playing the game, getting a feel of the process itself.

Wednesday I attended another part of the course “Agile for experienced project managers” with Leadway. This is a course that is going to be taught in public, I think sometime in november. Tell me if you are interested in attending, Ill tell you when it is. This part was about how to scale agile and project dependencies. It also included a short story about how this can be done in practice.

This involved parts from a seminar that I attended three weeks ago with Petri Haapio, talking about Scrum implementation at Nokia Networks.

Then of course I have my daily work, which at the moment includes both Ruby/watir cucumber test automation of a web application as a Test Automation Launch Pad product and some big systems functional and pre-delivery testing at a larger company. These could both be full time jobs, but doing them both at 50% has been possible.

Apart from these things I am in the program committee of Öredev conference, responsible for the test track. That has also been taking some time.

That was a summary, and not even everything is mentioned. And I actually do have a private life as well, a little. I will add some pictures to the different topics later.

So please go ahead and tell me what would be the most interesting to read about, and Ill focus on that for a blog post. I have a somewhat problem focusing sometimes. I like the context switching even if it doesn’t give me the efficiency I want.

Öredev is coming up next week! Please register if you havent already, it is going to be awesome! Meet me there and discuss something exciting….

Categories: Life Tags: , , , , , , , ,

What to experience at Öredev conference?

October 7, 2009 4 comments

Since I am in charge of the test track at Öredev conference, I feel I have to promote it somehow. I have to say that I am impressed with the whole line-up during the two days of testing, so I am not going to say any names. But what can I say about the conference without just talking about the test track?

Apart from the test track, the whole conference will offer so much more. I can only refer to the conference last year, and the things I know from behind the scenes, but thats not a pity.

First of all I have a feeling that the Agile and PM tracks will offer very much discussions I can relate to and am interesting in. There is a whole span of leadership seminars in a sweet mix of agile, soft skills and success stories. We have for example Stacia Broderick that has written the book “The software project managers bridge to Agility”, where she bridges the agile concepts with traditional project management.

Then we have the new track with focus on User Experience, where I expect alot of my user level issues to be discussed. For some people the testers should not be involved in talking UE, but for me its an important thing. The tester thinks as a user, acts as the user in projects and can act as a proxy between for example UI designers and developers. More on those thoughts later.

Apart from the seminar sessions, there will be shorter lightning talks to get taste of a variety of sessions without spending whole slots for it. You can of course spend longer sessions and even days on the courses like Rapid Software testing with James Bach, Scala with Jonas Bonér or Scrum master certification with Jens Östergaard and James Coplien. Then we have tutorials, and we have, and we have….and and……

I could write for hours about the program, but still you would get so much more overview in looking at the schedule. And actually I dont think that those things are the most important. The most important thing is the experience and feeling at the conference. There will be alot of space and time to mingle with the speakers over a snack, meet new people and share new ideas with people you wouldnt meet somewhere else. The diversity of speakers and participants that shows up for the conference and the different tracks is what gives Öredev its spirit.

Meet me at Öredev 2009!

Categories: agile, testing Tags: , , ,

Kanban + testing = ?

September 11, 2009 6 comments

“I have heard a little on Kanban, which somewhat could be called an evolved agile or at least something in that direction. What i now am interesting in knowing is if these ideas, that are sprung from lean very much, affect or should affect the view of testing. What I do know, is how I would like to perform testing in the agile context. So what about kanban? Does it change the agile testing even more?”

The above quote is how I started this blog post some time ago. I didnt know how to continue in a somewhat professional manner. Part of that was because I had not gotten any picture in my head about how kanban works.

Yesterday, I attended the monthly Agile Skåne meeting, and one of the topics that was brought up was kanban. Since I was not the only one that didnt have so much of knowledge in it, we got a crash course in the form of a success story. While listening, I got some kind of vision of it.

Since I have been working alot with scrum, I had to ancor this kanban thing in scrum. I will now see kanban as scrum with the shortest sprints you can have, the size of a story. Then I will take into account another thing from kanban, the value stream context, which in my perspective means having only two places on the board that are allowed to have piles of cards, that is “Wishlist” and “Value for customer”. Having only these piles, while other categories like “implemented”, “tested”, “ready for test” or “ready for deployment” are not allowed to stack cards, you will get a flow of value towards the customer without leaving stories behind as untested.

While thinking of this, I realized that one issue with agile testing that has been bothering me for some time, and still does, will be easier in kanban. Why does testing in a scrum team many times end up in the discussion on “definition of done”? This is exactly one of the bigger issues I see when testing in the agile context. With this and the value stream context of kanban in mind, I realized that the fact that you extend the number columns on the board past the “sprint demo done” will visualize the testing in a more obvious way. This has the positive effect that the agile tester does not have to advocate testing as hard as sometimes is the case within developers. In the agile context, you can always add some columns when realizing where you want done to be, but in kanban this is default, meaning that the tester does not even have to argue for getting the “To test” column on the scrum board.

By only looking at this aspect of it, and imagining a little more about the kanban context, I would say that kanban is better suited for test than agile and scrum are. But I think Ill have to get back to this later on when having more knowledge of kanban.