Some people have learned that I will give them my thoughts on anything they put in my hands. Most commonly this would be in the software domain, but just as much I will give my thoughts on anything that I can inspect, question or at least have some clue about how it should be, or not be. Sometimes I dont even know this much, but I am able to ask the right questions for the producer to realize the flaws himself. Apart from software this can be documents of some sort or even important emails they are sending.
I really enjoy questioning “products” with the sharp eye of a hard-to-please customer or just like a regular user. With my profession I also have some of the necessary skills to communicate what I believe are flaws or suggestions of improvement in constructive ways that will motivate or even compel changes to be made before deployment. Sometimes the bugs I find are very serious, and sometimes not so serious, but all of them at least become visible to the stakeholder that shows it to me.
Thinking about all the software products I have gotten my hands on, there is a remarkable amount of them where obvious bugs or flaws have appeared almost instantly when doing the first exploration of the product. I have no numbers to build this discussion on more than just a gut feeling that as a tester, if I just get a hand on a product, there will be questions to ask and bugs revealed almost instantly.
I dont know why this is the case, but there are of course several factors that are more obvious than others usually. Things like environment and configurations, testability in development, testing skills and heuristics of developers or even inattentive blindness are factors that usually cover a lot of this. But isn’t there another dimension? The dimension of not only letting another person look and inspect the product, but letting a tester have a look. Sometimes I wonder if my feeling “hand it to me, and Ill show you how broken it really is”, is it something that everyone can sign up on, or is it the way of the tester?
An example of this was when a friend and colleague of mine showed me the product he was currently working on, a mobile phone application. As a tester in the project he had been questioning this particular app for some weeks already. Within 5 minutes I discovered something that for me was an obvious and serious bug, but he and his team had not discovered it before. Was it a coincidence, or is it cause I am a tester and will look at the application in a different way than my colleague? I am quite sure that any other person than another tester would not have found the bug. Say what you want about it, but it was a good thing letting me play with the product for 5 minutes while having a break. And what if you include me fulltime as a tester in a project where you dont have any dedicated tester?
I think of this quite often, and especially when it come to smaller projects (1-5 people) that just say they cannot afford having a tester that does not produce value. And as a tester it is not so easy to prove the value added before getting involved. Though this is a whole new story, can any team of any size creating any product AFFORD to NOT have a tester if you not only think of the customer value created but also include the customer value that you risk to loose if/when delivering an untested product?
James Bach brings up the logging in a system as a tool for the exploratory tester. I really like this, and am going to see if I can straighten out his list to fit my specific project and situation. I have been thinking about this some time now, but mostly about the information within the log entries, and not the logging mechanism itself.
The system I am testing at the moment has extensive logging features, or at least it logs very many things at multiple levels. I use the logging sort of like he mentions, but not relying as extensively on it as it sounds in his post. The system is pretty straightforward relying on one input that creates one single output. Apart from that, there is only logging and status management to keep track of events.
In our case, when testing a part of the system and keeping my eyes on the logging, it struck me that there was a line “check that x has not failed”. This line was very common to see, since it was logged for every event created successfully. But thinking about it, was this entry a useful entry? NO! In my case it was so obvious that none of the developers had been thinking about it, but it was an horrible entry. Why? One of our statuses for events was “FAILED”. So if an event would fail, the obvious thing is to grep for this word in the logs. But having every successful event instance log the word failed would be devastating for any operator trying to find the cause of a failet event.
So what I would like to extend in James’ list, is that of bullet 5″- Event type description: short, unique human readable label” and “- Event information: any data associated with the event that may be useful for customer service or assessing test coverage, this data may be formatted in ways specific to that event type.”
I want to point out the value and importance of having these string fields not interfere between different types of log entries. Like my example above, keeping the word ‘Failed’ away from any successful events at all times is crucial. The problem is that it is so easy to get it wrong. Developers need to think about this when coding, and not only log what the code does, but create the human readable log entry consistent from the user/operator perspective. If something is successful, it does not matter that the code is “checkJobNotFailed(job)”, but the logged entry should verify the success.
Other types of risks with the logging content is of course that of usernames and passwords. If this type of information is logged in a readable way at any logging level, the log files must be handled according to the security policy of this information.
I have made two examples to point out less good logging events. So, what more examples of “good vs bad” log entries are there? Remember that I am not talking about the logging mechanism and its’ good and bad practices like the ones presented by James, but the content of the event entry messages.
Please enlighten me with more input.
Ok, first of all, I hope I wont scare everyone off just because my blog will start off talking about stuff I do at work. The thing is that it is almost like a hobby actually to learn more about those things, thats why I love my job.
I am not going to explain the basics about Scrum, if you are interested you can watch this.
In January, before I started to write my thesis, I had no clue what Scrum was. I barely knew anything about Agile either. I actually think its strange that we didn’t get to know much about it in school. Well, from the first day of thesis work, my interest in these things has been growing steadily and the fact that I am working in an Agile environment really gets me going. There is not a day that I dont talk about Scrum and Agile, and I try to pick up as much as I possibly can at seminars, workshops you name it. Next week my company is having a big conference, Öredev, where there will be so much knowledge running around just for me to catch it=)
As I work for Testway, my main area is actually testing (software). This is why I have gotten interested in the testing aspects of Scrum. But since I have attended different forums in the area, I have realised that all other disciplines within software engineering also have their aspects of it. Especially for example both testing and configuration management, which could in some cases be thought of as heavy in practice. This is where I state the final question:
How is it possible to get the absolutely best out of an Agile/Scrum project, when really considering the best practices within all the areas of requirements engineering (RE), testing, configuration management (CM), architecture (software of course), project management, portfolio management etc.?