Test Planning

When you are planning for your tests, either small-scale plans, just to look at a single feature or change, or more complex plans for testing and application there are a few steps you can go through to help ensure you have to revisit your plan only when your understanding of the SIT changes rather than the requirements for the plan changing.

Be clear from the start who is using your plan, is it just for you to help plan what you are going to test? is it for the test team to allow for a shared understanding? or will it be used by the wider project?

Having an understand of this will enable you to include the information that the rest of the audience need for the plan to be helpful for them. You may find that a single plan isn’t sufficient as different people require dramatically different information.

Where the test team may only need a loose plan to help the achieve good coverage the project manager could be wanting to understand resources or duration and stakeholders may want to be able to see that what they are worried about is getting sufficient testing to give them comfort once live it will be reliable.

Understanding the differing requirements for any plan you make can help you produce one that is fit for purpose, in my experience the easiest way to do this, is to create an outline of the plan, something that doesn’t take too much time to do, and share it with those who will be looking at your final plan, speak with them about what it tells them and what they need to know that they can not see, incorporate what you can, make compromises where there are too many conflicting needs.

This will help you create a plan that works well for most people the first time and hopefully reduce the number of times you need to revisit the plan.

Testing conferences; bring your Boss

I’ve been meaning to write this for some time now, work, however, has been very busy.

So I went to my very first Testbash in Manchester back in October. I did both the Friday and the open conference on Saturday. Both days were great, the talks on Friday were very enjoyable. My favourite talks were James Bach’s keynote on social and critical distance and Joep and Helena’s 4-hour tester experiment. I won’t go into any detail here on them as they aren’t the focus of my thoughts.

I was lucky enough to be able to take my two full-time staff along to the Friday, and  I had managed to convince my boss to come along as well. Now my boss’s connection to the world of testing is me. At work, he is responsible for the IT systems and applications team which includes testing. So up until the conference what he knew about testing, largely, was as a result of what I had told him. He’d worked with testers earlier in his IT career, he’d not managed them before though. So when I wrote IT’s test strategy what it drew from and the things it talked about were new to him, and he took my word about its suitability. By getting him to go to Testbash Manchester I was able to introduce him to a huge number of testers who spoke the same way I do about testing, have the same challenges and often refer to the same source materials. I am very grateful that the Testbash crowd is so open and friendly  as it made the whole thing so much easier. He took away from the event a much better understanding of the problems I have been talking with him about and a better appreciation of what it is we do. So much so that he’s now working with me on getting the company to think about testing differently, to get us involved earlier in the process, (thank you, Duncan Nisbet, for your talk) as he now gets it.

So next time you are planning on going to a test meet up, or conference, think about if taking you boss along could help you. If like me you’re working for someone who doesn’t know testing, no matter how supportive they are of you, hearing other people talking about the problems you are trying to work around will really help you. It gives a legitimacy that is hard to get any other way. To hear others talk gives a wider perspective and helps bring these things to life.

Thanks again to all the friendly Testbash people.




Technical, Non-Technical Testers, and what does that even mean!

So it turns out that I get quite annoyed when the terms Technical, Non-Technical testers are used to describe what my team of testers should, and should not be doing! I was recently in a heated conversation about getting access to read some SQL based ETL’s and one of the reasons for us not to have visibility of this was my team are not technical testers, and as such shouldn’t, and by implication, can’t read such things.

So let’s explore for a minute what I think when I hear those words, I think more about someone who builds tools to use in testing vs someone who does not, what I’ve also heard called test engineers and test analysts. At no point does it cross my mind that you’d be talking about a code illiterate person or even someone incapable of looking up the bits used to get a general understanding.

So upon hearing it used in this way has caused an irrational response, one of annoyance. I can’t help think that maybe it’s down then to how I think they see a person they are describing as incapable of reading code, my insecurities rather than their intent? beware your own biases they can really colour a conversation.

Unfortunately, the knowledge that I’m annoyed because of my personal interpretation hasn’t reduced my annoyance at the whole situation, life’s a bummer.

I disagree with the test team not having access to the ETL’s as I personally feel the documentation of them isn’t as good as the ETL’s themselves, sadly it’s not my call to make at the end of the day, we work with what we have, and I have to put my frustration aside about this and get one with the many tasks I still have to do.

How do you deal with situations that have both  an emotive content for you, and you disagree with from a project/task perspective?



Test planning, so you want more time!

If you’ve been involved in test planning you’ve no doubt encountered the questions, “Can you do it in less time?, What if you did test like this, is there a quicker way? what if we get some of the business to do all the testing for you?”

There are more questions like these as well, they all tell us something about the questioner, about how they are thinking, what matters to them, and they give you options to talk to them about. Testing can normally be done quicker, it’s just a question of do you want the risk that is associated with doing so?

So we are near the end of a change of software in part of the warehouse, there’s loads of config, some development, and a tonne of business understanding to gain. This is the second time we’ve tried this, the first time the business wasn’t at all happy with the config, so it was back to the drawing board for the project team. So now we are near the end and there won’t be a third chance so it has to go in, and be right. I haven’t been on this project, as you may have read in other posts my focus has been on the ERP we are putting in. However they are near the end and are worried it’s not going to happen, so I’ve been moved back to this one and basically asked to make the testing fit in the short time that is left. It’s all driven by the overall project budget, basically getting more money is going to be hard, so I have to look at what’s left and work out what not to do.

It’s all about risk here, the work is largely split into to pieces, the interface work, moving data via a bus between the new system and existing systems, and the new warehouse system itself. The interfaces have to work, but we control most of how the data arrives, so we can do a reasonable job on that without having to do to deep a piece of testing, and the warehouse system has had people all over it for months, there are in fact a group currently signing off business processes and user stories against what the set up in the system is, so they will have a good chance of seeing if the basic flows don’t work. So my sales pitch is more focus on the interfaces, less on the new piece. The focus of testing on the new warehouse piece would be the edge cases that the team signing off the basic flows are unlikely to go near.

Yes the state of what’s going on is far from ideal, but it rarely is, so we work with what we have, I’m often reminded of the scene in 13th warrior where they are on the boat and the northmen are laughing at the storm, show no fear, laugh at danger. There’s no point bemoaning the state it’s all in, get on and do what you can, make it work as best you can with the time and resources you have, flag risks if they are too much for the business to bare they will say. Your job is to let them know what these are and do the best you can to help get the project past the go live.



Long Chain Bugs

We’ve all seen them, your testing away, and something odd happens, application crash, unusual error message. you look back at what you were up to, retracing your steps waiting to see the same behaviour repeated, yet nothing shows, you end up digging around into system state, other applications that were running as well, other people who were in the environment as well, sometimes we can work it all out, a lot of the time, at least I, can’t. Well as I found out recently at a medical appointment I attended with my other half, it would seem that the medical community has long chain errors as well.

My partner has Lyme, she’s been very unwell with it for a number of years, and we recently had the chance to talk to a top-flight consultant about her condition. What they explained was Lyme Borreliosis Complex (LBC), as they have dubbed it, is a complex condition that has only in part to do with a single event. The system state also needs to have a variety of conditions meet for that bite to then cascade into the horrible condition that LBC is. They are not sure what the precise conditions need to be, they believe its a combination of genetics, familiar health, and environmental.

At, this point I had to try very hard not to hijack the appointment and start talking with them about how they are researching this and have they seen the very cool ted talk about a multi-disciplinary team that tackles big challenges that have eluded specialists.

It has made me think about how people who have practised testing, understand the mindset required to untangle these complex scenarios could have incredibly interesting and fulfilling jobs if we could convince other communities of the value and benefit we could bring.

Have any of you had the chance to bring your testing skill set to unexpected places?

Understanding risk from the perspective of stakeholders

A key part of getting software signed off is having the key stakeholders happy that it does or is what they wanted. So how do you make sure you address this in your testing?

There are a few different approaches that you can take to gain an understanding of the risks from the perspective of a stakeholder, at there core though there are a few skills which you need to use regardless of if you interview, produce an information gathering form, or run a brainstorming session, to name a few ways.

What you are trying to do is gain an understanding of their real needs, and the stakeholders may not understand what these are in the early stages of a project, or at least in any great detail. You need to draw these out of them with the use of open questions that lead them to think about the details that you need to understand. You have to do this without imposing your concerns on the conversation. Yes, it needs to be a conversation, it’s not a question and answer session, you have to build rapport and trust with them so that over time they will volunteer this information to you as they are beginning to trust your judgement and that you will take what they say seriously.

You need to set the initial conversation up wisely. You have to give the stakeholder plenty of warning about what you want to do, and what you want to get out of the conversation so that they can prepare. You’ll also need to prepare, make sure you can provide answers to any questions they are likely to ask that you have accurate information about, however, you have to be able to say, ” I don’t know that yet” or, “let me look into that”. nothing will damage the relationship you are trying to build faster than guessing at answers and getting it wrong.

It’s okay to go off script, when it is done for good reason, to gain greater understanding, try to bring it back on track at the appropriate time, when you have explored the topic you were discussing before you or they have started to ramble or moved too far from what you wished to talk about.

You’ll need to practice active listening, being flexible about your questions and asking questions to draw out more details, these are not easy skills and deserve a post in their own right, so I won’t go into them in detail hear. I’ll just say you need to be paying close attention to what you are being told so that you can ask the right questions, at the right time to draw greater detail out, all while writing down the important pieces of information.

you can also make great use of silent pauses to draw further details from them, people will naturally want to fill gaps in conversation, and you can use them when you think someone is holding back. Just don’t go overboard, it’s not an interrogation. You have to practice the suspension of the ego, as these conversations are not about what you think, they are about what the key stakeholders think. You have to cultivate your curiosity about what other think, this will help you forget about your opinions on the matter as you seek a greater understanding of the thoughts of others.

You have to practice the suspension of the ego, as these conversations are not about what you think, they are about what the key stakeholders think. You have to cultivate your curiosity about what other think, this will help you forget about your opinions on the matter as you seek a greater understanding of the thoughts of others.

And remember this is an ongoing conversation, you should get back in touch with these people on a regular basis, even if it’s just to let them know you are still actively using the information they have provided as a form of measure against the quality of what you are looking at. If you maintain good rapport with them, they will more naturally include you in conversations about what their concerns are, and if you have done it well they may even come straight to you when they have concerns or questions.


Let me know your experiences of build an ongoing relationship with key stakeholders, and how you went about it.





Do you know what testing is?

So you are working on a project, large or small and you are working with a variety of stakeholders, some may know what testing is, some may think they know what testing is, and some have never even thought about it.
So what do you do?
For me this one is an easy thing to address, I tell them all what testing is to me, in small bit size pieces. Using plain English, and alway checking back with them about what they mean when they use technical language, always phrased as ” just so I can understand..” not so I can correct them, so I can make sure I really know what they mean, This goes for technical people more than any, I sat in a meeting with a system architect lead, interface lead and developer lead, and they all had a different definition to unit testing (or developer testing). If I hadn’t asked it would have gone unsaid, assuming they all meant the same things.
People who don’t know what testing is, you have to show them the value you bring, demonstrate why what you do, makes their lives easier. Telling them isn’t enough, they have to see what you bring to the table otherwise you are just some strange IT role.
Be open and inclusive invite people to see what you do, help them see why you ask the questions you do, and never fear being challenged, just make sure when you respond to that challenge its positive, clear, easy to understand and relevant, they aren’t saying your wrong, or not needed, they are really saying I don’t understand why often with little realisation that this drives what they are saying.
If you want to do well in the professional world being able to demonstrate your worth is a skill that is worth developing, as the way you demonstrate it has to change according to the audience, you have to be able to explain in terms anyone can understand and defend why you have done or are doing what you are, especially if they come with preconceptions about what it is you do.
James Bach and Micheal Bolton tester story is an invaluable tool here (These are the people who I first saw this from not sure who’s idea it was first) It gives a framework within which to think about the differing side of what you are doing, and how to address questions about it to support why you have chosen to do what you have, when you have.
It’s all about understanding, what the concerns are, and how you can provide assurance to the stakeholder who raised it, that you will be able to provide them with proof/information/demonstration of that area so they can make a judgement on how happy they are with it, for example, in my current project the speed of order entry for the trade sales team is very important, they need to be able to work through an order flow, including placing individual lines on an order at the speed of speach. We are addressing this through demonstartaions and timings of screen loading, so the Trade sales manager can see how fast it is and make a judgement of if they are happy with that.

Tests, scripted, and taken

So I was watching some short videos on cognitive biases the other day when my mind wandered on to the topic of tests and testing. The things that we do to investigate the quality of software, and the thing most people do in large rooms under time pressure, and I wondered if the most commonly understood meaning of the word testing, as in formal highly structured, controlled and scripted process of answering questions on a specific topic under time pressure to receive a grade has poisoned, to use a dog training term, the ability to get people to understand testing as a process of exploring, learning, evaluating, modeling etc software to help make a value judgement on its quality.
So what do I mean by poisoned, in dog training terms if you have a cue you have taught your dog that elicits a specific behaviour, such as down, to lay on the floor, and then other people repeatedly, and constantly overuse the term, ie more than once per requested behaviour, and also use it to mean get off that thing/me/other person. The dog will stop having a cue down for laying on the floor, and the down cue will like not result in any particular behaviour.
So how am I applying this to the endless education issue around testing in software testing, we are bombarded with the word test and the school exam meaning for many of our formative years, and again if you enter an industry that has a high degree of continuous study and testing to maintain a level of knowledge and does this via formal testing. Rarely do any of these tests reflect our use of the word testing, they seem anathema to it, in fact, all formal, with a gravitas and pomp to give the appearance of great importance. 
Is this in fact the fight that we are having to get people to accept that testing software  can legitimately be exploratory and investigatory, rather than a highly formalised and scripted process, has the interpretation of test become an Alief in the minds of the general population?
What are your thoughts, why is it so difficult to shift the understanding of the word Test?

Passion in your job

So as you may know from earlier posts I’m quite new to testing as a full-time job, it’s something I’ve been involved in for a while. As I’ve progressed in my new role as a software tester, rather than just doing some testing on the side I have come to the realisation that testing, as I now understand it, is something I really enjoy. It speaks to many parts of my mind, it’s a constant challenge, with a great deal to learn about, and a huge variety. It turns out that I am very passionate about testing being done well, and that passion is translating to work listening to what I have to say to them.
So what does this meaning for me?
Work are more willing to accept when I tell them something about testing.
They are willing to send me on courses and to conferences.
They are willing to review what I do, as defined by my job title.
But more importantly, they are willing to give me more responsibility, areas of the business that were out of my original reach, yet really needed to review how they test software are becoming places I can influence, they are falling within my remit as a software tester. This hasn’t taken long either, my passion for high-quality testing totally changed how we are approaching testing the ERP we are installing, as a knock on from that they are willing to listen to me about minor software changes. letting me help the new testers we have, in terms of coaching, and getting them on courses to help them, inviting me to meetings with the senior developers so I can input early on projects.
All of this is happening because of my highly visible passion for what I’m doing.

If you want to go further where you work, show the passion you have for your role, bring ideas to your boss, push your self-development, let people know how much doing what you do, well, means to you, and how it helps them. Passion for your job does translate to progress.



Over the weekend I happened to read an article on assumptions and if we should be making them. The article talks about the writers progression from the assumptions are bad, to assumptions are okay if they are reasonable. It was interesting to read the progression of the writers thoughts over the use of assumptions. They went from, as a younger person following the adage about asses, to coming to realise that some assumptions are part and parcel of life. 
However, it struck me that  we could do with a better education when it comes to this type of reasoning within software testing. Too little time is spent talking about how you investigate something and the process that you go through to come up with theories to test, most sciences, both the more classical and social sciences spend time teaching how to investigate, and the use of assumptions.
It reads to me like the author is talking about beliefs, those things that drive our assumptions. If they had dug a little deeper and thought about these assumptions maybe they wouldn’t have held them in such low esteem.
We could rephrase the article, rather they are talking about justified beliefs, in the epistemological sense, or we are using them to form part of an ampliative argument. In either case, the negative connotations of the “assumptions” should be removed as you are no longer talking about the vague process of making a guess at something, but it is a considered idea that you believe is supported by the available evidence.
Both of these play a large part in the process of assessing a situation and forming a hypothesis that you are going to investigate further and are held in much higher regard than the simple assumption.
I think that if we could move to accepting that this is the normal process that is gone through, and start using the techniques that are available in other disciplines that rely on investigations we would be able to share in the developments in ways that you can investigate.