Test Planning

When you are planning for your tests, either small-scale plans, just to look at a single feature or change, or more complex plans for testing and application there are a few steps you can go through to help ensure you have to revisit your plan only when your understanding of the SIT changes rather than the requirements for the plan changing.

Be clear from the start who is using your plan, is it just for you to help plan what you are going to test? is it for the test team to allow for a shared understanding? or will it be used by the wider project?

Having an understand of this will enable you to include the information that the rest of the audience need for the plan to be helpful for them. You may find that a single plan isn’t sufficient as different people require dramatically different information.

Where the test team may only need a loose plan to help the achieve good coverage the project manager could be wanting to understand resources or duration and stakeholders may want to be able to see that what they are worried about is getting sufficient testing to give them comfort once live it will be reliable.

Understanding the differing requirements for any plan you make can help you produce one that is fit for purpose, in my experience the easiest way to do this, is to create an outline of the plan, something that doesn’t take too much time to do, and share it with those who will be looking at your final plan, speak with them about what it tells them and what they need to know that they can not see, incorporate what you can, make compromises where there are too many conflicting needs.

This will help you create a plan that works well for most people the first time and hopefully reduce the number of times you need to revisit the plan.

Testing conferences; bring your Boss

I’ve been meaning to write this for some time now, work, however, has been very busy.

So I went to my very first Testbash in Manchester back in October. I did both the Friday and the open conference on Saturday. Both days were great, the talks on Friday were very enjoyable. My favourite talks were James Bach’s keynote on social and critical distance and Joep and Helena’s 4-hour tester experiment. I won’t go into any detail here on them as they aren’t the focus of my thoughts.

I was lucky enough to be able to take my two full-time staff along to the Friday, and  I had managed to convince my boss to come along as well. Now my boss’s connection to the world of testing is me. At work, he is responsible for the IT systems and applications team which includes testing. So up until the conference what he knew about testing, largely, was as a result of what I had told him. He’d worked with testers earlier in his IT career, he’d not managed them before though. So when I wrote IT’s test strategy what it drew from and the things it talked about were new to him, and he took my word about its suitability. By getting him to go to Testbash Manchester I was able to introduce him to a huge number of testers who spoke the same way I do about testing, have the same challenges and often refer to the same source materials. I am very grateful that the Testbash crowd is so open and friendly  as it made the whole thing so much easier. He took away from the event a much better understanding of the problems I have been talking with him about and a better appreciation of what it is we do. So much so that he’s now working with me on getting the company to think about testing differently, to get us involved earlier in the process, (thank you, Duncan Nisbet, for your talk) as he now gets it.

So next time you are planning on going to a test meet up, or conference, think about if taking you boss along could help you. If like me you’re working for someone who doesn’t know testing, no matter how supportive they are of you, hearing other people talking about the problems you are trying to work around will really help you. It gives a legitimacy that is hard to get any other way. To hear others talk gives a wider perspective and helps bring these things to life.

Thanks again to all the friendly Testbash people.




Technical, Non-Technical Testers, and what does that even mean!

So it turns out that I get quite annoyed when the terms Technical, Non-Technical testers are used to describe what my team of testers should, and should not be doing! I was recently in a heated conversation about getting access to read some SQL based ETL’s and one of the reasons for us not to have visibility of this was my team are not technical testers, and as such shouldn’t, and by implication, can’t read such things.

So let’s explore for a minute what I think when I hear those words, I think more about someone who builds tools to use in testing vs someone who does not, what I’ve also heard called test engineers and test analysts. At no point does it cross my mind that you’d be talking about a code illiterate person or even someone incapable of looking up the bits used to get a general understanding.

So upon hearing it used in this way has caused an irrational response, one of annoyance. I can’t help think that maybe it’s down then to how I think they see a person they are describing as incapable of reading code, my insecurities rather than their intent? beware your own biases they can really colour a conversation.

Unfortunately, the knowledge that I’m annoyed because of my personal interpretation hasn’t reduced my annoyance at the whole situation, life’s a bummer.

I disagree with the test team not having access to the ETL’s as I personally feel the documentation of them isn’t as good as the ETL’s themselves, sadly it’s not my call to make at the end of the day, we work with what we have, and I have to put my frustration aside about this and get one with the many tasks I still have to do.

How do you deal with situations that have both  an emotive content for you, and you disagree with from a project/task perspective?



Training New Testers

So as my more regular readers will be aware, at work, we are changing some major systems for something a bit more modern. This means more testing is required than my test team is able to do. So we’ve recruited some more people to test. I decided that the  best approach for this was to recruit internally for people who had the familiarity with the business processes that are impacted by the software change that I could teach how to test.

Now there are obvious risks to this approach, my ability to assess someone’s ability to learn quickly, have an appropriate mindset for good quality testing, my ability to train them, the list goes on. However, I felt that connection with the business was worth the risk, the ability to see what’s going to be a challenge in the departments, and what’s going to work well, plus the increased trust for the business by having key people involved in the testing process.

So the challenge that I have is to train them all to be great testers, well as great as I can help them be. There are plenty of great resources out there, videos, blog, books etc. However there are 5 of them, and we have to be testing the system at the same time as I’m training them. So I recruited them all earlier than I actually needed them, so they could start learning before it became critical. My first step was to get them to watch James Bach’s open lecture, I used this as a starting point as James has an engaging style of presentation, and the lecture covers some of the basic challenges in testing in an easy to understand way, with little need for the watcher to already understand testing or a great deal about software development. I also ran some training on the basic housekeeping tasks that make up the test process we are using here, session notes, timings, bug reporting etc. Plus a have a small selection of books that they can borrow which cover aspects of testing.  We also have regular open discussions about testing where we can share what we have learnt, and ask questions about how to do things. I’ve also given each of them a cheat sheet with some helpful heuristics and information about data types.

I have also encouraged them to spend a little time every week on some self-directed learning about software testing, which I ask them about, and when they have found something that is useful to the team to share it with them. I have also encouraged them to just log into the system, have a poke about, see how stuff works, and practice session note taking while doing so. This has been of great use, it’s uncovered a few bugs, and given me the chance to run a few debrief’s, and provide some feedback on both the notes they have taken and how they have thought about what they were doing while logged into the system.  My next challenge is to give them a grounding in some test methodologies, without confusing them, so they can start applying differing approaches to what they are doing.


How have you approached training new testers? Do you have ready-made material or favoured techniques?

Test planning, so you want more time!

If you’ve been involved in test planning you’ve no doubt encountered the questions, “Can you do it in less time?, What if you did test like this, is there a quicker way? what if we get some of the business to do all the testing for you?”

There are more questions like these as well, they all tell us something about the questioner, about how they are thinking, what matters to them, and they give you options to talk to them about. Testing can normally be done quicker, it’s just a question of do you want the risk that is associated with doing so?

So we are near the end of a change of software in part of the warehouse, there’s loads of config, some development, and a tonne of business understanding to gain. This is the second time we’ve tried this, the first time the business wasn’t at all happy with the config, so it was back to the drawing board for the project team. So now we are near the end and there won’t be a third chance so it has to go in, and be right. I haven’t been on this project, as you may have read in other posts my focus has been on the ERP we are putting in. However they are near the end and are worried it’s not going to happen, so I’ve been moved back to this one and basically asked to make the testing fit in the short time that is left. It’s all driven by the overall project budget, basically getting more money is going to be hard, so I have to look at what’s left and work out what not to do.

It’s all about risk here, the work is largely split into to pieces, the interface work, moving data via a bus between the new system and existing systems, and the new warehouse system itself. The interfaces have to work, but we control most of how the data arrives, so we can do a reasonable job on that without having to do to deep a piece of testing, and the warehouse system has had people all over it for months, there are in fact a group currently signing off business processes and user stories against what the set up in the system is, so they will have a good chance of seeing if the basic flows don’t work. So my sales pitch is more focus on the interfaces, less on the new piece. The focus of testing on the new warehouse piece would be the edge cases that the team signing off the basic flows are unlikely to go near.

Yes the state of what’s going on is far from ideal, but it rarely is, so we work with what we have, I’m often reminded of the scene in 13th warrior where they are on the boat and the northmen are laughing at the storm, show no fear, laugh at danger. There’s no point bemoaning the state it’s all in, get on and do what you can, make it work as best you can with the time and resources you have, flag risks if they are too much for the business to bare they will say. Your job is to let them know what these are and do the best you can to help get the project past the go live.



Recruiting a new tester

So it’s back to one of those challenges again, I needed to recruit a tester. They didn’t need to be hugely experienced, or possess any sort of specialisation, so that was a plus at least.

We have an unusual recruitment process at work so that always adds to the challenge of getting the right person. We had the normal run of the mill applications, we spend quite a lot of time looking at people’s covering letters, and seeing what they tell us about the applicants. Quite often a poor covering letter will mean we don’t interview someone, as it doesn’t tell us the person has the qualities and skills that we are looking for in a role.

It might sound harsh, but if your first contact with us, your covering letter, doesn’t sell why you are the best possible person for the job, and this means talking about actual experiences that highlight all those great skills and qualities, not just saying, you’re a team player, tell us why. What experiences show that’s the case. Anyone can write a bunch of buzz words on a letter, you need to support them. Then we may pass you over to interview someone who can tell us why they are such a great catch for us. I love some enthusiasm in a covering letter, but it needs to be directed, know what a company does, find out what you’d be doing, call up, ask, do some research, stand out. If you don’t have work experience that show’s why you are ideal for the job, use personal experience or a time in education when you shined in a club or society.  Yes, those extracurricular things are really handy. You have to looks for ways of show you stand out from the crowd. Expect to get asked about them, if you have done awesome stuff we will want to know more, and we’ll dig deep for things that tell us about who you are as a person.

If you really want a job, show us why, if you are just applying, you may well find you lose out to someone with a passion for the role, they will sell themselves some much better.



MEWT 5, my day

So at the weekend, it was MEWT, a peer workshop for exploratory testers. There was a handful of us from across the UK. the topic for this, the 5th MEWT was, “What is professional testing”. I felt ill-equipped to attempt defining such a contentious thing, so I did a brief talk on my professional growth from definitely not a professional tester towards something that could better be described as containing professional elements My talk can be found here.

This was my first time talking to a group of peers, my first workshop, and conference, (although Mewt isn’t really a conference :)) I was incredibly nervous about giving my talk, especially as the day went on and there were so many very good talks, by some very knowledgeable people.  We got to my talk towards the end of the day, and by all accounts, it was enjoyed Dan Billing kindly tweeted

DanBtweetPassion  Although  at the time the only thing I felt I was an exemplar of was nerves.  The questions were straight forwards and simple to answer, mainly focused around how I got the momentum to make changes in my workplace and the responsibility to do so.

BillMresponsibility I described it, possibly rather glibbly as “stealing” the responsibility, I did this because I was never handed the responsibility to coach my colleagues or push the company to do better at testing, or even define and overall IT test strategy. I just did it, someone had to.

So more than enough about what I did, and on to the day at large.

The venue, at Attenbourgh Nature Reserve, is an excellent location, the room is lovely, well equipped and has a beautiful view, and small balcony for eating lunch on. Well chosen, heartily recommend staying there for as long as possible.

The other attendees are a great bunch of people and it was fantastic to meet them all, there is such a variety of experiences and personalities there, an education in itself.

mewt5 attendees Dan C is the tallest tester I have met, which I’m sure he’ll be pleased about.

Dan C is the tallest tester I have met, which I’m sure he’ll be pleased about.

The talks were very interesting, from the start of the day right through to the end, the high point for me was the meta, “what is, What is professional testing” A talk by James Thomas that I’ll long remember and aspire to be anywhere near as good as, his talk and write-up can be found on his blog  we worth reading.

The direction of the day was largely driven by the differential of professional work over notions of a Profession, and testing over testers. I think it’s right to say that as a group we were far more interested in seeing a good job done by someone, rather a person specifically based in a Profession doing a job, good or bad. I wonder if our predisposition to seeing all the ways something could go, bad and good, hinders our ability to move something forwards, we spoke at great lengths about how a Professional body could be a bad thing, could go wrong, might not live up to the needs of the industry that I doubt as a group we would ever be happy that we had understood the ramifications of instituting a Professional body, and that we had mitigated all the potential pitfalls to a sufficient degree that we would ever get around to starting a Professional body, That and I think we are a contrary bunch that don’t really like being told what we can and can’t do. I do strongly believe that is why we strive to do a good job though. we answer to ourselves, and it’s hard to lie to yourself about how happy you are with the quality of your own work.

So what were my learning from the day, simply put professional work is describable in a general and non-specific sense to a degree that allows a conversation about basic principles, high quality, integrity, honesty, customer focused, etc. However, it must have a context within which its being described if you wish detail, as that context can change whether a behaviour or action is professional, or unprofessional. And that context is poorly described, we all work in a wide variety of differing environments, that constitute differing standards, expectations, drives, and restrictions, to highlight just a few of the environmental constituents. There is a high degree of tacit knowledge about our own environments that would take many workshops such as MEWT to tease out, and the value of such an undertaking is, in my opinion, low at best. So where does that leave us in our understanding of the question asked, “What is professional testing?” we can quickly point out what it isn’t. We have a framework of driving values that we broadly agree are required, they have little to do with testing, and a lot to do with conduct, but why should we expect anything different. professionalism is mostly framed in behaviours that are to do with the conduct of the person towards colleagues, employers, peers and their fellows, and rarely to do with a specific set of technical skills that are used within that profession, so you can work within a profession, and you can be a Professional. Professions in a formal sense are not something we are seeking, yet professional testing is.

By that, we mean the technical work of testing, as to be defined somewhere else, done in a way that has the characteristics of professional behaviour. As defined as conscientious, honest, integrity driven, work towards the end of getting the project completed for the customer, regardless of any sort of title, role boundary, or other arbitrary delineation of task ownership, ie if it needs doing, and you can do it to the quality required (yes quality required is a deep question in its own right. In this sense, I mean to a level that adds value to the overall project objectives with a skill commensurate to doing a “good Job” and not detrimental, to timelines, quality ,other people or the overall project), get on and do it.

I will probably come back to this question and see if I can simplify my reasoning, make the steps easier to follow once I have internalised it further and read around it some more.

Did I enjoy MEWT 5, I most certainly did, I would love the opportunity to attend more peer workshop, and would strongly recommend that should the chance be presented take it with both hands.


Feedback question and challenges most welcome.


Mewt 5 is coming, tomorrow!

So tomorrow is MEWT 5, and I’m attending!

I’m very excited about the opportunity even if I’m some apprehensive about the whole standing up and talking bit. I hope that I can contribute to the conversations, I’m sure I will learn plenty as well.


Looking forward to meeting some more testers, and learning from their experiences, especially if any of them has done project of a similar scale to the one I’m currently involved in.

I will post up about my experiences at MEWT what I’ve learnt, how I found it, for now though I’m just very excited about the workshop.