Test Planning

When you are planning for your tests, either small-scale plans, just to look at a single feature or change, or more complex plans for testing and application there are a few steps you can go through to help ensure you have to revisit your plan only when your understanding of the SIT changes rather than the requirements for the plan changing.

Be clear from the start who is using your plan, is it just for you to help plan what you are going to test? is it for the test team to allow for a shared understanding? or will it be used by the wider project?

Having an understand of this will enable you to include the information that the rest of the audience need for the plan to be helpful for them. You may find that a single plan isn’t sufficient as different people require dramatically different information.

Where the test team may only need a loose plan to help the achieve good coverage the project manager could be wanting to understand resources or duration and stakeholders may want to be able to see that what they are worried about is getting sufficient testing to give them comfort once live it will be reliable.

Understanding the differing requirements for any plan you make can help you produce one that is fit for purpose, in my experience the easiest way to do this, is to create an outline of the plan, something that doesn’t take too much time to do, and share it with those who will be looking at your final plan, speak with them about what it tells them and what they need to know that they can not see, incorporate what you can, make compromises where there are too many conflicting needs.

This will help you create a plan that works well for most people the first time and hopefully reduce the number of times you need to revisit the plan.

Technical, Non-Technical Testers, and what does that even mean!

So it turns out that I get quite annoyed when the terms Technical, Non-Technical testers are used to describe what my team of testers should, and should not be doing! I was recently in a heated conversation about getting access to read some SQL based ETL’s and one of the reasons for us not to have visibility of this was my team are not technical testers, and as such shouldn’t, and by implication, can’t read such things.

So let’s explore for a minute what I think when I hear those words, I think more about someone who builds tools to use in testing vs someone who does not, what I’ve also heard called test engineers and test analysts. At no point does it cross my mind that you’d be talking about a code illiterate person or even someone incapable of looking up the bits used to get a general understanding.

So upon hearing it used in this way has caused an irrational response, one of annoyance. I can’t help think that maybe it’s down then to how I think they see a person they are describing as incapable of reading code, my insecurities rather than their intent? beware your own biases they can really colour a conversation.

Unfortunately, the knowledge that I’m annoyed because of my personal interpretation hasn’t reduced my annoyance at the whole situation, life’s a bummer.

I disagree with the test team not having access to the ETL’s as I personally feel the documentation of them isn’t as good as the ETL’s themselves, sadly it’s not my call to make at the end of the day, we work with what we have, and I have to put my frustration aside about this and get one with the many tasks I still have to do.

How do you deal with situations that have both  an emotive content for you, and you disagree with from a project/task perspective?

 

Doug

Training New Testers

So as my more regular readers will be aware, at work, we are changing some major systems for something a bit more modern. This means more testing is required than my test team is able to do. So we’ve recruited some more people to test. I decided that the  best approach for this was to recruit internally for people who had the familiarity with the business processes that are impacted by the software change that I could teach how to test.

Now there are obvious risks to this approach, my ability to assess someone’s ability to learn quickly, have an appropriate mindset for good quality testing, my ability to train them, the list goes on. However, I felt that connection with the business was worth the risk, the ability to see what’s going to be a challenge in the departments, and what’s going to work well, plus the increased trust for the business by having key people involved in the testing process.

So the challenge that I have is to train them all to be great testers, well as great as I can help them be. There are plenty of great resources out there, videos, blog, books etc. However there are 5 of them, and we have to be testing the system at the same time as I’m training them. So I recruited them all earlier than I actually needed them, so they could start learning before it became critical. My first step was to get them to watch James Bach’s open lecture, I used this as a starting point as James has an engaging style of presentation, and the lecture covers some of the basic challenges in testing in an easy to understand way, with little need for the watcher to already understand testing or a great deal about software development. I also ran some training on the basic housekeeping tasks that make up the test process we are using here, session notes, timings, bug reporting etc. Plus a have a small selection of books that they can borrow which cover aspects of testing.  We also have regular open discussions about testing where we can share what we have learnt, and ask questions about how to do things. I’ve also given each of them a cheat sheet with some helpful heuristics and information about data types.

I have also encouraged them to spend a little time every week on some self-directed learning about software testing, which I ask them about, and when they have found something that is useful to the team to share it with them. I have also encouraged them to just log into the system, have a poke about, see how stuff works, and practice session note taking while doing so. This has been of great use, it’s uncovered a few bugs, and given me the chance to run a few debrief’s, and provide some feedback on both the notes they have taken and how they have thought about what they were doing while logged into the system.  My next challenge is to give them a grounding in some test methodologies, without confusing them, so they can start applying differing approaches to what they are doing.

 

How have you approached training new testers? Do you have ready-made material or favoured techniques?

Test planning, so you want more time!

If you’ve been involved in test planning you’ve no doubt encountered the questions, “Can you do it in less time?, What if you did test like this, is there a quicker way? what if we get some of the business to do all the testing for you?”

There are more questions like these as well, they all tell us something about the questioner, about how they are thinking, what matters to them, and they give you options to talk to them about. Testing can normally be done quicker, it’s just a question of do you want the risk that is associated with doing so?

So we are near the end of a change of software in part of the warehouse, there’s loads of config, some development, and a tonne of business understanding to gain. This is the second time we’ve tried this, the first time the business wasn’t at all happy with the config, so it was back to the drawing board for the project team. So now we are near the end and there won’t be a third chance so it has to go in, and be right. I haven’t been on this project, as you may have read in other posts my focus has been on the ERP we are putting in. However they are near the end and are worried it’s not going to happen, so I’ve been moved back to this one and basically asked to make the testing fit in the short time that is left. It’s all driven by the overall project budget, basically getting more money is going to be hard, so I have to look at what’s left and work out what not to do.

It’s all about risk here, the work is largely split into to pieces, the interface work, moving data via a bus between the new system and existing systems, and the new warehouse system itself. The interfaces have to work, but we control most of how the data arrives, so we can do a reasonable job on that without having to do to deep a piece of testing, and the warehouse system has had people all over it for months, there are in fact a group currently signing off business processes and user stories against what the set up in the system is, so they will have a good chance of seeing if the basic flows don’t work. So my sales pitch is more focus on the interfaces, less on the new piece. The focus of testing on the new warehouse piece would be the edge cases that the team signing off the basic flows are unlikely to go near.

Yes the state of what’s going on is far from ideal, but it rarely is, so we work with what we have, I’m often reminded of the scene in 13th warrior where they are on the boat and the northmen are laughing at the storm, show no fear, laugh at danger. There’s no point bemoaning the state it’s all in, get on and do what you can, make it work as best you can with the time and resources you have, flag risks if they are too much for the business to bare they will say. Your job is to let them know what these are and do the best you can to help get the project past the go live.

 

 

Long Chain Bugs

We’ve all seen them, your testing away, and something odd happens, application crash, unusual error message. you look back at what you were up to, retracing your steps waiting to see the same behaviour repeated, yet nothing shows, you end up digging around into system state, other applications that were running as well, other people who were in the environment as well, sometimes we can work it all out, a lot of the time, at least I, can’t. Well as I found out recently at a medical appointment I attended with my other half, it would seem that the medical community has long chain errors as well.

My partner has Lyme, she’s been very unwell with it for a number of years, and we recently had the chance to talk to a top-flight consultant about her condition. What they explained was Lyme Borreliosis Complex (LBC), as they have dubbed it, is a complex condition that has only in part to do with a single event. The system state also needs to have a variety of conditions meet for that bite to then cascade into the horrible condition that LBC is. They are not sure what the precise conditions need to be, they believe its a combination of genetics, familiar health, and environmental.

At, this point I had to try very hard not to hijack the appointment and start talking with them about how they are researching this and have they seen the very cool ted talk about a multi-disciplinary team that tackles big challenges that have eluded specialists.

It has made me think about how people who have practised testing, understand the mindset required to untangle these complex scenarios could have incredibly interesting and fulfilling jobs if we could convince other communities of the value and benefit we could bring.

Have any of you had the chance to bring your testing skill set to unexpected places?

Understanding risk from the perspective of stakeholders

A key part of getting software signed off is having the key stakeholders happy that it does or is what they wanted. So how do you make sure you address this in your testing?

There are a few different approaches that you can take to gain an understanding of the risks from the perspective of a stakeholder, at there core though there are a few skills which you need to use regardless of if you interview, produce an information gathering form, or run a brainstorming session, to name a few ways.

What you are trying to do is gain an understanding of their real needs, and the stakeholders may not understand what these are in the early stages of a project, or at least in any great detail. You need to draw these out of them with the use of open questions that lead them to think about the details that you need to understand. You have to do this without imposing your concerns on the conversation. Yes, it needs to be a conversation, it’s not a question and answer session, you have to build rapport and trust with them so that over time they will volunteer this information to you as they are beginning to trust your judgement and that you will take what they say seriously.

You need to set the initial conversation up wisely. You have to give the stakeholder plenty of warning about what you want to do, and what you want to get out of the conversation so that they can prepare. You’ll also need to prepare, make sure you can provide answers to any questions they are likely to ask that you have accurate information about, however, you have to be able to say, ” I don’t know that yet” or, “let me look into that”. nothing will damage the relationship you are trying to build faster than guessing at answers and getting it wrong.

It’s okay to go off script, when it is done for good reason, to gain greater understanding, try to bring it back on track at the appropriate time, when you have explored the topic you were discussing before you or they have started to ramble or moved too far from what you wished to talk about.

You’ll need to practice active listening, being flexible about your questions and asking questions to draw out more details, these are not easy skills and deserve a post in their own right, so I won’t go into them in detail hear. I’ll just say you need to be paying close attention to what you are being told so that you can ask the right questions, at the right time to draw greater detail out, all while writing down the important pieces of information.

you can also make great use of silent pauses to draw further details from them, people will naturally want to fill gaps in conversation, and you can use them when you think someone is holding back. Just don’t go overboard, it’s not an interrogation. You have to practice the suspension of the ego, as these conversations are not about what you think, they are about what the key stakeholders think. You have to cultivate your curiosity about what other think, this will help you forget about your opinions on the matter as you seek a greater understanding of the thoughts of others.

You have to practice the suspension of the ego, as these conversations are not about what you think, they are about what the key stakeholders think. You have to cultivate your curiosity about what other think, this will help you forget about your opinions on the matter as you seek a greater understanding of the thoughts of others.

And remember this is an ongoing conversation, you should get back in touch with these people on a regular basis, even if it’s just to let them know you are still actively using the information they have provided as a form of measure against the quality of what you are looking at. If you maintain good rapport with them, they will more naturally include you in conversations about what their concerns are, and if you have done it well they may even come straight to you when they have concerns or questions.

 

Let me know your experiences of build an ongoing relationship with key stakeholders, and how you went about it.

 

Doug

 

 

Recruiting a new tester

So it’s back to one of those challenges again, I needed to recruit a tester. They didn’t need to be hugely experienced, or possess any sort of specialisation, so that was a plus at least.

We have an unusual recruitment process at work so that always adds to the challenge of getting the right person. We had the normal run of the mill applications, we spend quite a lot of time looking at people’s covering letters, and seeing what they tell us about the applicants. Quite often a poor covering letter will mean we don’t interview someone, as it doesn’t tell us the person has the qualities and skills that we are looking for in a role.

It might sound harsh, but if your first contact with us, your covering letter, doesn’t sell why you are the best possible person for the job, and this means talking about actual experiences that highlight all those great skills and qualities, not just saying, you’re a team player, tell us why. What experiences show that’s the case. Anyone can write a bunch of buzz words on a letter, you need to support them. Then we may pass you over to interview someone who can tell us why they are such a great catch for us. I love some enthusiasm in a covering letter, but it needs to be directed, know what a company does, find out what you’d be doing, call up, ask, do some research, stand out. If you don’t have work experience that show’s why you are ideal for the job, use personal experience or a time in education when you shined in a club or society.  Yes, those extracurricular things are really handy. You have to looks for ways of show you stand out from the crowd. Expect to get asked about them, if you have done awesome stuff we will want to know more, and we’ll dig deep for things that tell us about who you are as a person.

If you really want a job, show us why, if you are just applying, you may well find you lose out to someone with a passion for the role, they will sell themselves some much better.

 

 

MEWT 5, my day

So at the weekend, it was MEWT, a peer workshop for exploratory testers. There was a handful of us from across the UK. the topic for this, the 5th MEWT was, “What is professional testing”. I felt ill-equipped to attempt defining such a contentious thing, so I did a brief talk on my professional growth from definitely not a professional tester towards something that could better be described as containing professional elements My talk can be found here.

This was my first time talking to a group of peers, my first workshop, and conference, (although Mewt isn’t really a conference :)) I was incredibly nervous about giving my talk, especially as the day went on and there were so many very good talks, by some very knowledgeable people.  We got to my talk towards the end of the day, and by all accounts, it was enjoyed Dan Billing kindly tweeted

DanBtweetPassion  Although  at the time the only thing I felt I was an exemplar of was nerves.  The questions were straight forwards and simple to answer, mainly focused around how I got the momentum to make changes in my workplace and the responsibility to do so.

BillMresponsibility I described it, possibly rather glibbly as “stealing” the responsibility, I did this because I was never handed the responsibility to coach my colleagues or push the company to do better at testing, or even define and overall IT test strategy. I just did it, someone had to.

So more than enough about what I did, and on to the day at large.

The venue, at Attenbourgh Nature Reserve, is an excellent location, the room is lovely, well equipped and has a beautiful view, and small balcony for eating lunch on. Well chosen, heartily recommend staying there for as long as possible.

The other attendees are a great bunch of people and it was fantastic to meet them all, there is such a variety of experiences and personalities there, an education in itself.

mewt5 attendees Dan C is the tallest tester I have met, which I’m sure he’ll be pleased about.

Dan C is the tallest tester I have met, which I’m sure he’ll be pleased about.

The talks were very interesting, from the start of the day right through to the end, the high point for me was the meta, “what is, What is professional testing” A talk by James Thomas that I’ll long remember and aspire to be anywhere near as good as, his talk and write-up can be found on his blog  we worth reading.

The direction of the day was largely driven by the differential of professional work over notions of a Profession, and testing over testers. I think it’s right to say that as a group we were far more interested in seeing a good job done by someone, rather a person specifically based in a Profession doing a job, good or bad. I wonder if our predisposition to seeing all the ways something could go, bad and good, hinders our ability to move something forwards, we spoke at great lengths about how a Professional body could be a bad thing, could go wrong, might not live up to the needs of the industry that I doubt as a group we would ever be happy that we had understood the ramifications of instituting a Professional body, and that we had mitigated all the potential pitfalls to a sufficient degree that we would ever get around to starting a Professional body, That and I think we are a contrary bunch that don’t really like being told what we can and can’t do. I do strongly believe that is why we strive to do a good job though. we answer to ourselves, and it’s hard to lie to yourself about how happy you are with the quality of your own work.

So what were my learning from the day, simply put professional work is describable in a general and non-specific sense to a degree that allows a conversation about basic principles, high quality, integrity, honesty, customer focused, etc. However, it must have a context within which its being described if you wish detail, as that context can change whether a behaviour or action is professional, or unprofessional. And that context is poorly described, we all work in a wide variety of differing environments, that constitute differing standards, expectations, drives, and restrictions, to highlight just a few of the environmental constituents. There is a high degree of tacit knowledge about our own environments that would take many workshops such as MEWT to tease out, and the value of such an undertaking is, in my opinion, low at best. So where does that leave us in our understanding of the question asked, “What is professional testing?” we can quickly point out what it isn’t. We have a framework of driving values that we broadly agree are required, they have little to do with testing, and a lot to do with conduct, but why should we expect anything different. professionalism is mostly framed in behaviours that are to do with the conduct of the person towards colleagues, employers, peers and their fellows, and rarely to do with a specific set of technical skills that are used within that profession, so you can work within a profession, and you can be a Professional. Professions in a formal sense are not something we are seeking, yet professional testing is.

By that, we mean the technical work of testing, as to be defined somewhere else, done in a way that has the characteristics of professional behaviour. As defined as conscientious, honest, integrity driven, work towards the end of getting the project completed for the customer, regardless of any sort of title, role boundary, or other arbitrary delineation of task ownership, ie if it needs doing, and you can do it to the quality required (yes quality required is a deep question in its own right. In this sense, I mean to a level that adds value to the overall project objectives with a skill commensurate to doing a “good Job” and not detrimental, to timelines, quality ,other people or the overall project), get on and do it.

I will probably come back to this question and see if I can simplify my reasoning, make the steps easier to follow once I have internalised it further and read around it some more.

Did I enjoy MEWT 5, I most certainly did, I would love the opportunity to attend more peer workshop, and would strongly recommend that should the chance be presented take it with both hands.

 

Feedback question and challenges most welcome.

Doug

Mewt 5 is coming, tomorrow!

So tomorrow is MEWT 5, and I’m attending!

I’m very excited about the opportunity even if I’m some apprehensive about the whole standing up and talking bit. I hope that I can contribute to the conversations, I’m sure I will learn plenty as well.

 

Looking forward to meeting some more testers, and learning from their experiences, especially if any of them has done project of a similar scale to the one I’m currently involved in.

I will post up about my experiences at MEWT what I’ve learnt, how I found it, for now though I’m just very excited about the workshop.