Jumping to conclusions

Good testers continuously ask questions about the product, the customer and the project environment and invariably on themselves.
No question, no test.

When were satisfied with the explanation we stop asking questions, we stop being inquisitive. For testers, its essential to keep asking questions for as long as possible.

On the other hand, a test manager deals in conclusions in response to deadlines and an expectation from stakeholders.

This puts a test manager in the unenviable position where on one hand they need to encourage their testers to question, but on the other they need to be able to satisfy their stakeholders with conclusions.

A test manager has to deal with this conflict of inquiry and conclusion in the testing they deliver. If a test manager focuses only on deadlines and delivery, the desire to reach conclusions quickly will filter into the testing they manage. Testers will start feeling the pressure to deliver answers instead of ask questions.

Test managers need to be conscious of the impact deadlines and being project driven can have on a tester’s ability to find bugs.

If you’re a test manager, be mindful of the impact that deadlines & resulting conclusions may have on your testers. Avoid the temptation to drive testing with the goal of delivering ‘the simple answer’ because stakeholders expect it.

Strive for open mindedness and inquiry in testing. Avoid easy explanations and quick conclusions.

Or even better, encourage stakeholders to reach their own conclusions about the testing that’s being performed. Now that would be a real victory!

12 replies on “Jumping to conclusions”

I agree that the ‘product managers are the ones that must draw conclusions, and make product- and project-related decisions based on those conclusions. I’m less convinced that test managers are in the role of making conclusions (other than those related to the management of the testing work and the testers). As a product manager, I didn’t want the test manager to be making conclusions. I much preferred to hear about inferences that could help to inform my choices.
As a tester (and even as a test manager), I prefer to supply inferences. If I’m also a product manager at the same time, then I’m in the conclusions and decisions business.

—Michael B.

Testers and test managers DO have to make decisions but perhaps the decisions they need to make differ to those of product managers. Is it a bug? Is a decision that testers face every time they test.
Test Managers conclude that testing is complete. Product Managers make conclusions about releasing.

–added comment later:

I like what you say about inferences though, that makes sense. Perhaps as testers and test managers we ought to try and steer away from conclusions, regardless of how much stakeholders want us to.

As a TM I might advise, but I rarely conclude that testing is complete. I’m done when my stakeholders tell me that I’m out of time or determine that the risks of not doing further testing are outweighed by the need to ship.

I’ve witnessed the behaviour you’re referring to. It seems quite common amongst TMs who have moved from project management (rather than having been testers) and see their role as “get testing done” rather than “provide information”. I’d also say that this applies to PMs who aren’t terribly good PMs – who are focused on date and delivery rather than having any real insight into the interplay between scope, time, cost, quality.

I started out this way, before deciding to stop being a TM and to become a tester. I now believe that a big part of my role is getting alignment on mission, defining what will constitute “done” and managing the ongoing dialogue about the degree to which we’ve answered the questions we started out with, what remains unknown, and what new questions we’ve raised along the way.


I like what to say about asking questions. I ask a lot. I’m never satisfied with explanations and I always want more of those. My test leader may be a bit annoyed but I don’t care. I want answers and explanation to understand and test better. Keep posting!

“No question, no test.”
Asking questions is very needed, but sometimes that cannot be done. I wrote about what to do in that situation once:

Although I understand that your ‘questioning’ means thinking critically about everything (not only asking questions about the mission, the product and context), my post should still be relevant.

Hi Kristan,

thanks for commenting on my post. I agree its helpful to make a ‘scaffolding of assumptions’ in testing. I use assumptions when I need to simplify the context I’m testing in, or as you suggest in your post, when the information is not available to me.

*But* if all the test consisted of was assumptions, there would be no question and therefor no test.The fundamental question your asking in testing: “Is there a problem”? If you’re not asking that, your not testing.


Hi Anne-Marie. It was good to have heard you talk and to have spoken with you at the STP conference yesterday. I just thought that I’d develop on that thought that I started babbling about yesterday.
Good testers continuously ask questions about the product, the customer and the project environment and invariably on themselves.

Yes they do and the question that I was considering on the way home was what are we doing really when we ask questions?

The conclusion that I came to (OK I had some pre-conceived ideas too) was that we all carry around a model in our minds of the system that we are testing.
When we ask questions, or read a document or explore some version of the system, what we are doing is comparing our mental model with something else, i.e. testing.
The results of that process are either a confirmation of our model or not. In the latter case, we figure out if we need to update our own internal model, aka understanding, or take some actions that result in an external change – or both.

Make sense?

Looking forward to reading the book that you are working on with James Bach. I have good feelings about that.

Hi Duncan,
yes, absolutely that makes sense. Its also why I use a socratic method of coaching which requires that I ask questions of a testers testing model. What I’m doing in coaching is challenging their existing mental model in order to create new understanding, new meaning ultimately leading to new skill. The challenge for me is to do this in a way that motivates a tester through discovering their ability.

You were the one to discover the ‘rule’ at the keynote. Remember how that felt? How encouraged you were? That’s my goal in coaching.

Hopefully we will get to meet again, and discuss this more. Maybe at CAST one year or another STP Conference.


I think when you examine this kind of abstract and strategy oriented thinking, you always have to go back the end goal of testing to begin with, which is IMPROVE QUALITY. So, with that in mind, stakeholders and testers alike need to keep in mind, given the time at hand (deadlines will always exist) what is the best use of my time to increase quality? Sometimes it could be pounding through test cases and giving some type of formal report. Other times it may be more lucid and lax in order to foster creativity in thinking and exploring ways to find different types of defects, or other means to not only increase the product quality, but the end users’ perception of the product quality which may not always be the same.

Hi Phil,
Many thanks for your comment.

Yes indeed I don’t fault your thinking. But even when you do have this as your mission, there are times when you are challenged to make assumptions about what quality is. In fact, my experience tells me that the more challenging the deadline, the greater the temptation to make assumptions about quality.

We need to be creative about understanding quality too, I think.

I’d love to hear more thoughts on this.

Leave a Reply

Your email address will not be published. Required fields are marked *