It’s all about the context

Are you con.text aware ?

Kem Caner in his post on what is context-driven testing writes on the difference between context-driven and context-aware:

“Similarly, some people create standards, like IEEE Standard 829 for test documentation, because they think that it is useful to have a standard to lay out what is generally the right thing to do. This is not unusual, nor disreputable, but it is not context-driven. Standard 829 starts with a vision of good documentation and encourages the tester to modify what is created based on the needs of the stakeholders. Context-driven testing starts with the requirements of the stakeholders and the practical constraints and opportunities of the project. To the context-driven tester, the standard provides implementation-level suggestions rather than prescriptions.”

He then goes on to write:

“Context-driven testing is an approach, not a technique”

You know, I read this about six months ago and whilst I appreciated the difference between the two, I didn’t understand what the big deal was.

Consequently, I’ve been a bit dismissive and slightly irritated by some of the discussions held regarding the differences.   I couldn’t understand the need to differentiate. As long as  ‘context’  is taken into account during testing, who cares what it’s called?

(I suspect you know where this post is heading….)

Of course, I now realise that I was wrong and it does matter.

Because, the difference is all about attitude.

Context-driven testing is an attitude that refuses to confine and constrict software testing to one approach, goal or technique and asks the tester to come to each test exercise without prejudice, assumption or  pre-conditioning.

That doesn’t mean that once the context is understood, any previous learning doesn’t come into play, it’s just that for context-driven testing, you are purposefully open minded about how you will test.

With context-aware, some methods have already been pre-determined. This doesn’t matter if it’s a standard, a checklist or a template, a decision has been made before the testing starts to use some approach.

In fact, a context-driven tester and a context-aware tester may both end up testing a product using the same techniques. The difference would have been,  a context-driven tester makes a decision on the approach after talking to stakeholders whilst the context-aware tester has already decided on approach and technique before speaking to a stakeholder.

I’m still to be convinced about the Unicorn Question though………

By the way, the image does have to do with context. Can you work it out?

6 replies on “It’s all about the context”

(It’s actually Cem Kaner, BTW.)
You write that context-driven testing “…asks the tester to come to each test exercise without prejudice, assumption or pre-conditioning.” That’s close, but not quite true.

We would say that to be context-driven involves (among other things) becoming aware of your prejudices, assumptions, and pre-conditioning, so that you can make real choices about how you test, rather than being on autopilot.

We all have prejudices, assumptions, and pre-conditioning. We differ in how we deal with them.

The issue with approach vs. technique is simple. Think of an approach as an adjective, and a technique as a verb. “Casual” is an adjective. You don’t “do casual”, you do something “casually” or else, perhaps “seriously.” You can take a casual approach to any test technique, or instead you could take a very serious approach to it.

Approaches apply to techniques.

This can be confusing because you can certainly imagine applying techniques to techniques, too. What we’re getting at is that approaches to testing are patterns of thinking and behavior that aren’t very interesting in and of themselves, but when applied to any given technique of testing, cause you to perform that technique in a characteristic way.

A unicorn question is just a question that makes no sense, and therefore should not be answered, because to answer it would likely be unhelpful. “Is there a school unicorn at Mountain Laurel Montessori school?” The correct answer is yes, based on the fact that the questioner meant “uniform”, not “unicorn.” The correct answer is “no” if the questioner actually meant “unicorn.” But the responsible answer would be to clarify the question. “Did you mean ‘uniform’?”

Hi Cem,I won’t comment on your first two points just yet. I need some time to absorb and think about your reply.

However I can comment on point on the Unicorn Question.

I’m not a fan of the concept of the ‘Unicorn Question’ as a reason for not answering questions.

Here’s why:
A unicorn question is just a question that makes no sense, and therefore should not be answered, because to answer it would likely be unhelpful.
So why does a question “make no sense”?

Your example is because of clarity. OK I see your point.

What if a question doesn’t make sense to you because either you don’t understand its purpose or you are not aware of the context?

Does that make it a Unicorn question?

As the person answering the question, do you have to know either before the question can be answered? The context and purpose may help you understand and make you feel better, but perhaps the questioner is aware of that. Perhaps the questioner doesn’t want you to know the context because a) it may cloud the issue or b) it may taint your answer.

Could the Unicorn Question be more about the answerer being comfortable with the question, than with the question making sense?

I’m going to use an example of a question I put on the software testing club a while ago. It was “how long does it take you to write up a bug” which was cited as a Unicorn Question.
Why should we not answer this question?

The question is pretty clear but there is little context or purpose. Does that make the question unanswerable? Not really, but it does invite the answer to step out of their comfort zone to answer the question, or as Michael put it “make it a real issue”

In gaining the purpose and context to the question, immediately the discussion headed off on the rights and wrongs of using bugs to estimate effort. It was insightful, but it didn’t answer the question I wanted.

As well as estimating I was genuinely interested in what people’s averages where in writing up bugs.

To summarise, I go back to the Unicorn Question description:

A unicorn question is just a question that makes no sense, and therefore should not be answered, because to answer it would likely be unhelpful

My feeling is that if a question genuinely doesn’t make sense, its fair to question it, but if you can, leave it up the the questioner to decide if the answer is unhelpful or not.

Hi, Anne-Marie…
It was James Bach, not Cem Kaner, that replied above.

The missing words in all this are “to me”, or “to someone”.

A unicorn question is not necessarily a unicorn question to the person asking it. Indeed, people generally don’t ask questions that don’t make sense to themselves. However, to the person on the receiving end, there are all kinds of ways in which a question (or a statement, for that matter) can seem strange, confusing, or nonsensical without the required context.

I could tell you about a fellow I know who has at least 30 ears. You’d be entitled to think it a ridiculous claim, but if you were confused, I’d hope instead you’d ask me what I meant.. When I told you that he was a greengrocer in my neighbourhood, and that I was talking about corn (maize?), then things would make sense. But you see the problem? I don’t think I’ve said anything odd, but at first hearing the question you do. You can’t believe your ears, because you can’t believe that anyone might have 30 of them. It wasn’t a unicorn statement to me, but it was to you.

The reason not to answer a unicorn question is that an answer outside of a context is unlikely to be helpful. On the contrary; the answer is likely to be unhelpful, or misleading, or both. A responsible person will not give you an answer to such a question without seeking clarification.

You suggest above that the question “How long does it take you to write up a bug?” is pretty clear “but there is little context or purpose.” I would argue that without context or purpose, the question isn’t clear to the degree that my possible answers would help you. My answers would range from zero (I don’t write it up at all; just tell it to the programmer) to infinity (because I leave open the possibility of adding information to the report as I discover something new); what’s the average of that? If I said, “three minutes”, would you be concerned that you’re too slow? If I said “forty minutes”, would you fear that you’re not adding enough detail? What kind of average? A one-day average? Over the life of the project? A few days at the beginning of the project? At the end? For a medical device? For an online dating service? Am I in a regulated environment? A nuclear reactor? A games company? A retail organization? Do I include the things that are not bugs, but requests for information? Enhancement suggestions? Why or why not? And how will it help you?

It doesn’t help that the links on Software Testing Club are in a green that’s nearly indistinguishable from the normal black text. So maybe you missed this: the original question, the one that prompted James’ response about the unicorn, was “How many times should you run a test?” James’ reply can be found here:

All the best,

—Michael B.

Hi Michael,
Thanks for your comment.

I get the point, but in a way don’t you think that many questions in life are like this? In any conversation we will ask an immeasurable amount of unicorn questions. Its just that we make assumptions on the context, without explicitly being told them.

For example, : “How is your day?”

I’m sure you’re not recommending that we reply “that is a unicorn question and answering it is likely to be unhelpful?”

Going back to the 30 ears (I like that example!), I like to think you would still try to answer the question, even if it was
“you idiot, I have two ears, what you talking about”? In this way you have answered the question, but you are also inviting the person (well perhaps not by calling them an idiot) to provide context.

I think the way the unicorn question is described doesn’t lend itself to trying to answer the question. Rather, it seems to dismiss the question at hand as inadequate.

I hope I’ve explained myself well enough.

Hi, Anne-Marie…
Questions have structures and context of their own. The forum in which a question is asked informs the context. The person or people being asked informs the context. The person asking the question makes implications and the person reading it or hearing it draws inferences which inform the context. I’ll make inferences an danswer the question “How is your day?” differently depending on whether it’s my wife asking me in the kitchen, or a colleague asking me at the end of a stressful project, or a doctor asking me the day after surgery.

To me, that’s a dramatically different question, asked in dramatically different contexts, from the one that prompted this thread (what is the average amount of time required to write up a bug report), and the one that prompted James’ original thread (how many times should you run a test case). In those cases, I make several inferences about the question. In both cases, I infer that the questioner is asking about something in a work-related domain; that there are too many free variables to answer the question usefully; that there is risk of me misinterpreting the question and the questioner misinterpreting the answer; and, most importantly, that the questioner will attempt to make some kind of value judgement on his or her own answer in comparison with someone else’s. In addition, the forum for the question is heavily mediated; not only can I not hear your tone of voice nor see your facial expression, but I can’t even project them onto previous experience, since we’ve never met.

Now, I’ll confess that the tagging of such a question as a unicorn question might seem rather dismissive. However, I’d like to be clear that it’s dismissive of the question and of the class of question, rather than of the questioner. My intention in supplying a shorthand and somewhat provocative reply without belabouring it is a kind of backhanded mark of respect: I’m trying to encourage you to spot the pattern of questions like these and think on their lack of utility–because I agree with you: the question is inadequate. I don’t go to that kind of trouble for people who aren’t interesting to me.

—Michael B.

Leave a Reply

Your email address will not be published. Required fields are marked *