2020: The year of dumping old heuristics

When software testing, it’s handy to use mnemonics/heuristics to refer to. They can be useful as generators of test ideas, or reminders to test in a particular way, or to consider some aspect.

What’s a heuristic? Think of a heuristic as a rule of thumb that helps you make a decision about a problem you’re trying to solve. How do we get to the fireworks for New Years Eve? Rule of thumb says its easier to take public transport. But that’s not always the case.  Maybe with 10 small children, its safer to go by taxi-bus.

Heuristics are great for one you need to make a decision where there’s a whole lot of uncertainty and complexity and you don’t have all the time you want. It stands to reason that the software testing community has been using heuristics since the previous century. In an software engineering profession where complexity and uncertainty rules, they have become a tester’s best friend.

Heuristics help simplify decision-making by providing patterns that can be useful in some situations, some times.

Notice how I put a whole load of provisos in that sentence? That’s because its useful to treat heuristics with a certain amount of disrespect. Heuristics can’t be applied like a checklist, something you diligently tick off like your shopping list at the supermarket. Think instead of a laundry list*, only tick the ones you absolutely need because it’s so damn bloody expensive to get clothes cleaned at a hotel. I mean, sure who wouldn’t love their underpants ironed and pressed, but do you really need it?

Heuristics can be amazing tools but they can also stop you thinking about other options. Many, many testers are familiar with the San Francisco DePOT heuristic. SFIDPOT** is a mnemonic to describe a set of heuristics to identify product coverage. But here’s the thing. SFDIPOT is not necessarily useful for testing stories because it explores at wrong level of abstraction.  Stories are not products, so trying to apply that type of coverage to a slice of a product can prove to be an ineffective method of identifying valuable story coverage.

When I work on testing a story, I like to break it down as much as possible. Think of a typical story structure…As <user x> I want to do <some action> so that I can <achieve some outcome>.  To break that down I use the following questions:

Heuristics can be really useful to us in testing. Sometimes they help us think of new ways to test, prompting us to ask new questions about the story (or the product) and the context in which we’re testing.

But they can also bias us, channelling our ideas about testing into narrow ruts that over time make it hard to know any other route.

Here’s some questions to ask yourself that may prevent those soft malleable laundry lists to becoming hard concrete checklists:

• Ask yourself what has changed outside my frame of reference and how might that impact me and my testing?
• Ask yourself, given a difference in context, what ideology could I retire?
• Ask others testing (technical and/or product) what they think are good patterns to investigate
• Ask team members what product wise concerns them the most
• Ask team members what’s stopping them from doing their work optimally?
• Ask your head of engineering what information would they ideally like to have?
• Ask operations what’s keeping them up at night (literally)

I find that there are consistent themes in quality that most companies wish to adopt and retain. For example, we want products that do what they are supposed to do. We want websites that perform under heavy than usual load. Increasingly we want other quality attributes that are becoming essential to contemporary engineering practises. Security is key. Speed to Market is key. Smaller &  key. Adopting newer technologies to mitigate risk is key.

Sometimes, just sometimes, we in software testing love to challenge others in software engineering about handling uncertainty and complexity, but our own behavior fails to reflect that.

How about in 2020 we ask ourselves this question? What ideas from people outside of software testing might I learn about and might be worthwhile embracing?  I feel if we can embrace this ideology, then we will bring so much value an benefit into our lovely community.

Anyway, that’s it for 2019, I’m off to celebrate the fireworks. Happy New Year!

*Laundry Lists is an idea I got from Gerry Weinberg

** SFDIPOT comes from work by  James Bach and some others I believe, I tried finding it on the satisfice website but no joy. If someone has a link to the original link, please let me know.

Like this post? Here’s a couple of others you may want to read;