Shallow Testing gets a bad rap

In my Exploratory Testing class, I talk about shallow and deep exploration. When you say the word exploratory testing, many think, ‘just playing around’ on the product. And to some extent that’s true, a good true. So when I talked about shallow exploration it seemed to fit. Different from deep exploration where a systematic approach is used to observe, model, use and evaluate the product.

More often, I’m seeing the phrases ‘shallow testing’ and ‘deep testing’. These to me are problematic. Shallow testing used in this way to suggest that simple functional testing (potentially automated) is trivial. Deep Testing, on the other hand, requires going beyond the superficial and delving into the product often at a technical level. An example of each would be, testing at the GUI level,(Shallow Testing), removing the covers and performing testing under the surface(Deep Testing)

Right now I’m with a client and I’m performing functional tests that have been written in a test case management tool. I specifically avoid any kind of investigative testing. I perform the test and I pass or fail the test. Is this testing confirmatory in nature? Yes! and what’s more, this is providing huge value to my client.

There’s nothing trivial about it and by using the word shallow it dismisses the importance of this type of testing to a stakeholder. There is nothing like a functional test to give confidence to a business user that the product is ready to go.

There’s something else. These tests are often essential beginnings to discovery. When you have a largely unknown problem space that needs exploration, we first need to make sense of what we are working with. In David Khlar’s book “Exploring Science: The Cognition and Development of Discovery Processes” a common heuristic was to confirm an initial hypothesis which allowed a springboard to further investigation.

So what’s the alternative? Anything similar to shallow will have the same connotation. Perhaps we don’t need a distinction at all? Is it really useful to describe our testing in these terms? Most people want a shippable product and don’t give a flying koala what the software testing is called. What’s more, do you hear developers talk about deep coding and shallow coding?

If you really want to classify testing what about the old but familiar black box testing and white box testing?