Categories
insight repeatability reuse

Recession Testing is the new RegressionTesting

Its time to retire the idea of Regression Testing folks. Regression testing, at least the way its being performed today is typically a value free, wasteful exercise and falls into the category of  “bad testing”.
“Regression testing is any type of software testing that seeks to uncover software errors by partially retesting a modified program.” 

In fairness to Regression Testing. I’m not opposed to the above ideology(except for the partially retesting bit, thats stupid).  I think it has some merit. The concept that modifications in code add risk which testing needs to address is a sound idea and worth taking into account while testing.

But we testers know, that this intent turns out to a different beast.

What gets called ‘Regression testing’ in many companies in my view is not very valuable and has a different intention.  Regression testing ends up a packaged set of tests ( I wince to call them that, as they typically have the same idea repeated over and over) that get repeated at the end of testing to validate that nothing’s broken.

It’s more about maintaing the status quo than any in-depth testing. This to me seems wasteful.

It seems wasteful to me to repeat the same tests giving an illusion of repeatability when in reality we know that each test can never be exactly the same and that getting the same result in a test does not mean for certain that the test has passed.

It seems wasteful to me to exercise a feature using the same testing idea again and again when a different test idea might offer new information about the system

It seems wasteful to me to ask testers to perform tasks that result in the tester disengaging from the activity because its boring.

It seems wasteful to suggest that somehow new features need to be tested differently to older features. Why? Bugs are not ageist.

It seems wasteful to suggest that somehow regression testing demands less cognitive and skeptical thinking.

I think we’re looking at this problem the wrong way. I’d like to suggest a different paradigm.

I’d like to offer up the idea of Recession Testing. Where Regression Testing does its best to test as little as possible, Recession testing insists of focusing on value and removing waste, just like we need to do in a Recession in order to be competitive.

This means, instead of splitting a product into new and existing features, lets test all parts of a product with equal aggression, equal skepticism. When it comes to regression vs new feature bugs, lets make all bugs equal, not some more equal than others!

If we do need to prioritise our work, lets do so on the basis of risk, What are the impact of change on the feature?  What is the importance of the feature being tested?

If you’re going to test a feature, do everyone a favour and test it like you mean it! Don’t give it some half baked, wishy washy run over, to check that its ok and then give the false assumption that you’ve properly tested it. Put the feature through its paces. The feature deserves your respect and  more importantly you deserve to test in a cognitively challenging way.

You know it makes sense!

I wrote this post for Kim, one of my amazing testers who wanted to find out more about regression testing.

I’m sure there are many posts on this topic (if you know of some good ones, do us a favour and add a link in the comments).

 

 

Categories
BlogRoll repeatability reuse template test script

Software Test Templates – scoping out your tests

I wrote a blog recently “the value of a software testing process” in which I questioned the traditional benefits of software test scripts based on the re-use and repeatability theory.
My approach to any rapid change software development is to make a some basic assumptions

1) The code is going to change quickly
2) The person who coded won’t necessarily be there for the next release
3) The person who tested the last time, won’t necessarily be there for the next release
4) Test documentation helps to provide structure and is excellent as a guide to ensure adequate coverage
5) Testers are able to think outside the square and aslo be able to fill in the gaps in a test

I use one spreadsheet (if possible) to track the following information;

a) test script number
b) test link (if available which is a reference to requirements etc)
c) test purpose – a clear concise description of what I am planning to test
d) test results – Pass or Fail
e) defect number – I assign a number of use the defect tracking system
f) test estimate – the time I anticipate it will take to test this functionality

I use the document to first scope out what I want to test, a quick and dirty overview of what I am planning. I also use it to estimate how long testing is going to take. This is helpful if I am ever asked to substantiate my estimates.

Once all stakeholders are in agreement onto the scope, I create a concise and descriptive purpose of each test. This is in essence my test script. I use the same spreadsheet to enter results, defect numbers and to calculate simple metrics such as the pass rate.

I have uploaded an example below;

Software Testing Template

I hope you you find this post useful. Please feel free to use the ideas I have here. Do us a favour though, leave a comment, or digg the post. Thanks !!

Categories
BlogRoll repeatability reuse

The value of a software testing process

Does a software testing process provide a best approach to testing? Coming from a background in engineering I have always been a firm believer in the benefits of a sold and dependable testing process which includes test planning, design, building, execution and reporting. So, how come I find it hard to convince others of its merits? After much soul searching, I’ve come down to the conclusion and it’s this. A rigid and formal testing process doesn’t work in many of today’s software projects. For example, some of the benefits of a software testing process are to providere-usability and repeatability of results Re-use is cited as one of the benefits of a software testing process as it reduces long term cost both in resources and time. For example, test scripts once written can be used again in future releases reducing the upfront resources.
But, in my experience, I’ve rarely used the same test script twice. This is because each time a new release is planned, its markedly different to the previous one. Add into the mix, new developers, new testers and soon the amount of flux necessitates re-starting the test planning, design and building from first principles. I question the value of re-use when it ends up costing similar or more in time and resources.
Another benefit of a solid testing process is the repeatability of the results. Again though, if your code change is that significant, you will have to question the validity of the expected result. If resources are required to confirm or update the expected results, I’d have to argue time may be better spent in beginning again. This is especially true if the software tester is new to the project and requires analysis time to fully understand a product and its environment.
In summary,
I believe that where rapid change exists, you need an adaptable approach and is lighter on documentation
In industries with little change there is much benefit to a repeatable, re-usable approach, but where rapid change exists, I think spending lots of energy on paper documents which in the end will only get used once is a waste of time and cost.
Testing is essential, just that the process to perform testing must always be up for review and change too!
my two (Aussie) cents for the day….