Categories
BlogRoll business statistics

How low do you go?

A survey* sponsored by the Australian Computer Society, found the following percentage allocation for software testing:
5% allocate more than 40% of the initial development budget to testing

25% allocate between 10 -19% of the initial development budget to testing

25% allocate between 20 – 29% of the initial development budget to testing

14% allocate between 30 – 39% of the initial development budget to testing

12% allocate less than 10% of the initial development budget to testing

I’ll leave you to make up your own mind about these stats.

*A preliminary survey on software testing practices in Australia by S.P Ng, T. Murnane, K. Reed, D. Grant and T.Y Chen

Categories
BlogRoll sustainability

Sustainable software testing

Its time to expand our traditional view of a software testing scope and to start including the word sustainability into software testing.
Up to this point, areas of focus have been functionality, performance, maintainability, security, usability etc. As customers become energy conscious, a focus on sustainability will become more important.

Some ‘sustainable’ test heuristics could be:

Sustainability: how energy efficient is the application under development?
Peripherals: how well does the application perform using energy efficient peripherals
Monitor:Does the application function properly under “minimal power configuration”
Switching Off: How well does the application function when peripherals are switched off
Virtualization: How well does the software function in a virtual environment
Printing: What alternatives does the application provide to printing (e.g PDF )

Categories
benefits BlogRoll business cross browser testing template

Getting ROI from your freelance software tester

Development houses have a right to expect a lot from a freelance tester.
Firstly, without the endless budget of some larger companies, they can ill afford time and money caused by improper scoping and testing.

Secondly, they have recognised the advantage of having an independent review of the product and that in its own right deserves to be appreciated.

In order to get the best return on investment from their tester, communication of priorities and expectations must be passed onto the tester. With knowledge, testers can focus their effort on what developer priorities instead of what they suspect is their priorities.

I’ve created a short questionnaire to clarify to developers what a tester needs. It contains some of these areas:

1) What do you as a developer value most? Consistency, Quality, Breadth of testing,
2) What specifically do you want tested in web testing?
3) What technology are you using?
4) What testing has already been performed?
5) Do you have any specs of any sort?
6) Is this new software, or updated software
7) What sort of feedback do you want? Defect reports, results?

Note: These questions have been created with web testing in mind, but can be changed for any type of testing

Here’s an example of what I use

Customer survey (pdf reader required)

Having this information upfront helps everyone because:
1) There is an agreed understanding of the scope of testing
2) Quotes can be validated through the questionnaire
3) More upfront information maximises your return on investment allowing focus on customer driven testing
4) It provides an insight into the software testing process

The above information is used to create tests in a spreadsheet which I use to track defects and results.

See example:Software Testing Template(pdf reader required)

With this document you have the benefit of evidence of testing which can be useful for contractual purposes. It also assists in future upgrades, streamlining the next round of testing.

Like what your read? Please feel free to use the ideas I have here. Do us a favour though, leave a comment, or digg the post. Thanks !!

Categories
BlogRoll repeatability reuse template test script

Software Test Templates – scoping out your tests

I wrote a blog recently “the value of a software testing process” in which I questioned the traditional benefits of software test scripts based on the re-use and repeatability theory.
My approach to any rapid change software development is to make a some basic assumptions

1) The code is going to change quickly
2) The person who coded won’t necessarily be there for the next release
3) The person who tested the last time, won’t necessarily be there for the next release
4) Test documentation helps to provide structure and is excellent as a guide to ensure adequate coverage
5) Testers are able to think outside the square and aslo be able to fill in the gaps in a test

I use one spreadsheet (if possible) to track the following information;

a) test script number
b) test link (if available which is a reference to requirements etc)
c) test purpose – a clear concise description of what I am planning to test
d) test results – Pass or Fail
e) defect number – I assign a number of use the defect tracking system
f) test estimate – the time I anticipate it will take to test this functionality

I use the document to first scope out what I want to test, a quick and dirty overview of what I am planning. I also use it to estimate how long testing is going to take. This is helpful if I am ever asked to substantiate my estimates.

Once all stakeholders are in agreement onto the scope, I create a concise and descriptive purpose of each test. This is in essence my test script. I use the same spreadsheet to enter results, defect numbers and to calculate simple metrics such as the pass rate.

I have uploaded an example below;

Software Testing Template

I hope you you find this post useful. Please feel free to use the ideas I have here. Do us a favour though, leave a comment, or digg the post. Thanks !!

Categories
benefits BlogRoll business

Benefits of Software Testing

Looking at the benefits of software testing from a business perspective can be quite a challenge if your a blue-hat, IT type of person as I am. To sell testing effectively though, its helpful to view testing from the perspective of the person who ultimately gets to make the decisions. So here goes!The way I see it, business is about Profit and Loss. So I’ve split up business as follows ;
1) Business want make money through sales
2) Business want cost cutting to improve the profit margin

Business want to sell things

If business is about selling products or services, how in business terms can software testing help sell a product? I’ve come up with the following possibilities;
1) Software testing discovers if critical functionality works. This is helpful to know when your trying to sell something (I’m assuming!)
2) Software testing makes sure that your product doesn’t negatively affect interacting systems. I suspect this helps encourage repeat sales.
2) It provides tangible results which can be used to sell the product. For example, you can use your proven high performance as a selling point
3) It demonstrates delivery from a contractual perspective through acceptance testing
4) It gives confidence to those selling the product. There is added benefit in knowing the product your selling works.
5) Certification can provide business a selling point. If your system conforms to a technical standard, it may help your product to be perceived as reliable.

Business want to save money

How can testing help save a company money?
1) Early fault detection reduces the cost of fault detection. The earlier a defect is found, the less development rework and re-test is required, minimising its implementation cost. The Baziuk Study (1995) estimates the relative cost to repair a defect found in Operations to be between 470 – 880 times the amount found in the Requirements phase of the lifecycle*
2)It delivers efficiencies in the software development process through metrics such as root cause analysis . These detect possible areas of improvement for software development.
3) Software testing is the source of information such as defect reports, metrics and results that assist IT perform their roles efficiently. Project managers rely on metrics to report on progress, operations, on tangible results to extrapolate future hardware requirements and developers on defect reports to fix their code.

As is plainly obvious, I do not a heavy background in business, so I’d be most interested in comments from those who have! And testers, how have you helped your company save money through testing?

*National Institute of Standards and Technology, May 2002

<a href="http://technorati.com/tag/software+testing" mce_href="http://technorati.com/tag/software+testing" rel="tag"> software testing

Categories
BlogRoll

Benefits of a software testing club

I’ve discovered Rosie Sherry’s excellent software testing club at

http://club.drivenqa.com/

Its great because you can setup your own local group and since I live in Australia I set up one for the ANZACS.

I think we need a logo or picture though? Anyone any ideas?

my two (aussie) cents for the day….

Categories
BlogRoll repeatability reuse

The value of a software testing process

Does a software testing process provide a best approach to testing? Coming from a background in engineering I have always been a firm believer in the benefits of a sold and dependable testing process which includes test planning, design, building, execution and reporting. So, how come I find it hard to convince others of its merits? After much soul searching, I’ve come down to the conclusion and it’s this. A rigid and formal testing process doesn’t work in many of today’s software projects. For example, some of the benefits of a software testing process are to providere-usability and repeatability of results Re-use is cited as one of the benefits of a software testing process as it reduces long term cost both in resources and time. For example, test scripts once written can be used again in future releases reducing the upfront resources.
But, in my experience, I’ve rarely used the same test script twice. This is because each time a new release is planned, its markedly different to the previous one. Add into the mix, new developers, new testers and soon the amount of flux necessitates re-starting the test planning, design and building from first principles. I question the value of re-use when it ends up costing similar or more in time and resources.
Another benefit of a solid testing process is the repeatability of the results. Again though, if your code change is that significant, you will have to question the validity of the expected result. If resources are required to confirm or update the expected results, I’d have to argue time may be better spent in beginning again. This is especially true if the software tester is new to the project and requires analysis time to fully understand a product and its environment.
In summary,
I believe that where rapid change exists, you need an adaptable approach and is lighter on documentation
In industries with little change there is much benefit to a repeatable, re-usable approach, but where rapid change exists, I think spending lots of energy on paper documents which in the end will only get used once is a waste of time and cost.
Testing is essential, just that the process to perform testing must always be up for review and change too!
my two (Aussie) cents for the day….

Categories
BlogRoll

Kid in a sweet shop

I have the dream job at the moment …. I think. I’ve been asked to create a testing methodology for a company that wants to implement a software testing processe. Fantastic! I thought to myself, just what I’ve always wanted to do.

So I busily set about talking to people and finding out what they wanted and finally 3 months later we had this lovely document with templates which sits pride of place on my desk. And thats the problem, its about the only place where you can find it.

I did a couple of things, like creating a test office(they don’t have a test team..yet) with representatives from each department. I also did some training with the test office and for each department on an individual basis. I then went away and left them to their own devices. Funnily enough, I got a call back 2 months later asking for more assistance!

So I came back and found the following problems

1) Some people just don’t get it. Not everyone, but a large enough group don’t seem to understand the point and to them its just another bit of paperwork to get through (or not as seems to be the case).

2) They don’t seem to have time to implement it.One of the things I said at the start was that it was going to take more time to test in a structured way and to prepare for it, but the reality is that despite additional budgeting they still don’t seem to have the time to perform the necessary tasks such as test preperation.

3) They feel its just too much work to introduce testing concepts such as creating a test plan, test scripts and reporting, so they don’t do it. I haven’t had much success on the defect tracking front either (excel spread sheets)

Its getting to the point where senior management are wanting results (no big suprise there!).

So I feel a bit like a kid in a sweet shop with all these different things to work on but I’m not too sure where to start.

This is what I have come up with:

1) Go to basics. Use minimum documentation (I’m thinking one spreadsheet) to take note of test script objectives, test log, defects and test report. Get them to use that and start taking some metrics down.

This will demonstrate that a) their is some form of process working and b) the metrics may prove vaulable in the future.

2) Get project managers to track actual effort against the project schedule to start tracking the test preparation, execution and reporting effort.

3) Go to the senior manager and ask him what he wants to get out of the process (what are his goals). Then work out the problems associated with the goals, work out some actions and try and benchmark against those goals.

Thats about it so far, I’d value anyones comments on this one!

my two (aussie) cents for the day….

Categories
BlogRoll

The right people for the right job

I want to share my ideas on software testing. Software testing is in my blood, which a lot of people are really happy about because then they don’t have to test or test as much anyhow. To be perfectly honest, thats how I like it.

My soapbox topic this week to my client has been “make sure you use people who like testing software to perform any testing tasks”.

I have seen clients trying to save money by getting business or support to perform testing. This ‘cost cutting’ exercise often results in low quality testing and delayed projects.

Experience is required to understand what testing is trying to achieve. If you don’t get the concepts of a structured process its hard to be expected to pick up these skills immediately.

Also, more often than not, every excuse as to why the job can’t be done is thrown at you, which ends up delaying the project.

You wouldn’t normally ask a developer to system test, so why ask a production support person to?

Get the right person for testing and you will save yourself and everyone else around you a lot of heartache and more money at the end to celebrate the success.

my two (aussie) cents for the day….