The DSI Test team attended the latest IT@Cork event hoping to gain some ‘Tricks or Treats’ that they could use to improve their automated test process and they weren’t disappointed. In addition to ‘Tricks and Treats’ Ken Brennock from Insight Test Services also offered some extremely helpful advice and some ‘Tips’, not least, that test automation can be in-valuable in supporting the delivery of quality software BUT only if used strategically.
It seems that even if not used strategically there are some quick wins to be made from test automation as follows:
- Time and cost savings through improved efficiency,
- Repeatability and accuracy of execution,
- Improved test coverage through automated regression test execution,
- Better visibility and control of the test process,
However, it’s when test automation is used correctly and with a holistic approach (holistic testing!) then the gains are long-term and tend to multiply. This is what Ken was proposing when he presented the view that organisations should approach their automated testing from the full software development lifecycle rather than just restricting automation testing to a phase during system testing.
Now, let’s examine this holistic approach as follows:
Costs – ‘no pain no gain’
Are there more costs involved in approaching our automated testing from the full Dev. Life cycle?
Yes, of course there are costs and the old adage of ‘no pain no gain’ comes to mind here due to some initial outlays.
- There are costs associated with purchasing a test tool and accompanying licenses.
- There are costs associated with training your chosen employee / employees (be aware of up-skilling just one employee because if they leave they take all that valuable knowledge with them)
- Test maintenance is costly if the application is constantly changing.
- There are probably some hidden gotchya’s along the way but of course the longer term gains do offset these initial costs.
Benefits – ‘a lot of gain from some initial pain’
There are some obvious benefits across the organisation in terms of reliability, repeatability, reusability and of course they are faster ‘faster speed is often of the essence’. There are also some more specific benefits as follows:
- At the Requirements Gathering Phase Test Management tools like ‘HP’s Test Director’ can be used to track requirements and assists both development and testing in their ability to uncover in-consistencies aka bugs at this early stage of the project lifecycle and provides traceability throughout the project allowing requirements to be linked to automated scripts as the project progresses.
- During the Development Phase the combination of automation testing with nightly builds from development can result in some big wins for both development and test teams.
- There can be an overall cost reduction, helping to offset the initial costs mentioned above, as the number of resources for regression test are reduced.
Available Tools – ‘the right tools for the job’
Choosing the right tools for the job is vitally important. Ken outlined some open-source tools that may help in the areas of Performance/Load Testing (JMETER , the Grinder) Defect Management (BUGZILLA, Bug Tracker) and Security Testing (Bad Boy, John the Ripper). In his presentation Ken made particular reference to Security Testing and that the use of open source tools, although valuable, tend to lag one step behind in overall security terms.We at DSI have benefitted first-hand from using JMETER and the Grinder for Performance Testing and iTracker for Defect Management and then more recently JIRA for tracking purposes.
Overall – ‘To automate or not to automate that is the question?’
Yes, organisations need to automate some of their testing tasks so that they can reduce the need for manual involvement in repetitive tasks. Organisations need to target the right tools and then identify the areas of the organisation where these tools can realise the benefits of that automation. The Overall benefits of test automation are Un-Boo-lievable.