Read Along- ‘Agile Testing’ Chapter-14

“An Agile Test Automation Strategy”

Use the Agile Test Quadrants to help you identify the different types of test automation tools you might need for each project, even each iteration.

Test Automation Pyramid (introduced by Mike Cohn)

Lowest Layer- Bulk of automated unit , technology facing tests. Quickest feedback, code much more quickly using xUnit family of tools

Middle layer – Automated business-facing tests that help the team. “Are we building the right thing” Tests operate at the API, behind the GUI level. Bypass the presentation layer – less expensive to write & maintain these tests. Fit & FitNesse are good examples, written in domain language

Top Tier – Should be the smallest automation effort as they  have the lowest ROI. Done through GUI, operating on the presentation layer. More expensive to write, more brittle and need more maintenance.

The Test Automation Pyramid
  • Any tedious or repetitive task involved in developing software is a candidate for automation.
  • AN automated deployment process is imperative – getting automated build emails listing every change made is a big help to testers. It speeds up testing & reduces errors.
  • A fast running continuous integration and build process gives the greatest ROI of any automation effort.
  • Another useful area for automation is data creation or setup. Cleaning up test data is as important as generating it. You data creation toolkit should include ways to tear down the test data so it doesn’t affect a different test or prevent rerunning the same test.

What we shouldn’t automate

  • Usability testing
  • Exploratory testing
  • Tests that will never fail
  • One-Off tests
  • Plan-in plenty of time for evaluating tools, setting up build processes, and experimenting with different test approaches in the initial iterations.
  • If management is reluctant to give the team time to implement automation, explain the trade-offs clearly. Work towards a compromise.
  • We will always have deadlines, and we always feel pressed for time. There is never enough time to go back and fix things. During your next planning meeting, budget time to make meaningful progress on your automation efforts.
  • Good test management ensures that tests can provide effective documentation of the system and of the development progress

Read Along- ‘Agile Testing’ Chapter-12

“Summary of Testing Quadrants”

This chapter reviews all the four Agile Testing Quadrants by illustrating an example of a team’s success story in testing their whole system using a variety of home-grown and open source tools.

The system is related to Monitoring of Remote Oil and Gas Production Wells. The software application had a huge legacy system, with very few unit tests. The team was slowly rebuilding the application using new technology. And describes how they used tests from all four quadrants to support them.

  • Using Test Driven Development and Pair Programming wholeheartedly. Also adding unit tests and refactoring all legacy code they encountered on the way.
  • The product engineer writing acceptance tests and sharing with the developers and testers before they began creating.
  • Automation involving functional test structure, web services and embedded testing
  • Exploratory testing to supplement the automated tests to find critical issues.
  • Don’t forget to Document… but only what is useful
  • Finding ways to keep customers involved in all types of testing, even if they are remote. Have UATs and end to end tests
  • Use lessons learnt during testing to Critique the product in order to drive the development in next iterations
Agile Testing Quadrants