The Agile Testing Quadrants matrix helps testers ensure that they have considered all of the different types of tests that are needed in order to deliver value.
Unit tests verify functionality of a small subset of the system. Component tests verify the behaviour of a larger part such as a group of classes that provide some services. Unit & Component tests are automated and written in the same programming language as the application. They enable programmers to measure what Kent Beck has called the internal quality of their code.
Tests in Quadrant-2 support the work of the development team but at a higher level. These business-facing tests define external quality and the features that the customers want. They’re written in a way business experts can easily understand using the business domain language.
The quick feedback provided by Quadrant 1 and 2 automated tests, which run with every code change or addition, form the foundation of an agile team. These tests first guide the development of functionality and when automated, then provide a safety net to prevent refactoring and the introduction of new code from causing unexpected results.
“Appraising a software product involves both art and science.”
Quadrant-3 classifies the business-facing tests that exercise the working software to see if it doesn’t quite meet expectations or won’t stand up to the competition. They try to emulate the way a real user would work the application. This is manual testing that only a human can do…use our senses, our brains and our intuition to check whether the development team has delivered the business value required by the customer.
Exploratory testing is central to this quadrant.
Technology-facing tests in Quadrant-4 are intended to critique product characteristics such as performance, robustness and security.
Creating and running these tests might require the use of specialised tools and additional expertise.
Automation is mandatory for some efforts such as load and performance testing.
Ward Cunningham coined the term “technical debt” in 1992, but we’ve certainly experienced it throughout our careers in software development.
By taking the time and applying resources and practices to keep technical debt to a minimum, a team will have time and resources to cover the testing needed to ensure a quality product. Applying agile principles to do a good job of each type of testing at each level will, in turn, minimize technical debt.
Each quadrant in the agile testing matrix plays a role in keeping technical debt to a manageable level.
The Agile Testing Quadrants provide a checklist to make sure you have covered all your testing bases. Examine the answers to questions such as:
Are we using unit & component tests to help find the right design for our application?
Do we have an automated build process?
Do our business-facing tests help us deliver a product that matches customer expectations?
Are we capturing the right examples of desired system behaviour?
Do we show prototypes of the UIs and reports to the users before we start coding them?
Do we budget enough time for exploratory testing?
Do we consider technological requirements like performance and security early enough?
There are many processes in a typical project that don’t transition well to agile because they require heavyweight documentation or are an inherent part of the phased and gated process & require signoffs at the end of each stage.
“Metrics can be controversial”
Measurements such as cycle time that involve the whole team are more likely to drive you toward success than measures confined to isolated roles or groups.
Lean development looks for ways to delight customers, which ought to be the goal for all software development.
Metrics that measure milestones along a journey to achieve team goals are useful.
When you are trying to figure out what to measure, first understand what problem you are trying to solve. If your goals are measurable, the measurements you need to gather to track the metrics will be obvious.
Figure each metrics Return on Investment and decide whether to track or maintain it. Does the effort spent collecting it justify the value it delivers? Can it be easily communicated and understood? Do what works for your situation. Experiment with keeping a particular metric for a few sprints and evaluate whether it is paying off.
“Projects succeed when people are allowed to do their best work”
Defects tracking systems (DTS) are too often used as communication tools & entering unnecessary bugs can be wasteful. Focus on using DTS for the right reasons.
Whether your team decides to create a test plan or not, the planning should be done. Each project is different, so don’t expect that the same solution will fit all.
Regarding Audits, Processes & Models
Traditional quality processes & process improvement models like SAS 70 and CMMI standards can co-exist with agile.
Quality assurance teams in traditional development organisations are often tasked with providing information for auditors and ensuring compliance with audit requirements.
Examples include what testing has been performed on given software release or proving that different accounts reconcile.
Testers can be tasked with writing test plans to evaluate the effectiveness of control activities.
Work together with the compliance and internal audit teams to understand your team’s responsibilities.
If your organisation is using some kind of process model or quality standard, educate yourself about it and work with the appropriate specialists in your organisation.
Process improvement models and frameworks emphasize discipline and conformance to process.
“Standards simply enable you to measure your progress towards your goal”
Working with existing quality processes and models is one of the biggest cultural issues you may face as you transition to agile development. All of these changes are hard, but when your whole team gets involves, none are insurmountable.
Testers on agile teams feel very strongly about their role as customer advocate and also feel they can influence the rest of the team with their quality thinking.
Testers also need training on pair testing, working with incomplete and changing requirements, automation and all of the other new skills that are required.
Programmers also might need coaching to understand the importance of business-facing tests and the whole-team approach to writing and automating tests.
The pairing of programmers and testers can only improve communication about the quality of the product.
“One major advantage of an integrated project team is that there’s only one budget and one schedule.”
The difference between a traditional cross-functional team and an agile team is the approach to the whole-team effort. Members are not just “representing” their functions in the team but are becoming true members of the team for as long as the project or permanent team exists.
The best teams are those that have learned to work together and have developed trust with one another.
Teams work better when they have ready access to all team members, easy visibility of all progress charts and an environment that fosters communication.
Teams make the best progress when they’re empowered to identify and solve their own problems. If you’re a manager, resist the temptation to impose all your good ideas on the team.
Make sure testers are involved in all meetings! If you are a tester and someone forgets to invite you to a meeting, invite yourself!
“Courage is especially important. Get up and go talk to people; ask how you can help. Reach out to team members and other teams for direct communication. Notice impediments ad ask the team to help remove them.”
Agile development works because it gets obstacles out of our path and lets us do our best work.
This chapter is dedicated to talking about organisational problems with agile adoption, mostly from a cultural point of view- how people perceive changes, how they work, giving up control and also taking charge. It is a very comprehensive description of many problems we see on a daily basis at our work and in teams struggling with agile transformation.
Points to remember and Quotable Quotes
Agile teams are best suites for organisations that allow independent thinking.
Fear is a powerful emotion, and if not addressed, it can jeopardize the transition into agile
Testers who don’t change their approach to testing have a hard time working closely with the rest of the development team
If the organisation culture is to push towards release without caring for quality, the teams will face an uphill battle in working in agile
Companies where testers assume the role of ‘Quality Police’ will also have a challenge since teams will not buy-into the idea of building quality in, as they are accustomed to badgering it in later.
If your organisation focuses on learning, it will encourage continuous process improvement and will likely adopt agile much more quickly.
Testers need time and training, like everyone else learning to work in agile
To help testers adjust, you may need to bring in an experienced agile testing coach to act as a mentor and a teacher.
Agile focuses on working at a sustainable pace, all the time. In contrast to the ‘fast and furious’ testing done at the end of release cycles in traditional projects (often amounting to overtime). In agile, if overtime is required, it is an exception, and that too for the whole team and not just the testers.
In agile, the relationship between the customer and the development team is more a partnership than a vendor-supplier relationship.
Even if an entire company adopts agile, some teams make the transition more successfully than others.
About Introducing Change-
“Expect and Accept Chaos as you implement Agile Processes.”
Find the areas of most pain, determine what practices will solve the problem so that you can get some immediate progress out of the chaos.
The critical success factor is whether the team takes ownership and has the ability to customise its approach
Celebrate success- Acknowledgement is important if you want a change to stick.
Rather than managing the team’s activities at a low level, managers of agile teams focus on removing obstacles so that team members can do their best work
“Agile development might seem fast-paced, but the change can seem glacial”
“Beware of the Quality Police mentality— Be a collaborator, not an enforcer“
The highlight of this chapter for me was reading the ‘Testers Bill of Rights’
I had not heard about this before , so reading this was pretty cool, and for sure fundamental to any tester’s life. Check it out-
***Update **About face-to-face communication** during Covid-19 ***
As I am reading this book during this bizarre time of social-distancing, working remotely and entire nations on lockdown, the part about ‘face-to-face’ communication has a new meaning now. As Janet Gregory also pointed out in response to this article, our definition of face-to-face has changed over the last few weeks over the entire world! We are lucky to have technology that helps us continue effective communication within our teams, have conversations, video calls, screen shares, continue learning over webinars and continue working, feeling useful and being productive.
Hoping things change soon and we can go back to having fun, productive discussions with our team mates over coffee. Until then — Happy social distancing!
“Several core practices used by agile teams relate to testing.”
Programmers using TDD, code integration tests being written contribute to a successful project.
“Agile Testing just doesn’t mean testing on an agile project.”
Some testing approaches like exploratory testing are inherently agile, whether done in an agile project or not.
Testers are integral members of the customer team as well as development team
The best part of this chapter is Lisa and Janet’s wonderful stories on beginning with their first agile projects, and a realization by Janet’s co-worker, a developer in a team following XP on how they saw Janet’s contribution to the project.
“Testers don’t sit & wait for work; they get up and look for ways to contribute throughout the development cycle and beyond.”
Traditional vs Agile Testing
In Traditional approach – “Testing gets “squished” because coding takes longer than expected, and because teams get into a code-and-fix cycle at the end.
In Agile – “As an agile team member, you will need to be adaptive to the team’s needs
“Participants, tests, and tools need to be adaptive.”
“An Agile team is a wonderful place to be a tester”
The Whole-Team Approach –
“Everyone on an agile team gets “test-infected.”
“An agile team must possess all the skills needed to produce quality code that delivers the features required by the organization.”
“The whole team approach involves constant collaboration”
“On an agile team, anyone can ask for and receive help”
“The fact is, it’s all about quality – and if it’s not, we question whether it’s really an ‘agile’ team.”
I used to love books, reading was a fun and satisfying hobby for the introverted teen I was. But lately I may have gotten away from it for known and unknown reasons. I want to pursue the passion again and hold myself accountable too. So, this year I am starting a ‘Read Along’ series on my blog.
I have learnt agile testing by doing it, learning it hands-on, training & running courses on agile testing for professionals. I wanted to enhance my knowledge by reading the professional work by these awesome ladies.
So, I will be reading the book and will post about learnings, things to remember & quotable quotes from each chapter as I progress. This is to hold myself accountable, as well as to help people looking for good reads or learnings. Hope this helps you. Have you read this book? Do share your thoughts & learnings too!