The bugs we find during
testing can tell us a lot about the application, the state of its quality and its
release-readiness. Bugs can also provide insights into our development
processes and practices — and lapses therein.
How can we study bugs to improve the overall state of our project? In my article published @Gurock TestRail blog, I have described three things to learn from the bugs you find. https://blog.gurock.com/three-learn-bugs/
The location of defect clusters
Defect clustering is one of the seven
principles of software testing, and keeping
an eye out for these clusters is the responsibility of a good tester.
As we log defects into a tracking tool or portal, teams
generally follow the practice of measuring relevant modules, components or
functional areas against each defect. When tracked over time, this information
can be real gold! It helps us track which areas of the application are having
We can plot these area metrics against the number of defects
raised and find the defect rates over time. We can also create filters to raise
concerns whenever the defect rates go over a certain limit in any specific area
or component. This can help us combat defect clustering by doing a fresh
analysis, revisiting the tests being performed and focusing more of our
exploratory test efforts in those areas.
Overall, knowing about these defect clusters, keeping an eye
out for them and regularly revisiting the areas will help us keep the quality
of the entire system in check.
Frequency of defects (and their resolution)
The frequency of defects being found and logged tells us a
lot about the maturity of the product.
In the beginning of construction sprints, defects are
supposed to be frequent and plentiful. We may not go by numbers here, but the
relativity of them. As we progress toward a release, the number of defects
generally declines, indicating that the system is now more mature and sturdier
after withstanding multiple test cycles. Some teams even use the metric of mean
time between failures as an exit criterion for testing, indicating that they
will only finish testing once they cannot find any new defect for a certain
number of days.
As defects are raised, triaged, resolved and verified, there
is a typical turnaround time that we expect. Most defects will go through this
lifecycle within a reasonable stipulated time or will be postponed with a
reason or business decision. Some defects may linger in the system for longer.
There may be a variety of reasons for these decisions:
- A defect requires more information, and the
developer is awaiting confirmation or details from the tester who raised it
- The defect was misunderstood and there are
comments going back and forth between the tester and developer about the
- The assigned developer was on vacation for a
week and the defects have not been fixed, leading to a plateau in the
- Defects are awaiting triage by the product owner
and do not have priorities or the correct people assigned to them
Whatever the reason, knowing the cause of defects remaining
open, in progress or unresolved for longer than a stipulated time is important.
We may have to fix people issues or communication gaps, or may just need to
schedule a short triage or discussion with the team to decide on the fate of
such issues. But understanding any delays gives us a much-needed insight into
team dynamics and helps us smooth out the process.
The reasons behind rejected defects
The number and type of defects
getting rejected — and the reasons behind the rejections — can also tell us a
lot about the state of the product and psychology of the team. If you see a
high number of irreproducible defects, it may mean that some data or
information is getting lost when reporting, or that the testers do not have
enough time or perspective to reproduce the defects.
A high number of duplicate
bugs may show that testers are unaware of the system’s history, or maybe they
are new to the team and need to get a little more background. It may also be a
case of the same bugs reoccurring, which might have been fixed and closed in
Incorrect defects marked with “Not
a bug” or “Working as designed” tell us about a lack of understanding of the
system on the testers’ side. Or it may be due to a lack of communication among
the team members, leading to different perceptions about the features that were
designed or implemented.
Our findings from these types
of defects can help test managers or project owners plan measures like internal
trainings and knowledge sharing, which can enhance communication among team
members and introduce prerequisites to fulfill before logging any issues.
There is a world of information that your defects can provide. If you take a good look at your bugs and talk about them as a team, you can find ways to use that information to your advantage.
Read more–> Click here for the full article