Are your Test cases really effective?

Test teams are forever designing and adding new tests, running them, and reporting results. But is your test team creating tests that are effective at finding real problems?

How do you know if your tests are actually working, and not just adding to the ever-increasing test count?

In my article published on the Testrail blog site, I discuss some ways you can gauge the effectiveness of your tests — and improve them.

Defects Found

The top and most obvious indicator of the effectiveness of your test cases is the defects you find when executing them. As you and your team execute the designed test cases, constantly ask yourself these questions:

  • Are these tests guiding me toward defects?
  • Am I finding problems with the predefined test cases? Or do I have to do more exploration before even getting close to a problem?
  • Are these tests exercising unique flows or use paths?

Metrics

You can also look at your defect lists and find related test cases for the defect logged (if you have that ability in your defect management system). This interlinking helps the team understand what test cases led to the issues found.

You can then further analyze whether that test case was created during test design or later added to the list when the issue was found.

Exploration

If your test cases are not effective, you will not find any useful bugs in test execution. That will mean most of your time is spent in unplanned exploration or ad hoc testing. So, by looking at the time spent in actual test execution versus the time spent on ad hoc testing, you can figure out the effectiveness of the test cases you designed.

If your test cases are effective, you will find issues, explore more use paths, navigate through different integrations with other features, and test different aspects of the same functionality.

If at the end of your test execution, you feel that you have not done all of that, you can infer that is because your test cases might be too simplistic or obvious, and therefore not effective enough to find any useful bugs.

History

Continue Reading here–>

Ways to Generate Quick Test Ideas

As testers, we look at everything with a critical eye. As soon as something comes up for testing, our instinct is to get down to examining it and looking for problem areas. After getting a written test script, a new tester would be tempted to begin executing scripted tests right away.

But stopping to gather our thoughts about possible test ideas first is a smarter approach that leads to better, more unbiased test coverage. However, we don’t always have a lot of time to imagine scenarios and different paths. Luckily, there is always some planning we can do beforehand.

In my article published at Gurock Testrail blog I shared some tips for generating test ideas in a time crunch.

Revisit classic test techniques

Our old, trusted test design techniques like boundary value analysis, equivalence class partitions, decision tables, and state flow diagrams are always a help when thinking about test cases. Although most of them are ingrained in the thought process of a tester and are mostly common sense, giving them a revisit, however informal, may still give us some more test ideas.

For example, creating a quick decision table for the interaction of two or more variables to observe the behavior of the system may reveal some unique combination that we might have missed. Or a quick boundary value analysis for the age field in our we form may show us a special case we might have missed.

Similarly, using state transition diagrams to draw end-to-end flows can help not only the testers, but also the developers in imagining the overall system flow and revealing problem areas.

Look at the history

The history of the project or the system can give us many insights into what we are dealing with, where the common defect clusters are, and the most problematic components.

If you are new to the test team, start by having a look at the defect tracking from past sprints or releases. You can then define and think of more test cases based on past defects and the components that have had the greatest number of defects.

If you’ve been part of the team for a while, you are probably intuitively bound to focus on these areas. But even then, it will help to consciously make an effort to list the most common types of bugs encountered and most problematic areas based on your experience. This will help not only you, but also your new and junior team members. Read full post->

Read More »

‘Just Enough’ documentation in an Agile Project

Agile poses many challenges to the development team, most of them pertaining to time. Teams are perpetually under pressure to deliver working software at a fast pace, leaving minimum time for anything else. When testing on an agile project, learning how to write lean documentation can save precious time. Furthermore writing lean documentation can help rework efforts by focusing only on what’s really necessary.

The Agile Manifesto emphasizes working software over comprehensive documentation, but most agile teams interpret this wrong and treat documentation as something to be avoided, owing to time constraints. The manifesto states a lesser focus on comprehensive documentation, but some documentation is still needed for the project and any related guidelines being followed. Attaining this balance is a challenge.

Documentation is a necessary evil. We may think of it as cumbersome and time-consuming, but the project cannot survive without it. For this reason, we need to find ways to do just enough documentation — no more, no less.

Read about how to focus on important areas like VALUE  , COMMUNICATION and  SUFFICIENCY when documenting in your agile project – in my article published at Gurock TestRail blog –> https://blog.gurock.com/lean-documentation-agile-project/

just enough

Click here to read the full article

For example, in a traditional test design document, we create columns for test case description, test steps, test data, expected results and actual results, along with preconditions and post-conditions for each test case. There may be a very detailed description of test steps, and varying test data may also be repeatedly documented. While this is needed in many contexts, agile testers may not have the time or the need to specify their tests in this much detail.

As an agile tester, I have worked on teams following a much leaner approach to sprint-level tests. We document the tests as high-level scenarios, with a one line description of the test and a column for details like any specific test data or the expected outcome. When executing these tests, the tester may add relevant information for future regression cycles, as well as document test results and any defects.

More examples and scenarios for learning leaner test document creation are included in the full article– Click here to read the full article

 

                 Are you interested in finding the right tool for your Agile processes? Here is a comprehensive assessment and comparison of the best agile tools available! 

https://thedigitalprojectmanager.com/agile-tools/

Prepared by Ben Aston, this list may be a useful guide for finding and selecting the best tool to support your agile journey. Check it out!

 

Happy Testing!

Nishi

Pesticide Paradox in Software Testing

Pests and Bugs sound alike?? They act alike too!! 

Boris Beizer, in his book Software Testing Techniques (1990) coined the term pesticide paradox to describe the phenomenon that the more you test software, the more immune it becomes to your tests.

Just like, if you keep applying the same pesticide, the insects eventually build up resistance and the pesticide no longer works. Software undergoing the same repetitive tests build resistance to them, and they fail to catch more defects after that.

  • Software undergoing the same repetitive tests eventually builds up resistance to them.
  • As you run your tests multiple times, they stop being effective in catching bugs.
  • Moreover, part of the new defects introduced into the system will not be caught by your existing tests and will be released onto the field.

Solution: Refurnish and Revise Test Materials regularly

In order to overcome the pesticide paradox, testers must regularly develop newer tests exercising the various parts of the system and their inter-connections to find additional defects.

Also, testers cannot forever rely on existing test techniques or methods and must be on the look out to continually improve upon existing methods to make testing more effective.

It is suggested to keep revisiting the test cases regularly and revising them. Though agile teams provide little spare time for such activities, but the testing team is bound to keep planning these exercises within the team in order to keep the best performance coming. A few ideas to achieve this:

  • Brainstorming sessions – to think of more ideas around the same component testing
  • Buddy Reviews – New joinees to the team are encouraged to give their fresh perspective to the existing test scenarios for the product, which might get some new cases added.
  • Strike out older tests on functionalities that are changed / removed
  • Build new tests from scratch if a major change is made in a component – to open a fresh perspective

 

UPDATE–

This article has been recommended and used as a reference by HANNES LINDBLOM in his blog at https://konsultbolag1.se/bloggen/veckans-testartips-15-tur-genom-variation