Tag: test strategy

Beyond the Code: Crafting the Perfect Test Automation Strategy

Nishi Grover GargLeave a comment

In the ever-evolving world of software development, test automation is a necessity. But effective test automation isn’t just about writing code to automate tests; it’s about creating a strategy that aligns with your team’s goals, ensures optimal coverage, and evolves with your product. Let’s explore how you can craft the perfect test automation strategy, one that goes beyond the code and truly delivers value.

Why Strategy Matters

Many teams dive headfirst into test automation, often tempted by the promise of speed and efficiency. However, without a clear strategy, automation efforts can quickly spiral into chaos. Flaky tests, redundant coverage, and maintenance nightmares become all too common. 

For example, a startup I worked with automated their entire regression suite without prioritizing critical paths. When their checkout system broke due to an API change, the automated tests failed to catch it because they were too focused on miniscule UI tests and edge cases.

The takeaway? A good strategy ensures that your test automation efforts are focused, reliable, and adaptable to change.

Step 1: Define Your Goals

Every automation strategy should start with a clear understanding of your team’s goals. Are you looking to reduce manual testing time? Improve test coverage? Speed up deployment cycles? Align your automation efforts with these objectives.

A fintech company aimed to release updates weekly without compromising quality. Their automation strategy focused on automating critical workflows like fund transfers and account creation while leaving exploratory testing to manual testers. By aligning their automation with their release goals, they achieved faster, safer deployments.

Step 2: Identify What to Automate

Not everything needs to be automated. Even if feasible, not everything would be Valuable to automate. So, we need to prioritize test cases for automation based on their :

  1. Criticality: Focus on features that are essential to your business. For instance, for an e-commerce platform, the checkout flow and payment gateway are non-negotiable.
  2. Frequency: Automate repetitive tasks, such as regression testing, to save time.
  3. Feasibility: Some tests, like those involving visual validation, might be better suited for manual testing or require specialized tools.

A SaaS team struggled with flaky UI tests for edge cases. By refocusing their automation on API-level tests and critical user journeys, they reduced test execution time and improved reliability.

Read More »

Tips and Tricks to Master API Testing in Postman

Nishi Grover Garg2 Comments

Modern software development relies on successful software testing services sessions. Postman is currently the best tool for challenging requests. It accomplishes your work involving primary APIs in diverse environments. A well-versed developer can undertake test automation and scripts with monitors. Organised and scheduled tests with team collaboration also can be conducted. They are critical for the efficiency, reliability, and overall effectiveness of this practical tool.

In the context of the increasingly interconnected global landscape, APIs have emerged as a crucial component in contemporary software growth practices. Consequently, API test has emerged as an essential element within the software development life cycle. Postman is widely recognized as a prominent means for API testing, owing to its considerable popularity and apparent advantages. This robust tool empowers developers to efficiently develop, and verify the (APIs). 

By offering a straightforward interface as well as a comprehensive set of functionality, Postman makes it possible for developers and API testers to streamline the procedure of testing (APIs). It does not matter whether you have years of experience as a software engineer or are just starting in the field of API testing; it is necessary to have a basic grasp of Postman to make full use of its capabilities.

In this article, we will thoroughly examine the software tool known as Postman and delve into various strategies and techniques which could successfully harness its extensive capabilities.

Ahead we will augment your skill sets as an experienced developer. If you are new and learning to adapt, it will encourage you to explore a wide variety of features of this tool. Eventually, it will not just be the proverbial ‘Postman’ exchanging and delivering requests. It will become the ideal go-to API testing solution as you become a Postman guru. 

Disclaimer:
We presume you have already installed Postman, the amazing tool for Application Programming Interface aka API testing. The following blog will only focus on the tool’s credibility for skilled developers. Using various features you can share requests, and create and test documents/files. The tool accepts requests to save complexities related to HTTP and read responses coming from the client-server.

POSTMAN Syntax

GET retrieves data from an API.

POST sends new data to an API.

PATCH and PUT update existing data.

DELETE removes existing data.

5 tips that make Postman beneficial 

  1. Collating all tests with collections.
  2. Managing test data under different environments.
  3. Automation with scripts. Use the API login token for all requests.
  4. Monitors for scheduled test runs. 
  5. Sharing results with the team and taking quick decisions on requests.  
Read More »

What is Smoke Testing: A Complete Guide

Nishi Grover GargLeave a comment

While the software development process is going on. It is necessary to test various functions of the software repeatedly. Repeated testing ensures that the software is working in an intended manner and bugs (if present) are eliminated.

Many software development companies use smoke testing to find if the project is ready for end users’ use or if are there any more necessary changes in the software.

When in-depth testing is done, the chances of solving even the smallest bugs increase. In this article, we will understand what smoke testing is, along with its benefits, risks, and when to use this technique. So, let’s dive into this together.

What is Smoke Testing?

Smoke test, aka, confidence testing, is a process to know the functionality, stability, integration ability, and efficiency of the project under development.

One can also call this process verification testing as the test checkers for errors with the software build. This technique generally involves multiple tests that testers can conduct manually or use automated test methods.

Testing different software components can enable the testers to discover breaches in the code issues without using more valuable assets. Smoke testing also helps the QA team to determine whether the software is ready to release or might require additional development and testing work.

Why is Smoke testing required?

Smoke testing is a process that is performed in the phase of software when it is released to the QA and testing team. Consider a situation where an eCommerce app is deployed in the testing phase, but when the tests are going on, it is found the user is not allowed to log into the app to proceed and perform different tasks; or one can say- the user is blocked from using the app because he/she is not able to perform basic login activities.

Then there is no meaning in performing detailed and exhaustive testing of other major modules of the app. First, the development team has to solve the issue where users cannot log in to the app. Such issues are determined and solved first while performing smoke testing.

Here are some prime reasons why smoke testing is used while developing software or an app:

  • Smoke testing finds major issues in the app in the beginning only
  • It helps in reducing the efforts and time of the QA team that could be wasted if the project is not stable
  • With smoke testing one can make early identification of the defects
  • It increases the stability of the software or app developed

Why is this technique called ‘Smoke’ testing?

Read More »

My experience speaking at World Test Engineering Summit (in-person, Bangalore)

Nishi Grover GargLeave a comment

It has been a while since I have made it to an in-person event. So, this one was extra special.

Though, a very busy week (or perhaps month 😛 ) left little time to prepare for my talk, I was excited for the day and to present in person, network and meet people at the conference. And the day matched my expectations!

It was really fun presenting to such an engaged crowd and I was amazed at the response and feedback I received during and after the session! There were some great discussions, very keen participants sharing their own stories & experiences and many fantastic comments by the delegates.

My Topic was – “Keep the Wheels of Continuous Testing in Motion”

Here are the details of the event and my session

The event was also a great platform to meet people and teams from esteemed organisations. I spent time chatting with very talented people and made some great connections.

I also took this opportunity to dust off my Sketch notes pad 🙂 I made sketchnotes for some interesting sessions of the day.

Test Automation in Digital World, Vadeesh Budramane, Algoshack
Accelarate DevOps for Data Science / BI , Sripad Hebbar, Infoblox
DevOps Test Strategy using Browser Stack, Narendra Babu

I thank 1point21GWS team for having me as a part of this esteemed speaker panel.

I am always grateful for opportunities like this and always look forward to more such days!

Happy Testing!

Nishi

How to create a document outlining your Test Strategy?

Nishi Grover Garg1 Comment

With the software revolution, all excellent project managers know that if you hasten into the testing process, that particular project is considered to face a lot of issues. 

Working with the best test management software and its particular strategy planned before testing starts is essential because software becomes more complicated with more sequences of code to evaluate. 

Without a definite outline to develop, the QA team might not be sure what their duties are, what type of testing should be performed first and how the project’s progress is being determined.

The initial action to developing a test plan is to understand what you are attempting to perform with the testing strategy document as it will change software testing strategies. 

According to this post created by TatvaSoft experts, the purpose of a test plan is to describe the complete scope of the QA method from both a comprehensive and granular standard. 

Let’s take a look at one of their sections and understand why they consider that having a Test Plan for every business is beneficial— 

A Test Plan is a thorough document that outlines the test strategy, objectives, timeline, estimation, deliverables, and resources needed to execute software testing. The Test Plan assists us in determining the amount of work required to confirm the quality of the application being tested. The test plan is a blueprint for conducting software testing operations as a well-defined procedure that is meticulously documented. Read More 

The software testing approach helps to create a test plan as it is one of the most essential aspects of the complete testing process, as it describes accurately what the testing team requires to succeed and how to proceed to achieve those aims. Test Strategy is a crucial part of the Test Plan.

The task of writing a whole document test strategy from start may seem daunting, especially for bigger projects. If QA managers need a systematic approach to this responsibility, however, they’ll discover that planning and compiling an efficient and comprehensive test strategy isn’t so hard.

Here are a few tips for your team’s benefit-

Steps To Create A Test Plan Using Best Test Strategies!

Understand the product or feature you’re testing

We will begin with one of the essential aspects of software testing strategy, which is having a broad knowledge of the product or feature before you begin developing a test plan for it. 

For instance, let’s assume you’ve just worked over a website redesign and need to test it before you launch it in the market. For that, you will require to know these below-mentioned factors: 

  • Discuss with the designer and developer to know the extent, goals, and functionality of the website.
  • Review the project documentation which is developed by the project manager that includes SOW, project plan, and the responsibilities in the project management tool.
  • Conduct a product examination to know the functionality, user movement, and defects.

This action provides you with the setting to write a good test strategy document, purposes and begin to plan out the sources you’ll require to create it.

Define the test objectives and their criteria

As you describe every other test you’re continuing to work on, you are required to apprehend when your test is “finished.” 

This implies establishing the pass and fail standards for each particular test, and some of the information we can discuss, such as departure and delay criteria.

To perform this, you’ll need to recognise different system metrics that you’re monitoring and select what progress means for them. 

For instance, if you were working on a performance test you need these measures to succeed:

  • Response time: Complete time to forward a request and receive a response.
  • Average load time: Average time it needs to address all requests.
  • Peak response time: The most extended time it needs to complete a request.
  • Wait time: It is the duration it needs to get the first byte after a demand is sent.
  • Requests per second: The number of requests it can manage during that one second.
  • Events passed/failed: The whole number of passed or failed requests.

Don’t worry you can proceed with testing and repeating forever. So you are required to determine what’s best for your software out and in the palms of users.

Plan the test environment

The outcomes of your test plan always depend on the point you’re testing as well as the environment you’re testing it in. 

As a component of the test strategy document scope, you are required to decide what hardware, software, operating system, and tool compounds you’re running to test.

This is a condition where it pays to be particular. For instance, if you’re going to define an operating system to be utilised throughout the test plan, specify the OS edition/version rather than just mentioning the name.

Define Test Objective

Test Objective is the complete purpose and performance of the test execution. The purpose of the testing is to discover as many software errors as possible; guarantee that the software during the test is bug-free before release.

To determine the test purposes, you need to perform these 2 steps:

  • Record all the software features such as functionality, appearance, and GUI that may require testing.
  • Determine the purpose or the object of the test according to features

Use these steps to get the test goal of your testing project

Also, you can use the ‘TOP-DOWN’ approach to discover the website’s features that may require testing. In this process, you break down the app following the test to component and sub-component.

Conclusion

The test strategy document provides a definite concept of what the test team will perform for the entire project. It is a latent document that means it won’t fluctuate during the project life cycle. 

The one who prepares this document must have enough expertise in the product field, as this is the record that is running to manage the complete team, and it won’t fluctuate during the project. 

Test strategy documents need to be distributed to the complete testing team before the testing ventures start.

<This is a guest post by Matthew Jones – who is a tech enthusiast. and he likes to share his bylines.>

How to Decide if You Should Automate a Test Case

Nishi Grover Garg1 Comment

Test automation is imperative for the fast-paced agile projects of today. Testers need to continuously plan, design and execute automated tests to ensure the quality of the software. But the most important task is to decide what to automate first.

In my article published on the TestRail Quality Hub, I have compiled & shared a list of questions to help you prioritise what you should automate next and guide your test automation strategy.

Here is a checklist of questions to ask yourself as you decide on automating a Test Case–

*******

Is the test going to be repeated?

Is it a high-priority feature? 

Do you need to run the test with multiple datasets or paths? 

Is it a Regression or Smoke Test?

Does this automation lie within the feasibility of your chosen test automation tool?

Is the area of your app that this is testing prone to change?

Is it a Random Negative Test?

Can these tests be executed in parallel, or only in sequential order?

Are you doing it only for the reports?

********

For the detailed explanation of each of these points, read the complete article here –>

Are your Test cases really effective?

Nishi Grover GargLeave a comment

Test teams are forever designing and adding new tests, running them, and reporting results. But is your test team creating tests that are effective at finding real problems?

How do you know if your tests are actually working, and not just adding to the ever-increasing test count?

In my article published on the Testrail blog site, I discuss some ways you can gauge the effectiveness of your tests — and improve them.

Defects Found

The top and most obvious indicator of the effectiveness of your test cases is the defects you find when executing them. As you and your team execute the designed test cases, constantly ask yourself these questions:

  • Are these tests guiding me toward defects?
  • Am I finding problems with the predefined test cases? Or do I have to do more exploration before even getting close to a problem?
  • Are these tests exercising unique flows or use paths?

Metrics

You can also look at your defect lists and find related test cases for the defect logged (if you have that ability in your defect management system). This interlinking helps the team understand what test cases led to the issues found.

You can then further analyze whether that test case was created during test design or later added to the list when the issue was found.

Exploration

If your test cases are not effective, you will not find any useful bugs in test execution. That will mean most of your time is spent in unplanned exploration or ad hoc testing. So, by looking at the time spent in actual test execution versus the time spent on ad hoc testing, you can figure out the effectiveness of the test cases you designed.

If your test cases are effective, you will find issues, explore more use paths, navigate through different integrations with other features, and test different aspects of the same functionality.

If at the end of your test execution, you feel that you have not done all of that, you can infer that is because your test cases might be too simplistic or obvious, and therefore not effective enough to find any useful bugs.

History

Continue Reading here–>

Achieving the Goal of In-Sprint Test Automation

Nishi Grover GargLeave a comment

Getting test automation done is a challenge, especially within the tight deadlines imposed by Scrum. As much as the thought of continuous in-sprint test automation sounds enticing, the practicality of it may elude most Scrum teams.

In my article published here– I look at some of the main things you need to consider in order to get your test automation done within the confines of your sprint.

Framework

The first thing to focus on is a framework that is useful, is easy to understand, and helps all stakeholders participate in test automation.

This is essential because you want to make test automation a continuous activity that is a part of daily work, not a once-a-sprint (or once-a-release) work item. For this to happen, the framework must make it equally comfortable for a businessperson, developer, functional tester or automation expert to add their contribution and see the results of their efforts.

There are many business-friendly frameworks and techniques, like behavior-driven development (BDD), as well as many tools that can create tests in a domain language and then translate them to script code.

All stakeholders must be trained on using the framework, and their area of contribution must be made clear to them, with practical hand-holding. The automation tester can then focus on maintaining the framework, generating test suites and editing failing scripts, while the creation of test automation will be a continuous task assigned to everyone involved.

Collaboration

The next thing to focus on is collaboration between the various stakeholders. A continuous automation framework can only survive when it is being fed and tended to by everyone on the team.

  • The business people, like a business analyst or a product owner, can help by adding user scenarios or defining the requirements in a framework-friendly format. This may require them to be trained on the preferred format based on the framework being used
  • The developers can help by creating reusable methods for steps of the script. They can also create and maintain an object repository for all elements they add to the UI while testers use the pseudo names of the elements in the test scripts. This means that the scripts can be created before (and independent of) the application UI, and such scripts won’t need editing when the UI changes, as long as the object repository is kept up to date
  • The testers can help by adding more scenarios, specifying and creating test data, and executing the scripts periodically

Strategy

How to strategize the development of test scripts is crucial to making in-sprint automation a reality. Using API-level automation whenever possible will reduce the time and effort.

Continue Reading –>

Read More »

How to Decide if You Should Automate a Test Case

Nishi Grover GargLeave a comment

Test automation is imperative for the fast-paced agile projects of today. Testers need to continuously plan, design and execute automated tests to ensure the quality of the software. But the most important task is to decide what to automate first. 

In my article published at the Gurock Blog website, I have have compiled a list of questions to help you prioritize what you should automate next and guide your test automation strategy.

Think of this like a checklist that helps you make automation decisions quickly and effectively and create a standard process around them for your team to follow. Here are the list of questions to ask yourself.

Do check out the complete article for a detailed discussion on each of these-

Is the test going to be repeated?

Is it a high-priority feature?

Do you need to run the test with multiple datasets or paths? 

Is it a Regression or Smoke Test?

Does this automation lie within the feasibility of your chosen test automation tool?

Is the area of your app that this is testing prone to change?

Is it a Random Negative Test?

Can these tests be executed in parallel, or only in sequential order?

Are you doing it only for the reports?

Test automation tools will provide you with useful insights into the quality of the software that you can showcase with the use of some insightful reports. But are these reports the only reason you are looking at automation? Just looking at the red or green status results of the test reports might not be the best way to assess the software quality. You will need to spend time analyzing the tests that failed, why they failed, and what needs to be corrected. Tests created once will need maintenance and continuous monitoring to keep them up to date. All of that needs to be kept in mind and the effort needs to be accounted for.

There is more to test automation than just the fancy reports!

Looking at the questions above, analyse the state of your test case, the intent behind its automation, and its feasibility, as well as the value that you might get out of it. Hope that helps you decide what tests you should or should not be picking for automation!

<Image credits – https://unsplash.com/photos/FlPc9_VocJ4 >

Read Along- ‘Agile Testing’ Chapter-19

Nishi Grover Garg1 Comment

“Wrap Up the Iteration”

  • Agile team delivers working software at the end of the iteration – demonstrate to the customers and get their feedback.
  • Having testers conduct the ’Iteration Review’ is a common practice as they’ve usually worked on all the stories. The Scrum Master, programmers or testers could demonstrate the new features – It is recommended to rotate this honor.
  • Retrospectives are an excellent place to start identifying what and how you can do better.
    • Start, Stop, Continue technique – Discussing What went well, What did not go well and what we can start doing to help.
    • Write task cards for actions to be undertaken to implement the steps
    • At the end of the next iteration, take a checkpoint to see if you improved

Retrospectives are a simple and highly effective way for teams to identify & address issues. The retrospective meeting is a perfect opportunity to raise testing-related issues. Bring up issues in an objective, non-blaming way.

Celebrate Successes

Make sure your team takes at least a little time to pat itself on the back and recognise its achievements.

Even Small Successes deserve a Reward.

Many agile teams have trouble taking time to celebrate success.

Have a weekly fun gathering or team games.

  • For big milestones such a big release or achieving a test coverage goal, the whole company can have a party to celebrate, bringing in catered food or go out.
  • It is also important to celebrate individual successes. A ‘Shout-Out Shoebox’ – is a great idea to recognize the value different team members contribute.
  • Taking time to celebrate successes lets your team take a step back, get a fresh perspective, and renew its energy so it can keep improving your product, giving team members a chance to appreciate each other’s contributions. Don’t fall into a routine where everyone has their head down working all the time!
  • Take advantage of the opportunity after each iteration to identify testing- related obstacles, and think of ways to overcome them.