Tag: test automation

I am speaking at DevOpsCon Singapore – Nov 2021

Nishi Grover GargLeave a comment

I am super excited to announce that the DevOpsCon Singapore conference is finally here and I am elated to be a part of the Speaker lineup!

After cancellations and rescheduling in 2020 due to the pandemic, this grand event is finally happening now from 22 to 25 Nov 2021 in an online edition.

My talk that was slated to be part of the 2020 edition remains the same topic which I have worked to enhance this year. Here are a few details of the session –

Title – The What, When and How of Test Automation

Description

Agile means pace and agile means change. With frequent time-boxed releases and flexible requirements, test automation faces numerous challenges. Haven’t we all asked what to automate and how to go about the daily tasks with the automation cloud looming over our heads? Here, we’ll discuss answers to some of these questions and try to outline a number of approaches that agile teams can take in their selection of what to automate, how to go about their automation, whom to involve, and when to schedule these tasks so that the releases are debt-free and of the best quality. 

  • What to automate: regression averse approach, selective approach, sanity automation, max automation approach
  • When to automate: sprint n-1 approach, continuous automation
  • How to automate – all-hands approach, shared automation expert, code-averse tool 

Let us have a look at the integration of these possibilities, the possible combinations, and what may or may not work.

My session will be held live on Monday, November 22 2021 17:00 – 17:45 SGT

To check out my speaker and session details, click here

For the detailed program and agenda of the conference, see this page

To Register for the event and related details, click here

Hope to see you there! 🙂

Advertisement

How to Decide if You Should Automate a Test Case

Nishi Grover GargLeave a comment

Test automation is imperative for the fast-paced agile projects of today. Testers need to continuously plan, design and execute automated tests to ensure the quality of the software. But the most important task is to decide what to automate first.

In my article published on the TestRail Quality Hub, I have compiled & shared a list of questions to help you prioritise what you should automate next and guide your test automation strategy.

Here is a checklist of questions to ask yourself as you decide on automating a Test Case–

*******

Is the test going to be repeated?

Is it a high-priority feature? 

Do you need to run the test with multiple datasets or paths? 

Is it a Regression or Smoke Test?

Does this automation lie within the feasibility of your chosen test automation tool?

Is the area of your app that this is testing prone to change?

Is it a Random Negative Test?

Can these tests be executed in parallel, or only in sequential order?

Are you doing it only for the reports?

********

For the detailed explanation of each of these points, read the complete article here –>

Achieving the Goal of In-Sprint Test Automation

Nishi Grover GargLeave a comment

Getting test automation done is a challenge, especially within the tight deadlines imposed by Scrum. As much as the thought of continuous in-sprint test automation sounds enticing, the practicality of it may elude most Scrum teams.

In my article published here– I look at some of the main things you need to consider in order to get your test automation done within the confines of your sprint.

Framework

The first thing to focus on is a framework that is useful, is easy to understand, and helps all stakeholders participate in test automation.

This is essential because you want to make test automation a continuous activity that is a part of daily work, not a once-a-sprint (or once-a-release) work item. For this to happen, the framework must make it equally comfortable for a businessperson, developer, functional tester or automation expert to add their contribution and see the results of their efforts.

There are many business-friendly frameworks and techniques, like behavior-driven development (BDD), as well as many tools that can create tests in a domain language and then translate them to script code.

All stakeholders must be trained on using the framework, and their area of contribution must be made clear to them, with practical hand-holding. The automation tester can then focus on maintaining the framework, generating test suites and editing failing scripts, while the creation of test automation will be a continuous task assigned to everyone involved.

Collaboration

The next thing to focus on is collaboration between the various stakeholders. A continuous automation framework can only survive when it is being fed and tended to by everyone on the team.

  • The business people, like a business analyst or a product owner, can help by adding user scenarios or defining the requirements in a framework-friendly format. This may require them to be trained on the preferred format based on the framework being used
  • The developers can help by creating reusable methods for steps of the script. They can also create and maintain an object repository for all elements they add to the UI while testers use the pseudo names of the elements in the test scripts. This means that the scripts can be created before (and independent of) the application UI, and such scripts won’t need editing when the UI changes, as long as the object repository is kept up to date
  • The testers can help by adding more scenarios, specifying and creating test data, and executing the scripts periodically

Strategy

How to strategize the development of test scripts is crucial to making in-sprint automation a reality. Using API-level automation whenever possible will reduce the time and effort.

Continue Reading –>

Read More »

How to Decide if You Should Automate a Test Case

Nishi Grover GargLeave a comment

Test automation is imperative for the fast-paced agile projects of today. Testers need to continuously plan, design and execute automated tests to ensure the quality of the software. But the most important task is to decide what to automate first. 

In my article published at the Gurock Blog website, I have have compiled a list of questions to help you prioritize what you should automate next and guide your test automation strategy.

Think of this like a checklist that helps you make automation decisions quickly and effectively and create a standard process around them for your team to follow. Here are the list of questions to ask yourself.

Do check out the complete article for a detailed discussion on each of these-

Is the test going to be repeated?

Is it a high-priority feature?

Do you need to run the test with multiple datasets or paths? 

Is it a Regression or Smoke Test?

Does this automation lie within the feasibility of your chosen test automation tool?

Is the area of your app that this is testing prone to change?

Is it a Random Negative Test?

Can these tests be executed in parallel, or only in sequential order?

Are you doing it only for the reports?

Test automation tools will provide you with useful insights into the quality of the software that you can showcase with the use of some insightful reports. But are these reports the only reason you are looking at automation? Just looking at the red or green status results of the test reports might not be the best way to assess the software quality. You will need to spend time analyzing the tests that failed, why they failed, and what needs to be corrected. Tests created once will need maintenance and continuous monitoring to keep them up to date. All of that needs to be kept in mind and the effort needs to be accounted for.

There is more to test automation than just the fancy reports!

Looking at the questions above, analyse the state of your test case, the intent behind its automation, and its feasibility, as well as the value that you might get out of it. Hope that helps you decide what tests you should or should not be picking for automation!

<Image credits – https://unsplash.com/photos/FlPc9_VocJ4 >

Introduction to ERP Testing and its Importance

Nishi Grover Garg2 Comments

<This is a guest post by Sohaib Zaidi>

In this era of cut-throat competition, global enterprises are facing tremendous pressure to enhance efficiency, reduce costs, increase sales and profitability. For this, more and more enterprises are embracing ERP (Enterprise Resource Planning) software. Apart from enabling enterprises to make accurate, informed and strategic decisions, ERP also helps them to stay compliant. Though the benefits of introducing cloud ERP solutions to your business are countless, yet these benefits come with several challenges.

Enterprises that have already embraced cloud are struggling to keep pace with the frequency of ERP software updates. ERP vendors like Oracle, SAP, Microsoft, Salesforce, etc are rolling out new releases and patches on monthly, quarterly, or biannually. Since cloud updates are rolled out at quick succession, enterprises are finding it hard to quickly test the updates and deploy these to production. Here arises the need of ERP testing. 

Why is it necessary to test ERP updates before deployment to production?

ERP updates bring new features and functionality, customer enhancement requests, and patches from previous releases. These updates need to be tested regressively since there are chances that they can impact a variety of functions that may cause disruption to business continuity. So, it is always recommended that before rolling out the ERP updates, QA teams should test critical business processes, validate reports, key workflows and test critical integrations with other applications.

Though manual application testing approach is still prevalent but it cannot be considered as a reliable solution in case of ERP testing. To understand this better, let us discuss an example of Oracle ERP. Oracle rolls out quarterly updates. These updates are first introduced to non-production environments. Oracle offers two weeks’ time to test these updates and raise issues. After two weeks, these get applied to the production environment. So, performing Oracle testing manually for these updates is non-feasible. Apart from time-consuming, manual testing is error prone, fragile and costly. Another disadvantage associated with manual testing is that it can adversely impact business continuity due to limited test coverage and its inability to identify change impact.

Embrace Automation Testing for seamless cloud adoption

Test automation not only reduces testing time of complex ERP systems but also ensures robust software quality. The biggest perks associated with test automation are maximum accuracy with minimum efforts, quick feedback, accelerated results, lower costs, and maximum coverage. Most of the test automation tools perform post release impact analysis to identify the impacted areas. Based on the impact assessment, QA teams can generate most relevant tests to execute validation. This not only defines the accurate testing scope but also delivers wider coverage which is not possible while performing manual testing.

When enterprises use test automation for security testing and constant maintenance, they get the opportunity to easily recognize defects. This approach significantly reduces vulnerabilities, helping enterprises to keep huge losses at bay. Automated testing also helps enterprises to overcome challenges of drowsy routine procedures crop up due to manual testing. Leveraging test automation tools, enterprises can accelerate routine procedures that consume time and cost to ensure a quick turnaround and superior ROI.

Author – Sohaib Zaidi – in his own words-
 
I am a technology enthusiast and professional writer with experience across niches like digital transformation, AI, IoT, & test automation. I love to write technology in simple tone so that readers can easily understand how embracing technology can deliver greater outcomes.    

4 Tips to Create a Simplified Test Plan for Your Agile Project

Nishi Grover Garg3 Comments

Planning is an integral part of initiating any project or activity, including software testing. Although formal test plans may seem outdated and unnecessary to most fast-paced agile teams that prioritize cutting down on documentation, it’s still a good idea to keep a pared-down test plan that can guide the testing effort throughout the project.

In my article published on TestRail blog, I talk about four useful tips to create a simplified test plan for your agile project.

1. Include the basics

A test plan can be as detailed as the team requires, but at the minimum, it must contain the context, timelines, and a brief overview of the testing activities expected to be performed in the project, release, or sprint.

Based on what level of test plan you are creating, begin with a basic heading, like schedule and timelines, testing activities and types of testing expected to be performed, people involved in testing, or any new hires or training required for the project or release.

2. Build off the project plan

Most of the test plan can be derived from the context set by the overall project plan, so give that a thorough read and get the details of the project lifecycle, expected delivery schedule, interaction points with other teams, etc.

Note any specifics that emerge, like shared resources with other teams, sync points for integration tests throughout the project’s lifecycle, or special types of testing needed, like performance, load or security testing.

3. Define a clear scope

Let’s say your project is a mobile application that needs to work on both Android and iOS. It would be beneficial to list all the environments and OS versions that need to be supported. Including them in the test plan ensures you set up the test environments early and allocate testing tasks across all of them. It also ensures that you are not bogged down by repeated testing on numerous test environments at the time of release. What is not defined in the initial test plan can automatically be considered out of scope.

Continue Reading ->

4. Use visual tools like mind maps

A simple Agile Test Plan using a Mind map

Read full article here

Continuous Testing in DevOps

Nishi Grover Garg1 Comment

Agile testers need to constantly rethink their processes and tooling in order to move toward faster and more reliable software delivery. The key there is to embrace the continuity. Continuous delivery is necessary for agile development, and that cannot happen without having continuity in testing practices, too.

In my article published on TestRail blog, I discuss the various aspects of Continuous Testing in DevOps-

Continuous testing

Continuous testing can be defined as a methodology focused on continuous quality and improvement. It can use a number of practices and tools to help do that.

Continuous testing encompasses the verification and validation of each piece of the software under development to ensure:

  • Code quality — Are developers creating code of good quality?
  • Application correctness — Are developers creating the correct features?
  • Place in the pipeline — Can the application code flow through the pipeline and across environments and specified tests successfully and easily?
  • A good customer experience — Are users seeing value in the delivered application?

Continuous testing is the way toward continuous delivery. Teams that struggle with continuously delivering on time or with high quality often find the solution to their problems by setting up good continuous testing practices.

Read the full article for some tips to improve your continuous testing framework and help your DevOps succeed, like-
  • Ensure test automation is the best fit
  • Leverage automation benefits in all aspects
  • Select the right tools
  • The Typical Pipeline & what it requires

Continue Reading ->

Agile teams must strive for continuous improvement of their continuous testing strategy. If they’re successful, they can reduce their release times from months or years to weeks or days (or even hours!). By adopting the correct practices and embracing the spirit of continuous learning and improvement, we can help our testers to become champions of agile.

<Image Credits – manufacturingchemist.com>

Is Test Automation Alienating Your Business Testers?

Nishi Grover GargLeave a comment

With numerous test automation tools and frameworks available today, many in the software testing industry are focused on learning them all. It is important to stay updated with new technology. But are testers losing something in the race to become more technical and equipped with automation skills?

In my article published at TestRail blog, I examine ways to see if your test automation is becoming so technical and code-intensive that it’s in danger of alienating the subject-matter expert testers who best know the core of your business?

Technology should serve people

It is important to understand and remember that test automation tools have been designed to make testers’ lives easier and better. They are not intended to replace testers or overpower them. They make tests execute faster, with more accuracy and fewer errors, so if they eliminate anything, it is redundancy and repetitive work. This technology is meant to serve testers — to save their time and effort and give them more freedom.

To this end, the first intent behind adopting any technology must be its fitness for use in the project, not its popularity in the market. The skills needed to adopt the tool and begin using it in the project should be easily obtained by hands-on learning or training. Read full article ->

Testing is creative

Testing is a creative job, and it always has been. The advent of new tools and technology has not changed this fact. Tools can do part of a tester’s job, but they still cannot test. Although some people may argue on behalf of artificial intelligence and machine learning that can take over many actively creative aspects, we are not there yet. We still want and need a human to capture the creative tests, discuss the pros and cons of design aspects, peer-review test cases, and report problems.

Everyone can contribute to test automation

When we look at testers’ resumes, the tendency is to look for tools they can work with. But the more important skill we need is their ability to contribute to test automation in one way or another. We cannot judge this fact just by asking if a person is able to write test automation scripts or knows a certain programming language. They may be able to learn the Gherkin format to design and write feature files for Cucumber tests. Or if you decide to adopt a keyword-driven framework, they could pick up the keywords and begin writing tests so that the same test cases can double as test scripts.

Read More »

Read Along- ‘Agile Testing’ Chapter-14

Nishi Grover Garg2 Comments

“An Agile Test Automation Strategy”

Use the Agile Test Quadrants to help you identify the different types of test automation tools you might need for each project, even each iteration.

Test Automation Pyramid (introduced by Mike Cohn)

Lowest Layer- Bulk of automated unit , technology facing tests. Quickest feedback, code much more quickly using xUnit family of tools

Middle layer – Automated business-facing tests that help the team. “Are we building the right thing” Tests operate at the API, behind the GUI level. Bypass the presentation layer – less expensive to write & maintain these tests. Fit & FitNesse are good examples, written in domain language

Top Tier – Should be the smallest automation effort as they  have the lowest ROI. Done through GUI, operating on the presentation layer. More expensive to write, more brittle and need more maintenance.

The Test Automation Pyramid
  • Any tedious or repetitive task involved in developing software is a candidate for automation.
  • AN automated deployment process is imperative – getting automated build emails listing every change made is a big help to testers. It speeds up testing & reduces errors.
  • A fast running continuous integration and build process gives the greatest ROI of any automation effort.
  • Another useful area for automation is data creation or setup. Cleaning up test data is as important as generating it. You data creation toolkit should include ways to tear down the test data so it doesn’t affect a different test or prevent rerunning the same test.

What we shouldn’t automate

  • Usability testing
  • Exploratory testing
  • Tests that will never fail
  • One-Off tests
  • Plan-in plenty of time for evaluating tools, setting up build processes, and experimenting with different test approaches in the initial iterations.
  • If management is reluctant to give the team time to implement automation, explain the trade-offs clearly. Work towards a compromise.
  • We will always have deadlines, and we always feel pressed for time. There is never enough time to go back and fix things. During your next planning meeting, budget time to make meaningful progress on your automation efforts.
  • Good test management ensures that tests can provide effective documentation of the system and of the development progress

Read Along- ‘Agile Testing’ Chapter-13

Nishi Grover Garg1 Comment

“Why we Want to Automate Tests and What holds us back”

  • Test automation is a core agile practice. Agile projects depend on automation.
  • Manual tests take too long and are error prone.
  • Automation regression tests provide a safety net. They give feedback early & often.
  • Automated builds, deployment, version control and monitoring also go a long way toward mitigating risk and making your development process more consistent.

“Build Once, Deploy to Many” – is a tester’s dream!

Projects succeed when good people are free to do their best work. Automating tests appropriately makes that happen!

If we become complacent about our testing challenges and depend solely on automated tests to find our issues, and then just fix them enough for the test to pass – we do ourselves a disservice.

However, if we use the tests to identify problem areas and fix them the right way or refactor as needed, then we are using the safety net of automation in the right way.

When tests that illustrate examples of desired behavior are Automated, they become ‘living’ documentation of how the system actually works.

Barriers to Automate –

  • Programmer’s attitude – Why automate at all
  • The Hump of Pain – the initial learning curve
  • Initial Investment
  • Fear – Non-programming testers fear they have nothing to contribute
  • Legacy code
  • Old habits, team culture
  • Without automated regression tests, manual regression testing will continue to grow in scope and eventually may simply be ignored.
  • Teams with automated tests and build processes have a more stable velocity.
  • Good test design practices produce simple, well-designed, continually refactored, maintainable tests.
  • Team culture & history may make it harder for programmers to prioritize automation of business-facing tests than coding new features. Using Agile principles & values helps the whole team overcome barriers to test automation.