Read Along- ‘Agile Testing’ Chapter-18

“Coding and Testing”

  • The beginning of coding is a good time to start writing detailed tests.
  • As testers think of new scenarios to validate with executable tests, they also think about potential scenarios for manual exploratory testing. Make a note of these for later pursuit.
  • Some quick risk analysis can help you decide what testing to do first and where to focus your efforts.

The Power of Three Rule – When unexpected problems arise, you may need to pull in more people or even the entire team. Tester, Developer and Customer (or businesspeople) can together decide on correct behavior and solutions.

Explore It!

As soon as testable chunks of code are available, and the automated tests that guided their coding pass, take time to explore the functionality more deeply. Try different scenarios and learn more about the code’s behavior. You should have task cards for tests that critique the product both business and technology-facing. The story is not ‘done’ until all of these test types are done.

If your exploratory tests lead the team to realise that significant functionality was not covered by the stories, write new stories for future iterations. Keep a tight reign on “Scope Creep” or your team won’t have time to deliver the value you originally planned.

Technology-facing tests that critique the product are often done best during coding. This is the time to know if the design doesn’t scale or if there are security holes.

  • MANAGING DEFECTS
  • Leaving bugs festering n the code base has a negative effect on code quality, system intuitiveness, system flexibility, team morale and velocity.
  • Strive for “zero tolerance” towards bug counts.
  • Teams have solved the problem of how to handle defects in different ways.
    • Some teams put all their bugs on task cards
    • Some teams chose to write a cared, estimate it & schedule it as a story.
    • Some teams suggest adding a test for every bug
  • The more bugs you can fix immediately, the less technical debt your application generates and the less ‘defect’ inventory you have.
  • Try making the estimate for each story to include (atleast) two hours or half a day for fixing associated bugs.

If a bug is really missed functionality, choose to write a card for the bug and schedule it as a story.

Code produced test-first is fairly free of bugs by the time it is checked-in.

  • The Daily Stand-Up helps teams maintain the close communication they need.
  • Use Big, visible charts such as story boards, Burndown charts and other visual cues to help keep focus and know your status.
  • Having story boards gives your team focus suring the stand-ups or when you are talking to someone outside the team about your progress.

Communication

  • Testers can help keep the iteration progressing smoothly by helping make sure everyone is communicating enough. They can help programmers and customers find a common language.
  • Use retrospectives to evaluate whether collaboration & communication need improving and brainstorm ways to improve.
  • Teams in different locations have to make a special effort to keep each other informed.

Build Process

  • Teams take different approaches to make sure their build stays ‘green’.
  • The build needs to provide immediate feedback, so Keep It Short.
  • Tests that take too long, such as tests that update the database, functional tests above Unit level or GUI test scripts, should run in a separate build process.
  • Having a separate, continual ‘Full’ build with all of the regression suites is worth the investment.

During the iteration, you are automating new tests. As soon as these pass, add them to the Regression Suite.

As you start the iteration, make sure that test environments, test data, and test tools are in place to accommodate testing.

You may have brought in outside resources for the iteration to help with performance, security, usability or other forms of testing. Include them in stand-ups and discussions. Pair with them to help them understand the team’s objectives. This is an opportunity to pick up new skills!!

  • Consider what metrics you need during the iteration – Progress and Defect Metrics are 2 examples.
  • Whatever metrics you choose to measure – Go for Simplicity!

Read Along- ‘Agile Testing’ Chapter-17

“Iteration Kickoff”

  • Most teams kickoff their new iteration with a planning session. – where they discuss one story at a time, writing & estimating all of the tasks needed to implement it.
  • Task cards need to be written along with development task cards and estimated realistically.
  • When writing programming task cards, make sure that coding task estimates include time for writing unit tests and for all necessary testing by programmers.
  • Testers should help make sure that all necessary cards are written and they have reasonable estimates.

Your job as a tester is to make sure enough time is allocated to testing and to remind the team that testing & quality are the responsibility of the whole team. When the team decides how many stories they can deliver in an iteration, the question isn’t “How much coding can we finish?” but “How much coding and testing can we complete?”

Commit Conservatively – It is always better to bring in another story later than to drop a picked story.

  • Working closely with customers or customer proxies is one of the most important activities as an agile tester. Good communication usually takes work.
  • We want “big-picture” tests to help the programmers get started in the right direction on a story. High level tests should convey the main purpose behind the story.
  • Don’t forget to ask the programmers what they think you might have missed. What are the high-risk areas of the code? Where do they think testing should be focused?

When Testability is an issue, make it the team’s problem to solve.

One beneficial side-effect of reviewing the tests with programmers is the cross-learning that happens.

High level test cases along with executable tests you’ll write during the iteration will form the core of the application’s documentation.

People unfamiliar with agile development often have the misconception that there’s no documentation. In fact, agile projects produce usable documentation that contains executable tests and thus, is always up to date.

Tips to write better Bug Reports

Writing defect reports is a constant part of a tester’s daily life, and an important one too! How you report the bugs you find plays a key role in the fate of the bug and whether it’s understood and resolved or ends up being deferred or rejected.

It is imperative for every tester to communicate the defects they find well. In my article published at TestRail blog, I discuss four simple tips to help you write better bug reports.

Study past bug reports

If you are new to your team or new to testing itself, the best way to learn about good bug reporting is to read the team’s past bug reports. Try to understand the bugs and see if you could reproduce them. 

By doing this exercise you learn the best way to present bugs so that the team can understand them. You’ll get a feel for the business language and the project’s jargon to be able to describe features and modules.You may also see some imperfections in the past reports, so you can think about how to improve them and what other information would be useful to include.

Create your own game plan

Create a shortcut for yourself, like writing down a summary or title of each bug you find as you go about testing and saving the screenshot. When you get down to reporting, you can quickly fill out the steps, as well as expected and actual results, and attach the saved screenshots. Doing this could be faster and save you the effort of repeating steps just to get the needed screenshots and logs. Continue Reading–>

Read More »

Read Along- ‘Agile Testing’ Chapter-9

“Toolkit for Business-Facing Tests that Support the Team”

  • As agile development has gained in popularity, we have more and more tools to help us capture and use them to write executable tests.
  • Your strategy for selecting the tools you need should be based on your team’s skill set, the technology your application uses, your team’s automation priorities, time, and budget constraints. Your strategy should NOT be based on the latest and coolest tool in the market.
  • In agile, simple solutions are usually best.
    • Some tools that can help us illustrate desired behavior with examples, brainstorm potential implementations and ripple effects and create requirements we can turn into tests are—
      • Checklists
      • Mind maps
      • Spreadsheets
      • Mockups
      • Flow diagrams
      • Software based tools
  • A picture is worth a thousand words, even in agile teams. Mock-ups show the customer’s desires more clearly than a narrative possibly could. They provide a good focal point to discussing the desired code behavior.
    • Visuals such as flow diagrams and mind maps are good ways to describe an overview of a story’s implementation, especially if it is created by a group of customers, programmers, and testers.
    • Tools such as Fit (Framework for Integrated Tests) and FitNesse were designed to facilitate collaboration and communication between the customer and development teams.
    • Finding the right electronic tools is particularly vital for distributed teams (chat, screensharing, video conferencing, calling, task boards etc.)
    • Selenium, Watir and WebTest are some examples of many open source tools available for GUI testing.
  • Home-Brewed Test Automation Tools
    • Bret Pettichord (2004) coined the term ‘home-brewed’ for tools agile teams create to meet their own unique testing needs. This allows even more customisation than an open source tool. They provide a way for non technical customer team members to write tests that are actually executable by the automated tool. Home-brewed tools are tailored to their needs, designed to minimize the total cost of ownership and often built on top of existing open source tools.

The best tools in the world won’t help if you don’t use them wisely. Test tools might make it very easy to specify tests, but whether you are specifying the right tests at the right time is up to you.

Writing detailed test cases that communicate desired behavior is both art and science.

Whenever a test fails in Continuous Integration (CI) and build process, the team’s highest priority should be to get the build passing again. Everyone should stop what they are doing and make sure that the build goes ‘green’ again. Determine if a bug has been introduced, or if the test simply needs to be updated to accommodate intentionally changed behavior. Fix the problem, check it in, and make sure all tests pass!

Experiment – so that you can find the right level of detail and the right test design for each story.

  • Keep your tests current and maintainable through refactoring.
  • Not all code is testable using automation but work with programmers to find alternative solutions to your problems.
  • Manual test scenarios can also drive programming if you share them with the programmers early. The earlier you turn them into automated tests, the faster you will realise the benefit.

Start with a simple approach, see how it works, and build on it. The important thing is to get going writing business-facing tests to support the team as you develop your product.

Raise your Exploration Game!

Exploration is an integral part of testing. Exploring the application is a great strategy for learning about how it works, finding new information and flows, and discovering some unique bugs too! 

Many testers perform exploratory testing as a matter of course, and agile teams may make it an integral part of their tasks. But how can you up your exploration game? Simply going around the application and looking or clicking here and there surely cannot be called creative exploration.

In my article published at Testrail blog, I outline what do you need to do to bring structure to your exploratory tests and get the most useful information out of them?

Image Source- xenonstack.com

Designate time for exploration

As we get into the flow of agile and its fast-moving sprints, we focus on testing tasks for each user story and are constantly thinking of what needs to be done next. But with minimal documentation and limited time to design tests, it is imperative to understand that just executing the written or scripted tests will not be enough to ensure the feature’s quality, correctness, and sanity.

Exploratory testing needs to be counted as a separate task. You can even add it to your user story so that the team accounts for the time spent on it and recognizes the effort.

Testers can use the time to focus on the feature at hand and try out how it works, its integrations with other features, and its behavior in various unique scenarios that may or may not have been thought of while designing the scripted tests. Having exploratory testing as a task also mandates that it be done for each and every feature and gives testers that predefined time to spend on exploration. 

In my testing days, this used to be the most creative and fun aspect of my sprints, and it resulted in great discoveries, questions, insights, and defects!

Read More »

Top Cross Browser Testing Challenges and How to Overcome them via Automation

Have you ever wondered how to successfully automate your cross-browser tests? With the number and type of mobile and tablet devices available in the market increasing daily and the crazy combination of browser types and browser versions making things even more complicated, if you are a website or web app developer then making sure your application renders and functions correctly on all those combination of browsers, devices and platforms is often enough to make you want to pull out your hair! Add things like compatibility and browser support for IE11 to the mix and things can get pretty tense. However, with the recent advancements in cross browser test accelerator technologies today we can perform these cross browser tests more reliably and more extensively than ever before.

Before we delve deeper into different approaches to automate your cross browser testing efforts, let’s first see what Cross Browser Testing is all about, why performing cross platform compatibility testing is often inadequate because of the various challenges associated with it, how to mitigate these challenges via test automation and finally, all the features to look for when comparing some of the best cross browser testing tools to automate such testing efforts.

What is Cross Browser Testing?

Cross Browser Testing is the type of testing where we verify to ensure that an application works as expected across different browsers, running on different operating systems and device types. In other words, by performing this type of functional testing a tester checks the compatibility of a website or web app across all supported browser types. Thus, by conducting specialized browser testing, you can ensure that the website / web app is able to deliver an optimal user experience, irrespective of the browser in which it is viewed or accessed.

Major Challenges with Cross-Browser Testing

Let us face it! Testing a web application across all major browser/device/OS platform combinations can be a seriously daunting task. One of the major pain-point with performing thorough Cross Browser Testing is that your testing team would have to test the same website or web application across all the different browsers, operating systems and mobile devices. This is when each browser uses their own different technology to render HTML. Mentioned below are some of the major aspects that make cross browser testing challenging.

1. It is IMPOSSIBLE to test in All Browser Combinations

Let’s assume that your contract with the client mandates that the website or web application being developed should support Chrome, Safari, Firefox, Opera, and Internet Explorer on Windows, macOS, and Linux operating systems. While this may rather seem a little too formidable at first, it actually is pretty manageable:

macOS: 4 Browsers (Chrome, Safari, Firefox, Opera)

Windows: 4 Browsers (Internet Explorer, Chrome, Firefox, Opera)

Linux: 3 Browsers (Chrome, Firefox, Opera)

That’s a total of 11 browser combinations.

But not all your end users are expected to be using the very latest version of each of these browsers. So it is often safe to test using at least the latest 2 versions of each browser.

macOS: 8 Browsers (Chrome, Safari, Firefox, Opera)

Windows: 8 Browsers (Internet Explorer, Chrome, Firefox, Opera)

Linux: 6 Browsers (Chrome, Firefox, Opera)


That’s a total of 22 browser types.

Now that we have taken the latest 2 versions of each browser type into consideration how about the latest versions of each OS? Surely, people upgrade their OS far less often than they upgrade their browsers, right? So to be safe, let’s test across the latest 3 versions of each OS platform.

macOS Catalina: 8 Browsers (Chrome, Safari, Firefox, Opera)

macOS Mojave: 8 Browsers (Chrome, Safari, Firefox, Opera)

macOS High Sierra: 8 Browsers (Chrome, Safari, Firefox, Opera)

Windows 10: 8 Browsers (Internet Explorer, Chrome, Firefox, Opera)

Windows 8.1: 8 Browsers (Internet Explorer, Chrome, Firefox, Opera)

Windows 8: 8 Browsers (Internet Explorer, Chrome, Firefox, Opera)

Ubuntu 20.04: 6 Browsers (Chrome, Firefox, Opera)

Ubuntu 19.10: 6 Browsers (Chrome, Firefox, Opera)

Ubuntu 18.04: 6 Browsers (Chrome, Firefox, Opera)


That’s a total of 66 browser combinations.

What started out as a manageable list is now already a substantial and daunting list of browser combinations to test against even for teams with a dedicated team of a good number of QA specialists. Add to the mix the possibility of testing across 32x and 64x variations of each OS type, testing across various possible screen resolutions and the fact that you’d need to retest across each of these combinations every time there is a bug fix, it is easy to feel frustrated and even give up!

Read More »

Testing is like…… Yoga

This post is inspired by the MOT bloggers club initiative to write about analogies to testing in real life!

Being a tester at heart, I always see things from a testers eyes and find relevance in testing in my day-to-day life. In the past I have thought and spoken about Testing being like… Cooking and also used analogies of Testing equating to Travelling when explaining the Software Testing lifecycle in my Tester Bootcamps and trainings. Lately I have gotten into Yoga and I now see how Testing is like Yoga in many ways…….

  • You can start anytime and anywhere you want, no matter your background.
  • You can learn it yourself — Researching and Reading will help but Practice is key!
  • You will learn better when you take help from a teacher / mentor / guru Or when you practice with a team
  • Even though on the surface level, people may think of it as one skill, there are many types of testing, just like there are of Yoga
    • Hatha Yoga, Vinyasa Yoga, Pranayama (Breathing exercises yoga), Pre-natal yoga and the fusion kind – Power Yoga
    • The same way we have Functional testing, Performance testing, Usability testing, Security testing, Automated testing and so on
    • You can dive into any one in-depth or have a taste of all of them!
    • There is one for every team, context and need- you need to find the right match(es)
  • Testing , like Yoga – is context-dependent
    • Just like Yoga for weight loss may be different than Yoga for an expectant mother, Yoga for a beginner may be different from Yoga for an athlete recovering from an injury; so is the case of Testing.
    • Testing for a medical application will be vastly different from Testing of a Car racing mobile game or testing for a banking website.
    • The basics and the fundamental concepts remain the same and apply equally to all though!
  • To a person looking from outside, it may not mean much in the beginning
    • Like, to a person looking at you holding a Yoga pose – It may not seem like you are doing much. But to the one experiencing it, it make the world of a difference.
Holding a Yoga pose is harder than it looks

And finally, for both Testing and Yoga—

The value is not realized in one day or one session. It is a prolonged effort, requiring consistent practice, patience and persistence.
Overtime people who see the changes and experience the difference come to appreciate the real benefits —- of both Yoga and Testing!! 🙂 🙂

************

Hope you enjoyed my take on Testing is like ….. Challenge. Please share your thoughts too!

Here is the link to follow the MoT Blogger Club group for many more interesting takes on this Challenge

https://club.ministryoftesting.com/t/bloggers-club-june-july-2020-testing-is-like/39734/8

Cheers

Nishi

<Image Credits – WebMD.com , youtube.com >

Four Goals of Testing Beyond Finding Defects

Are you testing with the sole purpose of finding defects? What if you don’t find any? Your testing should deliver more value than just finding bugs. In my article published at https://blog.gurock.com/, I examined the true goals of testing and how we can aim at achieving all four of them for the quality benefits of our software.

Gaining knowledge about defects 

While there is more to testing than pinpointing bugs, finding defects and problems is the first instinctive goal. Looking for places where the functionality breaks or does not work as expected is key. 

Testers can adopt a number of approaches, test techniques and strategies to find these problems before users do. This helps the team keep updated on the status of product quality, fix the problems, and improve the software for the users.

Proving functionality

If you have been testing diligently and going through a bunch of test cases and various scenarios but haven’t yet found a defect, it doesn’t mean it was all for nothing! If a test doesn’t fail, that means it passed, and that is useful information, too.

Another major goal of testing is to prove that the functionality works fine, and it is that proof that helps us make decisions about its future. Without this proof, we would never have a clear picture of the software’s quality, its intended functionality or whether it’s fit for use. Many teams would also get into problems with regulations, audits, and compliance without this proof of functionality.

Generating information

Testing also generates a lot more information than just passing or failing tests. Testers generally have loads of questions occur to them while testing. They may be about the need, implementation or design of the features, their related integrations with existing features, or actual usage scenarios. The answers to these questions are paramount in making the feature assimilate well within the software. 

Read More »

My experience speaking at Testbash Home

The Ministry of Testing (MoT) is definitely the biggest and the most supportive testing community. Having heard so much about their Testbash events conducted world-wide, speaking at one was a long time goal. And I was fortunate enough to be accepted to speak at Testbash Detroit this year. But as things progressed since the beginning of 2020, travels and conferences of any kind were far from possible in light of a the global pandemic of Covid-19. Alas! our dreams were shattered. And though, it was disheartening for sure, the awesome community jumped back from the jolt and got together to bring us all an awesome online event #Testbash Home 2020.

Preparations began and I too got back to preparing my talk, which I had given up on after the cancellations! Took a couple of weekends sorting out the content and slides. Then we had a call to record the talks with the community members Heather and Diana were ever so supportive and so kind with their emails, scheduling and feedback! This was a wonderful idea to have the talks pre-recorded so that we are not hampered by any technical glitches on the event day, while we speakers get to focus on engaging with everyone and answering questions from the community.

As the day of the event approached, I prepared for my live interview. The event had more than 1000 registrations! Definitely making it the biggest audience I have ever presented to. Though the event began late night hours for my timezone, my talk was at a convenient morning hour. So that is when I joined in. Had a wonderful chat with Richard who was the Backstage boss and handling the entire livestream for the entire 24 hours! Checked on my audio & video etc and also had introductions with James who was the host for that part. And then we were live!

The duration of the talk went great. It was surreal listening to myself presenting, and looking at the live chat and questions coming from the participants throughout the talk. Once it ended, I was back live with my video. Me and James continued to discuss the most popular voted questions asked and I answered them the best to my knowledge. It was amazing to see such great comments and kind appreciation by the listeners in the chats once we were done. #Grateful

Once my talk was done, I could now continue to enjoy the rest of the live event! #Testbash Home was an absolute treat with a mix of great content, discussions, community participation, fun hosts and great conversations! It sure has set the bar really high for all online events in the future. I stayed throughout the next 5 parts of the event and only left late at night when it was absolutely impossible to keep my eyes open 😛

It sure felt like a day away from our regular stay-at-home lives, and felt like we had met up with so many people in the virtual world. Some key highlights of the day were-

  • Awesome talks by speakers
  • Black Box puzzles played live with volunteers
  • 99 second talks with many enthusiastic participants, many of whom were presenting for the first time!
  • The breakout room was so much fun – where you could select your avatar, enter a virtual room and just chit chat (and play with Ralph the dogBoss 😛 )
  • The breaks in-between parts had the background noise of an actual conference hall with people chattering and plates clanking. It was so soothing to hear (given the times we are in!) A fantastic idea! 🙂
  • The hosts did an awesome job engaging everyone in informal chats, yoga, discussing shows we are watching, things we are cooking and what not. Considering that it was a 24 hour long event, it sure was a welcome change of pace every few hours.
  • The short intros of all the MoT community bosses was so much fun to watch and made it very relatable. Now we know the faces behind the names.

Overall, TestBash home was an awesome experience, and I was fortunate to get some great feedback for my first ever Testbash Talk! I also loved the sketch-note of my talk created by Louise Gibbs

Sketch note Created by Louise Gibbs

I look forward to taking it further and engaging with this community in a live Testbash event some day! 🙂

Cheers

Nishi

Read Along- ‘Agile Testing’ Chapter-4

“Team Logistics”

  • Testers on agile teams feel very strongly about their role as customer advocate and also feel they can influence the rest of the team with their quality thinking.
  • Testers also need training on pair testing, working with incomplete and changing requirements, automation and all of the other new skills that are required.
  • Programmers also might need coaching to understand the importance of business-facing tests and the whole-team approach to writing and automating tests.
  • The pairing of programmers and testers can only improve communication about the quality of the product.

“One major advantage of an integrated project team is that there’s only one budget and one schedule.”

The difference between a traditional cross-functional team and an agile team is the approach to the whole-team effort. Members are not just “representing” their functions in the team but are becoming true members of the team for as long as the project or permanent team exists.

  • The best teams are those that have learned to work together and have developed trust with one another.
  • Teams work better when they have ready access to all team members, easy visibility of all progress charts and an environment that fosters communication.
  • Teams make the best progress when they’re empowered to identify and solve their own problems. If you’re a manager, resist the temptation to impose all your good ideas on the team.
  • Make sure testers are involved in all meetings! If you are a tester and someone forgets to invite you to a meeting, invite yourself!

“Courage is especially important. Get up and go talk to people; ask how you can help. Reach out to team members and other teams for direct communication. Notice impediments ad ask the team to help remove them.”

Agile development works because it gets obstacles out of our path and lets us do our best work.

——- On to the next one!

Cheers

Nishi