The beginning of coding is a good time to start writing detailed tests.
As testers think of new scenarios to validate with executable tests, they also think about potential scenarios for manual exploratory testing. Make a note of these for later pursuit.
Some quick risk analysis can help you decide what testing to do first and where to focus your efforts.
The Power of Three Rule – When unexpected problems arise, you may need to pull in more people or even the entire team. Tester, Developer and Customer (or businesspeople) can together decide on correct behavior and solutions.
As soon as testable chunks of code are available, and the automated tests that guided their coding pass, take time to explore the functionality more deeply. Try different scenarios and learn more about the code’s behavior. You should have task cards for tests that critique the product both business and technology-facing. The story is not ‘done’ until all of these test types are done.
If your exploratory tests lead the team to realise that significant functionality was not covered by the stories, write new stories for future iterations. Keep a tight reign on “Scope Creep” or your team won’t have time to deliver the value you originally planned.
Technology-facing tests that critique the product are often done best during coding. This is the time to know if the design doesn’t scale or if there are security holes.
Leaving bugs festering n the code base has a negative effect on code quality, system intuitiveness, system flexibility, team morale and velocity.
Strive for “zero tolerance” towards bug counts.
Teams have solved the problem of how to handle defects in different ways.
Some teams put all their bugs on task cards
Some teams chose to write a cared, estimate it & schedule it as a story.
Some teams suggest adding a test for every bug
The more bugs you can fix immediately, the less technical debt your application generates and the less ‘defect’ inventory you have.
Try making the estimate for each story to include (atleast) two hours or half a day for fixing associated bugs.
If a bug is really missed functionality, choose to write a card for the bug and schedule it as a story.
Code produced test-first is fairly free of bugs by the time it is checked-in.
The Daily Stand-Up helps teams maintain the close communication they need.
Use Big, visible charts such as story boards, Burndown charts and other visual cues to help keep focus and know your status.
Having story boards gives your team focus suring the stand-ups or when you are talking to someone outside the team about your progress.
Testers can help keep the iteration progressing smoothly by helping make sure everyone is communicating enough. They can help programmers and customers find a common language.
Use retrospectives to evaluate whether collaboration & communication need improving and brainstorm ways to improve.
Teams in different locations have to make a special effort to keep each other informed.
Teams take different approaches to make sure their build stays ‘green’.
The build needs to provide immediate feedback, so Keep It Short.
Tests that take too long, such as tests that update the database, functional tests above Unit level or GUI test scripts, should run in a separate build process.
Having a separate, continual ‘Full’ build with all of the regression suites is worth the investment.
During the iteration, you are automating new tests. As soon as these pass, add them to the Regression Suite.
As you start the iteration, make sure that test environments, test data, and test tools are in place to accommodate testing.
You may have brought in outside resources for the iteration to help with performance, security, usability or other forms of testing. Include them in stand-ups and discussions. Pair with them to help them understand the team’s objectives. This is an opportunity to pick up new skills!!
Consider what metrics you need during the iteration – Progress and Defect Metrics are 2 examples.
Whatever metrics you choose to measure – Go for Simplicity!
This chapter reviews all the four Agile Testing Quadrants by illustrating an example of a team’s success story in testing their whole system using a variety of home-grown and open source tools.
The system is related to Monitoring of Remote Oil and Gas Production Wells. The software application had a huge legacy system, with very few unit tests. The team was slowly rebuilding the application using new technology. And describes how they used tests from all four quadrants to support them.
Using Test Driven Development and Pair Programming wholeheartedly. Also adding unit tests and refactoring all legacy code they encountered on the way.
The product engineer writing acceptance tests and sharing with the developers and testers before they began creating.
Automation involving functional test structure, web services and embedded testing
Exploratory testing to supplement the automated tests to find critical issues.
Don’t forget to Document… but only what is useful
Finding ways to keep customers involved in all types of testing, even if they are remote. Have UATs and end to end tests
Use lessons learnt during testing to Critique the product in order to drive the development in next iterations
“Critiquing the Product Using Technology-Facing Tests”
Technology-facing tests that critique the product are more concerned with the non-functional aspects – deficiencies of the product from a technical point of view.
We describe requirements using a programming domain vocabulary. This is the main of Quadrant-4 of our Agile Testing Quadrants.
Customers simply assume that software will be designed to properly accommodate the potential load, at a reasonable rate of performance. It doesn’t always occur to them to verbalize those concerns.
Tools, whether home-grown or acquired, are essential to succeed with Quadrant 4 testing efforts.
“Many teams find that a good technical tester or toolsmith can take on many of these tasks.”
Take a second look at the skills that your team already posseses, and brainstorm about the types of “ility” testing that can be done with the resources you already have. If you need outside teams, plan for that in your release and iteration planning.
The information these (Quadrant-4) tests provide may result in new stories and tasks in areas such as changing the architecture for better scalability or implementing a system-wide security solution. Be sure to complete the feedback loop from tests that critique the product to tests that drive changes that will improve the non-functional aspects of the product.
When Do you Do it?
Technical stories can be written to address specific requirements.
Consider a separate row on your story board for tasks needed by the product as a whole.
Find a way to test them early in the project
Prioritize stories such that a steel thread or a thin slice is complete early, so that you can create a performance test that can be run and continued as you add more functionality.
The time to think about your non-functional tests is during release or theme planning.
The team should consider various types of “ility” testing including – Security, maintainability, Interoperability, Compatibility, Reliability and Installability – and should execute them at appropriate times.
Performance, Scalability, Stress and Load tests should be done from the beginning of the project.
Writing defect reports is a constant part of a tester’s daily life, and an important one too! How you report the bugs you find plays a key role in the fate of the bug and whether it’s understood and resolved or ends up being deferred or rejected.
It is imperative for every tester to communicate the defects they find well. In my article published at TestRail blog, I discuss four simple tips to help you write better bug reports.
Study past bug reports
If you are new to your team or new to testing itself, the best way to learn about good bug reporting is to read the team’s past bug reports. Try to understand the bugs and see if you could reproduce them.
By doing this exercise you learn the best way to present bugs so that the team can understand them. You’ll get a feel for the business language and the project’s jargon to be able to describe features and modules.You may also see some imperfections in the past reports, so you can think about how to improve them and what other information would be useful to include.
Create your own game plan
Create a shortcut for yourself, like writing down a summary or title of each bug you find as you go about testing and saving the screenshot. When you get down to reporting, you can quickly fill out the steps, as well as expected and actual results, and attach the saved screenshots. Doing this could be faster and save you the effort of repeating steps just to get the needed screenshots and logs. Continue Reading–>
Are you testing with the sole purpose of finding defects? What if you don’t find any? Your testing should deliver more value than just finding bugs. In my article published at https://blog.gurock.com/, I examined the true goals of testing and how we can aim at achieving all four of them for the quality benefits of our software.
Gaining knowledge about defects
While there is more to testing than pinpointing bugs, finding defects and problems is the first instinctive goal. Looking for places where the functionality breaks or does not work as expected is key.
Testers can adopt a number of approaches, test techniques and strategies to find these problems before users do. This helps the team keep updated on the status of product quality, fix the problems, and improve the software for the users.
If you have been testing diligently and going through a bunch of test cases and various scenarios but haven’t yet found a defect, it doesn’t mean it was all for nothing! If a test doesn’t fail, that means it passed, and that is useful information, too.
Another major goal of testing is to prove that the functionality works fine, and it is that proof that helps us make decisions about its future. Without this proof, we would never have a clear picture of the software’s quality, its intended functionality or whether it’s fit for use. Many teams would also get into problems with regulations, audits, and compliance without this proof of functionality.
Testing also generates a lot more information than just passing or failing tests. Testers generally have loads of questions occur to them while testing. They may be about the need, implementation or design of the features, their related integrations with existing features, or actual usage scenarios. The answers to these questions are paramount in making the feature assimilate well within the software.
It sure is going to be a great event with some awesome talks lined up by internationally renowned speakers.
My talk is going to be about “Adopting a Simplified Risk Based Testing approach” which includes a little bit of a story from my past project and a showcasing how to implement a simplified risk based approach on a new project.
Here we are gearing up for another new year! As time flies by, we may start to feel stuck in one place, unable to move forward in our careers. Testers can get bogged down by too much to learn, too many directions to take, and so many tools and technologies.
But that’s no reason to stagnate. By making some goals now, you can aim to start improving yourself and your career development right away on January 1st.
The first resolution should be to create and maintain a healthy mindset. Mental peace and team harmony should be the goal.
There must be a routine, a drive to better oneself and a constant search for improvement. All testers must resolve to take up some kind of continuing education so they can always be adding to their skill sets. Learning cannot be a one-time activity.
Get Better at Networking
The next resolution a tester must make is to participate in the community in some way. The knowledge you have is better shared with others, and the pace of learning in a community will be much faster than alone.
A walkthrough is a great review technique that can be used for sharing knowledge, gathering feedback and building a consensus. Typically, walkthroughs take place when the author of a work item explains it in a brief review meeting — effectively walking the team through the work — and gathers people’s feedback and ideas about it. These meetings may be as formal or as informal as needed and may also be used as a knowledge-sharing session with many relevant stakeholders at once.
In my article published at https://blog.gurock.com/tester-agile-walkthrough/ , I have discussed three ways agile testers can make use of this type of review for their sprint- and release-level test plans and test cases to get the entire team involved in the quest for quality.
I have also discussed how I have used walkthroughs in my agile team as a mechanism to review our sprint test scenarios with the entire Scrum team. The main areas of application are-
Walkthroughs are a quick and easy review technique to adopt, and they can be especially useful for testers on an agile team to get reviews on their test plans, test cases, and scripts. Give this technique a try, even if in an informal sense, and see how beneficial it can be!
Testers who analyze quality in every aspect of the team’s deliverable also have a responsibility to mitigate risks and practical issues that are bound to come up, and help the team succeed in their product as well as at being agile. Here are five such issues that testers can help the team alleviate or avoid.
Let’s say an agile team has a user story to develop a webpage in a sprint. The product manager mentions only the mandatory fields of the webpage in the story’s description while scoping it in the sprint. The developer works on the story for a couple of days and adds the listed fields on the page, but because he going on leave for a few days, he only provides the basic form fields without any input validations or error messages on the page.
Once back from his vacation, he checks in the code and shares it for testing a day before the sprint ends. The tester tests it for a day and raises some minor severity issues. In the final sprint demo, the product manager points out that a number of fields are missing on the webpage. Because testing is not complete yet, the story is tagged as a spill-over item to the next sprint. Few UI changes are added in the next sprint’s scope, along with the pending validations and error messages.
What do you think went wrong in this story?
Agile is all about controlling and minimizing the typical risks of conventional software development techniques. When working with agile, we have the flexibility to control our direction and steer our path based on unforeseen and unplanned changes and events.
But it is the same flexibility and dynamic nature that makes agile prone to many risks during the project.
These risks are the practical issues that any agile team is bound to face eventually and that, if not handled quickly, may lead to failure of the sprint or the agile process as a whole. Testers who analyze quality in every aspect of the team’s deliverables also have a responsibility to mitigate these risks and help the team succeed in their product as well as at being agile.
Here are five such issues that testers can help the team alleviate or avoid.
Risk 1: Ambiguous User Stories
Agile assumes that each user story is a unit of work to be achieved in a sprint. The user story needs to be granular enough and have all relevant details and context when scoped. But because agile also recognizes changing requirements as likely, this gets used as an excuse to not list the requirements for each user story or give details early on. This leads to ambiguous requirements, often resulting in low-quality outcomes and rework on the feature.
The actual aim of agile is to work on a feature based on the best knowledge of its requirements at the given time. The sprint format in the Scrum process leaves scope to work on a feature again if enhancement is needed. But that is based on collecting all available information for one story and only then scoping it in a sprint. If that is not ready yet, the story should not be scoped in the current sprint.
Testers in an agile team have the responsibility of reviewing user stories when they are being planned for the sprint and requesting clarification from the product management. Similarly, developers can raise questions about technical and design details they feel are missing and get them defined during planning sessions. Based on these details, testers will create their test scenarios around that user story, and developers will create their implementation design and keep asking more questions if needed over the next few days of the sprint.
In the example at the beginning, the tester could have raised the concern that the user story did not have complete details for the webpage to be developed while scoping it in the sprint. The developer should have asked about relevant validations and error messages needed, as well as some broad layout details to work with.
Risk 2: Not Delivering the Highest Business Value
When a sprint is in progress, developers may dig into their assigned stories and, in trying to get the more complicated things done, digress from the priority of each story. As testers, we constantly need to question the delivery of the most valuable features in the sprint so that we are sure to test and deliver them as early as possible.
During the sprint planning discussion, we must assign business priorities to each story being scoped in. The sequence in which the stories will be delivered will depend on that.
Testers may also raise concerns when more time will be needed to test a feature, and request for it to be delivered early so that the testing can be completed well within the sprint deadline.
At all times we must assume that only the stories that are done from both development and testing perspectives are the ones that are designated as delivered in the sprint final build.
In the example, the developer could have shown the developed page to the product manager and asked for feedback during the sprint to avoid multiple iterations. The tester should have asked for a buddy build for the story before the developer left for vacation to provide more testing time and get maximum issues within the sprint. This would have avoided the task needing to spill over to the next sprint.
Risk 3: Quality Concerns
During the sprint testing cycles, all issues found pertaining to a feature must be tagged with the relevant user story. This can be done easily by linking the issues together in the issue-tracking tool being used.
The testing team must decide on a tolerance threshold for the number of issues for each user story based on their severity, beyond which they would raise a quality concern against that story. Testers should then request these issues to be fixed before any further development is done on that story. This helps immensely in avoiding delivery of half-baked features with too many known issues yet to be fixed at the end of the sprint. This way, at the end of the sprint, all user stories are at a minimum quality benchmark with no obvious or critical issues. Of course, improvements and enhancements are always welcome and may be taken in further sprints.
In the example story,the tester should have raised maximum issues against the user story within the sprint and brought them to the product manager’s attention, along with missing or undelivered functionalities.
Risk 4: Resource and Time Conflicts
When working in a sprint format, the process is bound to be very fast-paced, with each team member being relied on heavily for the assigned user stories.
There may be parallel tracks or other activities, like performance improvements, code refactoring, white box tests, or documentation and reviews, that team members may be expected to handle, which, if they go beyond a number of hours per sprint, may hinder the sprint deliverables. It is the team members’ responsibility to bring any such activities to the ScrumMaster’s notice as risks during the daily stand-up meeting.
The testing team may raise any planned resource or time conflicts during the sprint planning session itself, so that estimation may account for these time losses. Team members also are expected to bring up any planned vacations or leave during the planning session so that their absence may be accounted for during estimation.
In the example story,the developer’s leave plan should have been taken into account when assigning the user story, and consequently, there could have been a co-owner of the story to take up the additional parts, such as implementing validations and UI improvements to the page while the developer was away.
Risk 5: Failures in Collaboration and Communication
Scrum ceremonies, such as the sprint planning sessions, daily stand-ups, triage meetings, and sprint retrospectives meetings, rely heavily on the team’s self-drive and total involvement. Without these factors, the ceremonies become mere formalities, adding little value to the process.
Testers in an agile team must be involved in every phase of the agile development process and raise concern if the team is not performing any of the ceremonies in the correct fashion. Only with full participation and regular communication can we actually achieve success as an agile team.
In the example, the team could have communicated more over the user story, talked to the product manager, and proactively gotten more information, details, and feedback during development. The tester could have initiated more discussions around the requirements and worked closely with the developer to highlight issues instead of waiting for the final build at the end of the sprint.
A tester’s role in an agile team is to provide assistance and support in all areas, so they may need to switch hats frequently, depending on the team’s needs. The success of this endeavor, though, also depends on an open, communicative, and receptive environment fostered within the team. Following these basic pointers can make testing a fun and collaborative part of the agile environment and add a lot more value.