Tag: book

Read Along- ‘Agile Testing’ Chapter-13

Nishi Grover Garg1 Comment

“Why we Want to Automate Tests and What holds us back”

  • Test automation is a core agile practice. Agile projects depend on automation.
  • Manual tests take too long and are error prone.
  • Automation regression tests provide a safety net. They give feedback early & often.
  • Automated builds, deployment, version control and monitoring also go a long way toward mitigating risk and making your development process more consistent.

“Build Once, Deploy to Many” – is a tester’s dream!

Projects succeed when good people are free to do their best work. Automating tests appropriately makes that happen!

If we become complacent about our testing challenges and depend solely on automated tests to find our issues, and then just fix them enough for the test to pass – we do ourselves a disservice.

However, if we use the tests to identify problem areas and fix them the right way or refactor as needed, then we are using the safety net of automation in the right way.

When tests that illustrate examples of desired behavior are Automated, they become ‘living’ documentation of how the system actually works.

Barriers to Automate –

  • Programmer’s attitude – Why automate at all
  • The Hump of Pain – the initial learning curve
  • Initial Investment
  • Fear – Non-programming testers fear they have nothing to contribute
  • Legacy code
  • Old habits, team culture
  • Without automated regression tests, manual regression testing will continue to grow in scope and eventually may simply be ignored.
  • Teams with automated tests and build processes have a more stable velocity.
  • Good test design practices produce simple, well-designed, continually refactored, maintainable tests.
  • Team culture & history may make it harder for programmers to prioritize automation of business-facing tests than coding new features. Using Agile principles & values helps the whole team overcome barriers to test automation.

Read Along- ‘Agile Testing’ Chapter-12

Nishi Grover Garg1 Comment

“Summary of Testing Quadrants”

This chapter reviews all the four Agile Testing Quadrants by illustrating an example of a team’s success story in testing their whole system using a variety of home-grown and open source tools.

The system is related to Monitoring of Remote Oil and Gas Production Wells. The software application had a huge legacy system, with very few unit tests. The team was slowly rebuilding the application using new technology. And describes how they used tests from all four quadrants to support them.

  • Using Test Driven Development and Pair Programming wholeheartedly. Also adding unit tests and refactoring all legacy code they encountered on the way.
  • The product engineer writing acceptance tests and sharing with the developers and testers before they began creating.
  • Automation involving functional test structure, web services and embedded testing
  • Exploratory testing to supplement the automated tests to find critical issues.
  • Don’t forget to Document… but only what is useful
  • Finding ways to keep customers involved in all types of testing, even if they are remote. Have UATs and end to end tests
  • Use lessons learnt during testing to Critique the product in order to drive the development in next iterations
Agile Testing Quadrants

Read Along- ‘Agile Testing’ Chapter-11

Nishi Grover Garg2 Comments

“Critiquing the Product Using Technology-Facing Tests”

  • Technology-facing tests that critique the product are more concerned with the non-functional aspects – deficiencies of the product from a technical point of view.
  • We describe requirements using a programming domain vocabulary. This is the main of Quadrant-4 of our Agile Testing Quadrants.
  • Customers simply assume that software will be designed to properly accommodate the potential load, at a reasonable rate of performance. It doesn’t always occur to them to verbalize those concerns.
  • Tools, whether home-grown or acquired, are essential to succeed with Quadrant 4 testing efforts.

“Many teams find that a good technical tester or toolsmith can take on many of these tasks.”

Take a second look at the skills that your team already posseses, and brainstorm about the types of “ility” testing that can be done with the resources you already have. If you need outside teams, plan for that in your release and iteration planning.

The information these (Quadrant-4) tests provide may result in new stories and tasks in areas such as changing the architecture for better scalability or implementing a system-wide security solution. Be sure to complete the feedback loop from tests that critique the product to tests that drive changes that will improve the non-functional aspects of the product.

When Do you Do it?

  • Technical stories can be written to address specific requirements.
  • Consider a separate row on your story board for tasks needed by the product as a whole.
  • Find a way to test them early in the project
  • Prioritize stories such that a steel thread or a thin slice is complete early, so that you can create a performance test that can be run and continued as you add more functionality.
  • The time to think about your non-functional tests is during release or theme planning.

The team should consider various types of “ility” testing including – Security, maintainability, Interoperability, Compatibility, Reliability and Installability – and should execute them at appropriate times.

Performance, Scalability, Stress and Load tests should be done from the beginning of the project.

Read Along- ‘Agile Testing’ Chapter-10

Nishi Grover Garg1 Comment

“Business-Facing Tests that Critique the Product”

  • Critiquing or evaluating the product is what business users or tester do when they assess and make judgement about the product.
  • These are the tests performed in Quadrant 3 of our Agile Testing Quadrants
  • It is difficult to automate Business facing tests that critique the product, because such testing relies on human intellect, experience, and insight.
  • You won’t have time to do any Quadrant 3 tests if you haven’t automated tests in Quadrants 1 and 2.
  • Evaluating or critiquing the product is about manipulating the system and trying to recreate the actual experience of end users.

Demonstrations

  • Show customers what you are developing early & often.
  • End-of-iteration demos are important to see what has been delivered and revise priorities
  • Rather than just waiting for end of sprint demos, use any opportunity to demonstrate changes as you go.
  • Choose a frequency of demos that works for your team. Informal demos can be more productive

Scenario Testing – Business users can help define plausible scenarios & workflows that can mimic end user behavior

Soap Opera Testing – Term coined by Hans Buwalsa (2003) can help the team understand business & user needs. Ask “What’s the worst thing that can happen, and how did it happen?”

Exploratory Testing

  • As an investigative tool, it is a critical supplement to the story tests and our automated regression suite.
  • Sophisticated, thoughtful approach to testing without a script, combining learning, test design and test execution

Usability Testing

There are 2 types of usability testing. The first is done up front by user experience folks, using tools such as wire frames to drive programming. These are part of Quadrant 2.

The second type talks about the kind of usability testing that critiques the product. We use tools such as User Personas and our Intuition to help us look at the product with the end user in mind.

API Testing

Instead of just thinking about testing interfaces, we can also look at APIs and consider attacking the problem in other ways and consider tools like simulators & emulators.

Testing Documentation

User manuals & online help need validation just as much as software. Your team may employ specialists like technical writers who create & verify documentation. The entire team is responsible for the quality of documentation.

Read Along- ‘Agile Testing’ Chapter-9

Nishi Grover Garg1 Comment

“Toolkit for Business-Facing Tests that Support the Team”

  • As agile development has gained in popularity, we have more and more tools to help us capture and use them to write executable tests.
  • Your strategy for selecting the tools you need should be based on your team’s skill set, the technology your application uses, your team’s automation priorities, time, and budget constraints. Your strategy should NOT be based on the latest and coolest tool in the market.
  • In agile, simple solutions are usually best.
    • Some tools that can help us illustrate desired behavior with examples, brainstorm potential implementations and ripple effects and create requirements we can turn into tests are—
      • Checklists
      • Mind maps
      • Spreadsheets
      • Mockups
      • Flow diagrams
      • Software based tools
  • A picture is worth a thousand words, even in agile teams. Mock-ups show the customer’s desires more clearly than a narrative possibly could. They provide a good focal point to discussing the desired code behavior.
    • Visuals such as flow diagrams and mind maps are good ways to describe an overview of a story’s implementation, especially if it is created by a group of customers, programmers, and testers.
    • Tools such as Fit (Framework for Integrated Tests) and FitNesse were designed to facilitate collaboration and communication between the customer and development teams.
    • Finding the right electronic tools is particularly vital for distributed teams (chat, screensharing, video conferencing, calling, task boards etc.)
    • Selenium, Watir and WebTest are some examples of many open source tools available for GUI testing.
  • Home-Brewed Test Automation Tools
    • Bret Pettichord (2004) coined the term ‘home-brewed’ for tools agile teams create to meet their own unique testing needs. This allows even more customisation than an open source tool. They provide a way for non technical customer team members to write tests that are actually executable by the automated tool. Home-brewed tools are tailored to their needs, designed to minimize the total cost of ownership and often built on top of existing open source tools.

The best tools in the world won’t help if you don’t use them wisely. Test tools might make it very easy to specify tests, but whether you are specifying the right tests at the right time is up to you.

Writing detailed test cases that communicate desired behavior is both art and science.

Whenever a test fails in Continuous Integration (CI) and build process, the team’s highest priority should be to get the build passing again. Everyone should stop what they are doing and make sure that the build goes ‘green’ again. Determine if a bug has been introduced, or if the test simply needs to be updated to accommodate intentionally changed behavior. Fix the problem, check it in, and make sure all tests pass!

Experiment – so that you can find the right level of detail and the right test design for each story.

  • Keep your tests current and maintainable through refactoring.
  • Not all code is testable using automation but work with programmers to find alternative solutions to your problems.
  • Manual test scenarios can also drive programming if you share them with the programmers early. The earlier you turn them into automated tests, the faster you will realise the benefit.

Start with a simple approach, see how it works, and build on it. The important thing is to get going writing business-facing tests to support the team as you develop your product.

Read Along- ‘Agile Testing’ Chapter-6

Nishi Grover Garg1 Comment

“The Purpose of Testing”

  • The Agile Testing Quadrants matrix helps testers ensure that they have considered all of the different types of tests that are needed in order to deliver value.

Quadrant-1

Unit tests verify functionality of a small subset of the system. Component tests verify the behaviour of a larger part such as a group of classes that provide some services. Unit & Component tests are automated and written in the same programming language as the application. They enable programmers to measure what Kent Beck has called the internal quality of their code.

Quadrant-2

  • Tests in Quadrant-2 support the work of the development team but at a higher level. These business-facing tests define external quality and the features that the customers want. They’re written in a way business experts can easily understand using the business domain language.
  • The quick feedback provided by Quadrant 1 and 2 automated tests, which run with every code change or addition, form the foundation of an agile team. These tests first guide the development of functionality and when automated, then provide a safety net to prevent refactoring and the introduction of new code from causing unexpected results.

“Appraising a software product involves both art and science.”

Quadrant-3

  • Quadrant-3 classifies the business-facing tests that exercise the working software to see if it doesn’t quite meet expectations or won’t stand up to the competition. They try to emulate the way a real user would work the application. This is manual testing that only a human can do…use our senses, our brains and our intuition to check whether the development team has delivered the business value required by the customer.
  • Exploratory testing is central to this quadrant.

Quadrant-4

  • Technology-facing tests in Quadrant-4 are intended to critique product characteristics such as performance, robustness and security.
  • Creating and running these tests might require the use of specialised tools and additional expertise.
  • Automation is mandatory for some efforts such as load and performance testing.
Agile Testing Quadrants

Technical Debt

  • Ward Cunningham coined the term “technical debt” in 1992, but we’ve certainly experienced it throughout our careers in software development.
  • By taking the time and applying resources and practices to keep technical debt to a minimum, a team will have time and resources to cover the testing needed to ensure a quality product. Applying agile principles to do a good job of each type of testing at each level will, in turn, minimize technical debt.
  • Each quadrant in the agile testing matrix plays a role in keeping technical debt to a manageable level.

The Agile Testing Quadrants provide a checklist to make sure you have covered all your testing bases. Examine the answers to questions such as:

  • Are we using unit & component tests to help find the right design for our application?
  • Do we have an automated build process?
  • Do our business-facing tests help us deliver a product that matches customer expectations?
  • Are we capturing the right examples of desired system behaviour?
  • Do we show prototypes of the UIs and reports to the users before we start coding them?
  • Do we budget enough time for exploratory testing?
  • Do we consider technological requirements like performance and security early enough?

Read Along- ‘Agile Testing’ Chapter-5

Nishi Grover Garg2 Comments

“Transitioning Typical Processes”

  • There are many processes in a typical project that don’t transition well to agile because they require heavyweight documentation or are an inherent part of the phased and gated process & require signoffs at the end of each stage.

“Metrics can be controversial”

  • Measurements such as cycle time that involve the whole team are more likely to drive you toward success than measures confined to isolated roles or groups.
  • Lean development looks for ways to delight customers, which ought to be the goal for all software development.
  • Metrics that measure milestones along a journey to achieve team goals are useful.

When you are trying to figure out what to measure, first understand what problem you are trying to solve. If your goals are measurable, the measurements you need to gather to track the metrics will be obvious.

  • Figure each metrics Return on Investment and decide whether to track or maintain it. Does the effort spent collecting it justify the value it delivers? Can it be easily communicated and understood? Do what works for your situation. Experiment with keeping a particular metric for a few sprints and evaluate whether it is paying off.

“Projects succeed when people are allowed to do their best work”

  • Defects tracking systems (DTS) are too often used as communication tools & entering unnecessary bugs can be wasteful. Focus on using DTS for the right reasons.
  • Whether your team decides to create a test plan or not, the planning should be done. Each project is different, so don’t expect that the same solution will fit all.
  • Regarding Audits, Processes & Models
    • Traditional quality processes & process improvement models like SAS 70 and CMMI standards can co-exist with agile.
    • Quality assurance teams in traditional development organisations are often tasked with providing information for auditors and ensuring compliance with audit requirements.
    • Examples include what testing has been performed on given software release or proving that different accounts reconcile.
    • Testers can be tasked with writing test plans to evaluate the effectiveness of control activities.
    • Work together with the compliance and internal audit teams to understand your team’s responsibilities.
  • If your organisation is using some kind of process model or quality standard, educate yourself about it and work with the appropriate specialists in your organisation.
  • Process improvement models and frameworks emphasize discipline and conformance to process.

“Standards simply enable you to measure your progress towards your goal”

  • Working with existing quality processes and models is one of the biggest cultural issues you may face as you transition to agile development. All of these changes are hard, but when your whole team gets involves, none are insurmountable.

Read Along- ‘Agile Testing’ Chapter-3

Nishi Grover Garg1 Comment

“Cultural Challenges”

This chapter is dedicated to talking about organisational problems with agile adoption, mostly from a cultural point of view- how people perceive changes, how they work, giving up control and also taking charge. It is a very comprehensive description of many problems we see on a daily basis at our work and in teams struggling with agile transformation.

Points to remember and Quotable Quotes

  • Agile teams are best suites for organisations that allow independent thinking.
  • Fear is a powerful emotion, and if not addressed, it can jeopardize the transition into agile
  • Testers who don’t change their approach to testing have a hard time working closely with the rest of the development team
  • If the organisation culture is to push towards release without caring for quality, the teams will face an uphill battle in working in agile
  • Companies where testers assume the role of ‘Quality Police’ will also have a challenge since teams will not buy-into the idea of building quality in, as they are accustomed to badgering it in later.
  • If your organisation focuses on learning, it will encourage continuous process improvement and will likely adopt agile much more quickly.
  • Testers need time and training, like everyone else learning to work in agile
  • To help testers adjust, you may need to bring in an experienced agile testing coach to act as a mentor and a teacher.

Agile focuses on working at a sustainable pace, all the time. In contrast to the ‘fast and furious’ testing done at the end of release cycles in traditional projects (often amounting to overtime). In agile, if overtime is required, it is an exception, and that too for the whole team and not just the testers.

  • In agile, the relationship between the customer and the development team is more a partnership than a vendor-supplier relationship.
  • Even if an entire company adopts agile, some teams make the transition more successfully than others.

About Introducing Change-

“Expect and Accept Chaos as you implement Agile Processes.”

Find the areas of most pain, determine what practices will solve the problem so that you can get some immediate progress out of the chaos.

  • The critical success factor is whether the team takes ownership and has the ability to customise its approach
  • Celebrate success- Acknowledgement is important if you want a change to stick.
  • Rather than managing the team’s activities at a low level, managers of agile teams focus on removing obstacles so that team members can do their best work

“Agile development might seem fast-paced, but the change can seem glacial”

“Beware of the Quality Police mentality— Be a collaborator, not an enforcer“

The highlight of this chapter for me was reading the ‘Testers Bill of Rights’

I had not heard about this before , so reading this was pretty cool, and for sure fundamental to any tester’s life. Check it out-

Read Along- ‘Agile Testing’ Chapter-1

Nishi Grover Garg1 Comment

“What is Agile Testing, Anyway?”

Points to remember and Quotable Quotes:

  • “Several core practices used by agile teams relate to testing.”
    • Programmers using TDD, code integration tests being written contribute to a successful project.
  • “Agile Testing just doesn’t mean testing on an agile project.”
    • Some testing approaches like exploratory testing are inherently agile, whether done in an agile project or not.
  • Testers are integral members of the customer team as well as development team
  • The best part of this chapter is Lisa and Janet’s wonderful stories on beginning with their first agile projects, and a realization by Janet’s co-worker, a developer in a team following XP on how they saw Janet’s contribution to the project.
  • “Testers don’t sit & wait for work; they get up and look for ways to contribute throughout the development cycle and beyond.”
  • Traditional vs Agile Testing
    • In Traditional approach – “Testing gets “squished” because coding takes longer than expected, and because teams get into a code-and-fix cycle at the end.
    • In Agile – “As an agile team member, you will need to be adaptive to the team’s needs
    • “Participants, tests, and tools need to be adaptive.”
  • “An Agile team is a wonderful place to be a tester”
  • The Whole-Team Approach –
    • “Everyone on an agile team gets “test-infected.”
    • “An agile team must possess all the skills needed to produce quality code that delivers the features required by the organization.”
    • “The whole team approach involves constant collaboration”
    • “On an agile team, anyone can ask for and receive help”
  • “The fact is, it’s all about quality – and if it’s not, we question whether it’s really an ‘agile’ team.”

Wanna Read along? Get your copy of the book at

https://www.flipkart.com/agile-testing/p/itmdytt85fzashud?pid=9788131730683

OR find a paperback or ebook version!

Read Along- ‘Agile Testing’ by Lisa Crispin & Janet Gregory

Nishi Grover GargLeave a comment

Book Read Blogs Series

I used to love books, reading was a fun and satisfying hobby for the introverted teen I was. But lately I may have gotten away from it for known and unknown reasons. I want to pursue the passion again and hold myself accountable too. So, this year I am starting a ‘Read Along’ series on my blog.

“Agile Testing” by Lisa Crispin & Janet Gregory

I will begin by reading the book I had bought last year. “Agile Testing” by Lisa & Janet is a coveted read for all agile enthusiasts & testers and is also featured in the best books for testers at https://continuoustesting.blog/2020/01/17/most-recommended-software-testing-books-to-read-in-2020-and-beyond/

I have learnt agile testing by doing it, learning it hands-on, training & running courses on agile testing for professionals. I wanted to enhance my knowledge by reading the professional work by these awesome ladies.

So, I will be reading the book and will post about learnings, things to remember & quotable quotes from each chapter as I progress. This is to hold myself accountable, as well as to help people looking for good reads or learnings. Hope this helps you. Have you read this book? Do share your thoughts & learnings too!

Here are the links to the Chapter-wise posts for this book-

Chapter-1 What is Agile Testing Anyway

Chapter-2 The Principles for Agile Testers

Chapter-3 Cultural Challenges

Chapter-4 Team Logistics

Chapter-5 Transitioning Typical Processes

Chapter-6. The purpose of Testing

Chapter-7 Technology Facing Tests that support the team

Chapter-8 Business Facing Tests that support the team

Chapter-9 Toolkit for Business facing tests that support the team

Chapter-10 Business facing tests that Critique the product

Chapter-11 Critiquing the product using Technology facing tests

Chapter-12 Summary of Testing Quadrants

Chapter-13 Why we want to automate tests and what holds us back?

Chapter-14 An Agile Test Automation Strategy

Chapter-15 An Iteration in the Life of a Tester

Chapter-16 Hit the Ground Running

Chapter-17 Iteration Kickoff

Chapter-18 Coding & Testing

Chapter-19 Wrap up the Iteration

Chapter-20 Successful Delivery

Chapter-21 Key Success Factors

Hope this Read Along Series offers some meaningful insights into this wonderful book and some quick pointers for those looking to get started or get more proficient in their agile testing ways!

Happy Testing!

Nishi