Tag: book read

Read Along- ‘Agile Testing’ Chapter-11

Nishi Grover Garg2 Comments

“Critiquing the Product Using Technology-Facing Tests”

  • Technology-facing tests that critique the product are more concerned with the non-functional aspects – deficiencies of the product from a technical point of view.
  • We describe requirements using a programming domain vocabulary. This is the main of Quadrant-4 of our Agile Testing Quadrants.
  • Customers simply assume that software will be designed to properly accommodate the potential load, at a reasonable rate of performance. It doesn’t always occur to them to verbalize those concerns.
  • Tools, whether home-grown or acquired, are essential to succeed with Quadrant 4 testing efforts.

“Many teams find that a good technical tester or toolsmith can take on many of these tasks.”

Take a second look at the skills that your team already posseses, and brainstorm about the types of “ility” testing that can be done with the resources you already have. If you need outside teams, plan for that in your release and iteration planning.

The information these (Quadrant-4) tests provide may result in new stories and tasks in areas such as changing the architecture for better scalability or implementing a system-wide security solution. Be sure to complete the feedback loop from tests that critique the product to tests that drive changes that will improve the non-functional aspects of the product.

When Do you Do it?

  • Technical stories can be written to address specific requirements.
  • Consider a separate row on your story board for tasks needed by the product as a whole.
  • Find a way to test them early in the project
  • Prioritize stories such that a steel thread or a thin slice is complete early, so that you can create a performance test that can be run and continued as you add more functionality.
  • The time to think about your non-functional tests is during release or theme planning.

The team should consider various types of “ility” testing including – Security, maintainability, Interoperability, Compatibility, Reliability and Installability – and should execute them at appropriate times.

Performance, Scalability, Stress and Load tests should be done from the beginning of the project.

Read Along- ‘Agile Testing’ Chapter-10

Nishi Grover Garg1 Comment

“Business-Facing Tests that Critique the Product”

  • Critiquing or evaluating the product is what business users or tester do when they assess and make judgement about the product.
  • These are the tests performed in Quadrant 3 of our Agile Testing Quadrants
  • It is difficult to automate Business facing tests that critique the product, because such testing relies on human intellect, experience, and insight.
  • You won’t have time to do any Quadrant 3 tests if you haven’t automated tests in Quadrants 1 and 2.
  • Evaluating or critiquing the product is about manipulating the system and trying to recreate the actual experience of end users.

Demonstrations

  • Show customers what you are developing early & often.
  • End-of-iteration demos are important to see what has been delivered and revise priorities
  • Rather than just waiting for end of sprint demos, use any opportunity to demonstrate changes as you go.
  • Choose a frequency of demos that works for your team. Informal demos can be more productive

Scenario Testing – Business users can help define plausible scenarios & workflows that can mimic end user behavior

Soap Opera Testing – Term coined by Hans Buwalsa (2003) can help the team understand business & user needs. Ask “What’s the worst thing that can happen, and how did it happen?”

Exploratory Testing

  • As an investigative tool, it is a critical supplement to the story tests and our automated regression suite.
  • Sophisticated, thoughtful approach to testing without a script, combining learning, test design and test execution

Usability Testing

There are 2 types of usability testing. The first is done up front by user experience folks, using tools such as wire frames to drive programming. These are part of Quadrant 2.

The second type talks about the kind of usability testing that critiques the product. We use tools such as User Personas and our Intuition to help us look at the product with the end user in mind.

API Testing

Instead of just thinking about testing interfaces, we can also look at APIs and consider attacking the problem in other ways and consider tools like simulators & emulators.

Testing Documentation

User manuals & online help need validation just as much as software. Your team may employ specialists like technical writers who create & verify documentation. The entire team is responsible for the quality of documentation.

Read Along- ‘Agile Testing’ Chapter-9

Nishi Grover Garg1 Comment

“Toolkit for Business-Facing Tests that Support the Team”

  • As agile development has gained in popularity, we have more and more tools to help us capture and use them to write executable tests.
  • Your strategy for selecting the tools you need should be based on your team’s skill set, the technology your application uses, your team’s automation priorities, time, and budget constraints. Your strategy should NOT be based on the latest and coolest tool in the market.
  • In agile, simple solutions are usually best.
    • Some tools that can help us illustrate desired behavior with examples, brainstorm potential implementations and ripple effects and create requirements we can turn into tests are—
      • Checklists
      • Mind maps
      • Spreadsheets
      • Mockups
      • Flow diagrams
      • Software based tools
  • A picture is worth a thousand words, even in agile teams. Mock-ups show the customer’s desires more clearly than a narrative possibly could. They provide a good focal point to discussing the desired code behavior.
    • Visuals such as flow diagrams and mind maps are good ways to describe an overview of a story’s implementation, especially if it is created by a group of customers, programmers, and testers.
    • Tools such as Fit (Framework for Integrated Tests) and FitNesse were designed to facilitate collaboration and communication between the customer and development teams.
    • Finding the right electronic tools is particularly vital for distributed teams (chat, screensharing, video conferencing, calling, task boards etc.)
    • Selenium, Watir and WebTest are some examples of many open source tools available for GUI testing.
  • Home-Brewed Test Automation Tools
    • Bret Pettichord (2004) coined the term ‘home-brewed’ for tools agile teams create to meet their own unique testing needs. This allows even more customisation than an open source tool. They provide a way for non technical customer team members to write tests that are actually executable by the automated tool. Home-brewed tools are tailored to their needs, designed to minimize the total cost of ownership and often built on top of existing open source tools.

The best tools in the world won’t help if you don’t use them wisely. Test tools might make it very easy to specify tests, but whether you are specifying the right tests at the right time is up to you.

Writing detailed test cases that communicate desired behavior is both art and science.

Whenever a test fails in Continuous Integration (CI) and build process, the team’s highest priority should be to get the build passing again. Everyone should stop what they are doing and make sure that the build goes ‘green’ again. Determine if a bug has been introduced, or if the test simply needs to be updated to accommodate intentionally changed behavior. Fix the problem, check it in, and make sure all tests pass!

Experiment – so that you can find the right level of detail and the right test design for each story.

  • Keep your tests current and maintainable through refactoring.
  • Not all code is testable using automation but work with programmers to find alternative solutions to your problems.
  • Manual test scenarios can also drive programming if you share them with the programmers early. The earlier you turn them into automated tests, the faster you will realise the benefit.

Start with a simple approach, see how it works, and build on it. The important thing is to get going writing business-facing tests to support the team as you develop your product.

Read Along- ‘Agile Testing’ Chapter-8

Nishi Grover Garg2 Comments

“Business-Facing Tests that Support the Team”

A look at tests in Quadrant-2 – Business-Facing tests

Agile Testing Quadrants
  • On an agile project, the customer team and the development team strike up a conversation based on a user story.
  • Business-facing tests address business requirements. They express requirements based on examples and use a language and format that both the customer and development teams can understand. Examples form the basis of learning the desired behavior of each feature and we use those examples as the basis of our story tests in Quadrant-2
  • Business-facing tests are also called “customer-facing”,”story”,”customer” and “acceptance” tests. The term ‘acceptance tests’ should not be confused with ‘user acceptance tests’ from Quadrant-3.
  • The business-facing tests in Q-2 are written for each story before coding started, because they help the team understand what code to write.
    • Quadrant-1 activities ensure internal quality, maximize team productivity, and minimize technical debt.
    • Quadrant-2 tests define and verify external quality and help us know when we are done.

The customer tests to drive coding are generally written in executable format, and automated, so that team members can run the tests as often as they like to see if functionality works as desired.

  • Tests need to include more than the customer’s stated requirements. We need to test for post-conditions, impact on the system as a whole, and integration with other systems. We identify risks and mitigate those with our tests. All of these factors then guide our coding.
  • The tests need to be written in a way that is comprehensible to a business user yet still executable by the technical team.
  • Getting requirements right is an area where team members in many different roles can jump in to help.
  • We often forget about non-functional requirements. Testing for them may be a part of Quadrants 3 and 4, but we still need to write tests to make sure they get done.

There are conditions of satisfaction for the whole team as well as for each feature or story. They generally come out of conversations with the customer about high-level acceptance criteria for each story. They also help identify risky assumptions and increases team’s confidence in writing & correctly estimating tasks needed to complete the story.

  • A smart incremental approach to writing customer tests that guide development is to start with a “thing-slice” that follows a happy path from one end to the other. (also called a “steel-thread” or “tracer-bullet”). This ‘steel-thread’ connects all of the components together and after it’s solid, more functionality can be added.
  • After the thin slice is working, we can write customer tests for the next chunk.
    • It’s a process of  “write tests — write code— run tests — learn”
  • Another goal of customer tests is to identify high-risk areas and make sure code is written to solidify those.
  • Experiment & find ways your team can balance using up-front detail and keeping focused on the big picture.

Quadrant-2 contains a lot of different types of tests and activities. We need the right tools to facilitate gathering, discussing, and communicating examples and tests.

>>Simple tools such as Paper or Whiteboard work well for gathering examples if the team is co-located.

>>More sophisticated tools help teams write business-facing tests that guide development in an executable, automatable format.

Read Along- ‘Agile Testing’ Chapter-7

Nishi Grover Garg1 Comment

“Technology-Facing Tests that Support the Team”

A look at tests in Quadrant-1 – Technology Facing tests

Agile Testing Quadrants
  • Unit tests and component tests ensure quality by helping the programmers understand exactly what the code needs to do and providing guidance in the right design
  • The term ‘Test-Driven Development’ misleads practitioners who do not understand that its more about design than testing. Code developed test-first is naturally designed for Testability.
  • When teams practice TDD, they minimize the number of bugs that must be caught later.

The more bugs that leak out of our coding process, the slower our delivery will be, and in the end, it is the quality that will suffer. That’s why programmer tests in Quadrant-1 are so critical. A team without these core agile practices is unlikely to benefit much from agile values and principles.

  • Source Code Control, Configuration Management and Continuous Integration are essential to getting value from programmer tests that guide development.
  • CI saves time and motivates each programmer to run the tests before checking in the new code.
  • An advantage of driving development with tests is that code is written with the express intention of making tests pass.
  • A common approach in designing a testable architecture is to separate the different layers that perform different functions in the application.

Teams should take time to consider how they can take time to create an architecture that will make automated tests easier to create, inexpensive to maintain and long-lived. Don’t be afraid to revisit the architecture is automated tests don’t return value for the investment in them.

“The biggest value of unit tests is in the speed of their feedback.”

  • Each unit test is different and tests one dimension at a time
  • Learning to write Quadrant-1 tests is hard.
  • Because TDD is really more of a design activity, it is essential that the person writing the code also writes the tests, before writing the code.
  • To Managers—
    • If a delivery date is in jeopardy, push to reduce the scope, not the quality.
    • Give the team time to learn and provide expert, hands-on training.
  • Technology-facing tests cannot be done without the right tools and infrastructure

Read Along- ‘Agile Testing’ Chapter-6

Nishi Grover Garg1 Comment

“The Purpose of Testing”

  • The Agile Testing Quadrants matrix helps testers ensure that they have considered all of the different types of tests that are needed in order to deliver value.

Quadrant-1

Unit tests verify functionality of a small subset of the system. Component tests verify the behaviour of a larger part such as a group of classes that provide some services. Unit & Component tests are automated and written in the same programming language as the application. They enable programmers to measure what Kent Beck has called the internal quality of their code.

Quadrant-2

  • Tests in Quadrant-2 support the work of the development team but at a higher level. These business-facing tests define external quality and the features that the customers want. They’re written in a way business experts can easily understand using the business domain language.
  • The quick feedback provided by Quadrant 1 and 2 automated tests, which run with every code change or addition, form the foundation of an agile team. These tests first guide the development of functionality and when automated, then provide a safety net to prevent refactoring and the introduction of new code from causing unexpected results.

“Appraising a software product involves both art and science.”

Quadrant-3

  • Quadrant-3 classifies the business-facing tests that exercise the working software to see if it doesn’t quite meet expectations or won’t stand up to the competition. They try to emulate the way a real user would work the application. This is manual testing that only a human can do…use our senses, our brains and our intuition to check whether the development team has delivered the business value required by the customer.
  • Exploratory testing is central to this quadrant.

Quadrant-4

  • Technology-facing tests in Quadrant-4 are intended to critique product characteristics such as performance, robustness and security.
  • Creating and running these tests might require the use of specialised tools and additional expertise.
  • Automation is mandatory for some efforts such as load and performance testing.
Agile Testing Quadrants

Technical Debt

  • Ward Cunningham coined the term “technical debt” in 1992, but we’ve certainly experienced it throughout our careers in software development.
  • By taking the time and applying resources and practices to keep technical debt to a minimum, a team will have time and resources to cover the testing needed to ensure a quality product. Applying agile principles to do a good job of each type of testing at each level will, in turn, minimize technical debt.
  • Each quadrant in the agile testing matrix plays a role in keeping technical debt to a manageable level.

The Agile Testing Quadrants provide a checklist to make sure you have covered all your testing bases. Examine the answers to questions such as:

  • Are we using unit & component tests to help find the right design for our application?
  • Do we have an automated build process?
  • Do our business-facing tests help us deliver a product that matches customer expectations?
  • Are we capturing the right examples of desired system behaviour?
  • Do we show prototypes of the UIs and reports to the users before we start coding them?
  • Do we budget enough time for exploratory testing?
  • Do we consider technological requirements like performance and security early enough?