Agile Quality Assurance

Filed under [ Agile Quality ]

by Bill Rinko-Gay

When I first started researching Agile Quality Assurance I was told it wasn’t necessary because Agile methods have quality built in.  Agile teams developed processes such as Extreme Programming (XP) and Test Driven Development (TDD) to produce quality code in a short amount of time.  Because these processes work well, it is believed Agile teams don’t need Quality Assurance.  I disagree.  I have studied and implemented Agile methods based on over 28 years of practical test and quality assurance experience. My viewpoint gives me a different perspective on Agile and Quality Assurance.

Developers have a specific expertise and viewpoint.  Early Agile methods focused on development tools and practices because that’s what developers could control.  The work they did and the results they achieved are good.  They showed quality code can be developed in a short amount of time.  But there’s more to running a development project than just writing the code.  That’s why project management methods have been developed to compliment the development methods.  Now we need to adopt new methods of process evaluation and independent testing that can add value to Agile teams – Agile Quality Assurance.

As with anything Agile, I’m not saying that Agile teams need to add one or more QA members to the team.  I’m defining principles and activities that provide QA.  How the team organizes to implement those principles and accomplish those activities is up to the team.

I intend to write about individual subjects in the future, but here are some key principles involved in Agile Quality Assurance:

  • Quality software is software that makes the customer happy.  You have to build the right thing, and you have to build it right.
  • The only way to assure quality software is to build quality in.  Good quality assurance focuses on how to build quality in.  Finding and removing bugs is quality control.
  • It is easier to build quality software and verify it is right if you develop code in small increments.
  • Development and Test is a single activity, not two separate activities

Key roles in Agile Quality Assurance are:

  • Making sure the features, as designed, will meet the need (building the right thing)
  • Helping to find the technical solution with the highest quality
  • Analyzing the process to identify and remove barriers to building quality in
  • Testing the software to verify the process is working, and that the software is built right

In future posts we’ll look at these principles and activities, and some of the team behaviors that flow from them.

Open Book Testing

Filed under [ Agile Quality ]

by Bill Rinko-Gay

There are two components of quality software: building the right thing and building the thing right.  One of the most difficult things to get right is knowing what the right thing is.  In non-Agile methods this puts the focus on defining complete and detailed requirements.  Experience shows that it is very difficult, if not impossible, to specify exactly what you want when you haven’t seen or worked with it yet.

In the absence of complete and detailed requirements we have to build software with a knowledge of what that software must do to satisfy the user.  This knowledge has to be shared by the entire Agile team.  We start to share this knowledge with a user story, but that only gives part of the picture.  The rest of the picture is given in implementation and test plans.

When we share our plans for testing with the user and with the developer we share a statement of what the right thing is.  This statement, developed correctly, becomes an excellent tool to understand what we have to build.  Having this plan before development begins is an important part of assuring we build the right thing.

We start by writing the test story.  In traditional testing language this would be called a test scenario.  It’s a statement of the contents of the test without being hampered by the execution details.  Written properly it allows the user to say, “Yes, if it passes these tests, it will be what I want.”

Each user story should have a test story.  The test story should be written as soon as it can be after the user story has been accepted.  The QA analyst needs to be involved in the user story development so he can have the knowledge to develop the test story.  When written, the test story should contain the following elements:

  • A specific validation of each acceptance criterion in the user story
  • A statement of each behavior being demonstrated
  • For each behavior:
    • A brief description of how it will be verified
    • Any special data setup that will be required
    • Any specific system setup that will be required
    • Success criteria

Once the user has verified the test story matches the expected use of the system, the developer has a target.  The developer can be sure if he hits the target, he has built the right thing.

In the next blog I will talk about the process of refining this test story to the appropriate level for the Agile team and planning out its execution.

Teaching to the Test

Filed under [ Agile Quality ]

by Bill Rinko-Gay

This blog is a follow-on to the post entitled “Open Book Testing.” If you haven’t already, you may want to read that first.

Once you have reviewed all your test stories it’s time to prepare to execute them.  A well planned test execution, in conjunction with a well planned code implementation, helps to actually build quality in as well as verify the quality of the completed software.

To begin with, you need to understand how the users will interact with the program.  Remember, the developers will be focused on making features work properly.  You need to focus on software that meets the user’s needs.  As an example,  I once did a comparison of two circuit board design packages.  Technically, one product (I’ll call it A) was far superior to the other (B), but A was much harder to use.  A would give the users far less clean up work after the auto-placement.  B left more clean up work, but all the work would be done in less time because of the ease of use.  The result was that we bought B.  The poor usability decisions cost company A money.

As you break your test stories down into discrete steps you will be the one looking at how the package works as a whole.  This is the time to share with your developers whether features are usable.   Walk through the steps with them and help them understand what it will be like for the users to use the software.  In the case of my example, this meant showing the difficulty in moving an integrated circuit and re-routing its traces, and why that made an excellent initial placement algorithm unimportant.

NOTE: You may have become used to using poorly designed software because of a key feature.  Don’t fall into the trap of assuming that’s acceptable.  If a trade-off must be made, it should be an explicit trade-off with the user’s needs in mind.

In addition to verifying usability, work with the developers to plan the development and test effort in groups of stories that make sense.  A common test problem is that releases that change what’s already been tested create “test thrash”.  I’ll focus on minimizing this test thrash in my next blog.

By working through your test steps during the implementation you help your developers create a good user experience from the beginning, rather than trying to patch it in the end.  This is a powerful contributor to building quality in.

Functional Grouping

Filed under [ Agile Quality ]

by Bill Rinko-Gay

Now that your stories are chosen and the iteration has been kicked off, is there any reason not to have developers simply start in as quickly as they can on writing the code for the stories? I believe there are 3 reasons to set up a plan for delivering groups of stories to test. First, functional grouping allows for higher quality code delivered faster. Second, functional grouping makes the testing job far easier. Third, functional grouping allows the team to re-plan an incomplete iteration. Let’s look at each of these claims in detail.

Quality: An iteration will always have stories that impact different functional areas. If you functionally group your stories you can consider the potential impacts one may have on another. Planning the development with these impacts in mind is likely to get the development done more quickly. Since the development will be more coherent from the start, it is likely to be of higher quality. Of course, you can achieve that same level of quality when taking the stories piecemeal by re-designing and re-factoring to account for the interactions. This is what you have to do when the stories fall in different iterations. Re-designing and re-factoring takes time, so it benefits the team to avoid it whenever possible.

Testing: Functional Grouping aids the testers because they can focus their efforts on a single functional area at a time. When that area is complete its manual testing will not have to be revisited, reducing test thrash. Completing a functional group and then turning it over to testing means that functionality can be finished when testing is completed. The software moves forward without the delays caused by going two steps forward and one step back.

Re-planning: Reasonably stable release candidates are made ready for test at the completion of each functional group. The team may have some other partially completed stories in the build, but the completed functionality can be the basis for a release if the team runs behind in finishing the iteration. If there have been no stable builds prepared for test and you decide to cut back on the stories in the iteration, you risk destabilizing the code. If you have one of these test builds to fall back on, you will have an easier time limiting the release to the smaller group of stories.

The functional groups should be worked on in priority order. In some cases the priority may be set by technical risk. Otherwise they are set by the business. The planning should be based on the question, if the iteration could only deliver on a subset of the stories planned, which is the most important subset.

Efficient Dev-Test

Filed under [ Agile Quality ]

by Bill Rinko-Gay

One of the advantages of Agile is the ability to deliver continuous improvements to your customer.  If you have experience in traditional development paradigms you a release is usually a mixed bag filled with new functionality and new (and sometimes old) defects.  This is frustrating for users and the development team.  You seem to always be under a backlog of bugs that keeps the software from delivering all the business value you intended.

This constant backlog doesn’t just occur in traditional development, it can occur on Agile projects as well.  The hurry to try to reduce the backlog by fixing last-minute bugs leads to thrash, and thrash can lead to reduced quality as hurry cause the team to introduce new bugs.   Your goal is to move always forward with your code, but the reality is often “two steps forward, one step back.”

One way to avoid this problem is to think through the activities as a single phase: “dev-test.”  When the work is ready to start on an iteration the developers and testers map out how the software is going to be changed.  The team should plan to bundle stories into builds.  The build strategy should maximize the team’s efficiency while delivering continuously increasing functionality at high quality.  The more compact the changes between builds, the easier it is to verify those changes are made correctly.  Unfortunately, this isn’t easy.  Making this possible sometimes requires extra work.  Still, the payoff is worth the effort.

As a specific example, suppose all the stories in a particular iteration require a new save functionality.  Also, suppose the save functionality should be written last because developers need to see the other stories’ implementation before writing the save function.  Rather than delivering all the functionality at once, the developers could write a temporary save function that would allow the early stories to be tested.  This is “wasted” work for the developers, but it increases the efficiency of the team because they can get fast feedback on the early stories and get any defects fixed before the real save function is implemented.  If the other stories are tested complete, then problems caused by the save function are isolated.  Achieving these goals would not be possible without developers accepting a little less efficiency.

Similarly, suppose the GUI changes are the least risk for introducing bugs, leading the team to agree to write the GUI last.  This means the testers will need to re-visit all the GUI functionality after testing the other stories.  If the GUI stories had been written first, the testers would not have to repeat tests at the end to exercise those GUI functions, but the developers would not get fast failure on more risk-prone stories.  In this case testers agree to a less efficient test plan to improve the efficiency of the developers.

The advantage of doing multiple releases with small increments of functionality is the reduced time to test each incremental build leading to higher overall quality.  Development and test are both more efficient, and the customer is much happier.

Older Posts »