Defining Quality

To define quality, you have to ask the customer team lots of questions during the planning game. What is quality for this story? How will we know it when we see it? What one story in the iteration, or one part of a given story, absolutely must work? It's often easier to think about it the other way: "What's the worst thing that could happen here?"

For example, if you're developing a retail Web site, the ability to make a purchase may be the critical function. To go further, perhaps correctly validating the credit card is the most business-critical item on the purchase screen. These exercises help the customer prioritize stories and help you design tests and focus resources. This information will go into the acceptance tests to help the programmers estimate and plan each iteration and to ultimately meet customer expectations for completed stories.

What happens when you ask the customer questions for which he has no answers? "We don't really know which operating systems or browser versions should be supported." "Gee, we aren't sure how many concurrent users will log on to the system." The customer may not be familiar with the technical aspects of her application. Or perhaps you're developing a new product and the customer has no historical data to fall back on.

Make sure you're asking questions for which the answers really do matter. Although this may seem obvious, it's sometimes easy to get carried away with all kinds of scenarios that are just never going to happen. Assuming they do matter, help the customer find answers! However hard it is to come up with some kind of answer now, it can be much harder later to undo the result of a bad assumption. Even if you can't determine the answer at this point, at least everyone will know it's an open question and not make assumptions about it.

In XP, the customer's role is to make business decisions, not to be a quality expert. Face it, some people are always on the "happy path" (not looking for problems)…just as Lisa was when she signed a contract with a builder for a home addition. And although the customer may be conscious of budgetary limitations, he may not articulate them in a way programmers understand. Also, if he's not technically savvy, it might not occur to him that he can't have features A through Z plus 99.995% uptime in three iterations.

While the customer is responsible for coming up with quality criteria, the development team can help. Programmers can research and present several alternatives to the customer. For example, say you're developing a system where users are updating records in a database. If two users try to update the same record concurrently, a deadlock may occur. Your team can give the user three different ways to deal with the problem. The least expensive creates the risk of a race, while the most expensive provides record locking, to prevent such a condition. Now the customer can weigh the cost against the risk and decide what to do this iteration. Later, when other high-priority features have been taken care of, she can come back and implement a more expensive alternative if she wishes.

We've sometimes had customers who were passive and unsure of exactly what they wanted in terms of quality. Sometimes they don't have ready answers to questions like "How many concurrent transactions does this application need to support?" Your team needs full information from the customer to make progress. With your focus on quality, you can check to make sure the customer proactively defines the quality criteria. After all, he's usually paying the bills.

It's hard to build up trust between the customer and the development team, and easy to lose it. It's better for everyone to be clear on the standards for quality up front and make accurate estimates than to disappoint the customer with an unpleasant surprise when the end of an iteration or release rolls around.