Let Worry Be Your Guide


The following interchange is from a conversation on the agile-testing mailing list postings in December 2001. The discussion centered around letting the customer decide the level of quality for a software package. XP gives the customer responsibility for influencing quality through acceptance tests. No matter what software development process you're using, the customer is still responsible for defining what quality is for a software package under development.

Mike Clark contributed this scenario:

Let's say our product is supported to run on 2 application servers, each having three different versions. Each application server version is supported on three different operating systems. In the interest of time and resources, we've been testing these configurations using a pair-wise strategy in general. We've never shipped a defect related to running the product on application server A.x on any operations system. We're confident in the product's support for these permutations, so we only pair-wise test our product with application server A.1 using operating system A. In other words, the application server version and operating system are arbitrary, and we don't test all the permutations. The customer makes a conscious decision to accept the risk.

On the other hand, we know that we've shipped more defects over time related to the running of the product on the combination of application server B.x and operating system C. Perhaps there's something subtle about this combination that we don't understand, so we ensure that we always devote resources to thoroughly testing this specific permutation. The customer makes a conscious decision to reduce the risk.

In these cases, the customer knows that she's getting a reduction of risk for her money if she chooses to test a specific permutation. Testing all permutations is ideal, but probably not necessary or feasible. The product may ship sooner by relaxing the quality constraints for some configurations, without deliberately sacrificing quality. Nevertheless, it's a business decision, so I'll let the customer choose the testing level, and thereby influence the quality.

Brian Marick responds:

I suggest that the metaphor "setting a quality level" isn't a good one. It makes the process seem precise, quantitative, and product-wide. To me, it seems rough and uncertain, qualitative, and focused on particular risks.

(I have no objection to things that are rough, uncertain, or qualitative, by the way.)

Here's an alternate slogan: "assigning worry tasks." Because that doesn't mean anything obvious, you'd always be forced to explain it the first time you use it. (I think that's an advantage.) Here's an explanation.

There's someone responsible for spending money wisely. In XP, that's the Customer. In a mass-market project, it would be the project manager (or the combination of the project and product manager). I'll call that person Bob.

Bob says two things.

  1. "Make X work." The programmers then spring into action. In XP, they implement a story. In other development styles, they implement some chunk of a specification or list of requirements.

  2. "I'm worried about Y." The programmers and testers spring into action. The testers check if that worry is justified. The programmers do things to make it not justified. Both of them calibrate their effort by discovering how worried Bob is.

How much Bob delegates worry tasks depends on the project and Bob. For example, every time Bob says, "Make X work," he probably worries about whether X really will.

  • In some XP projects, Bob would create the customer tests that would persuade him to stop worrying.

  • In others, Bob talks with a tester about X. She creates the tests for him.

  • In the mass-market projects I've seen, Bob staffs the project with testers, based on his experience of what proportion of testers to code is required to achieve good enough quality, then lets them decide how to test each X. They'll let him know if something comes up that requires he make a business decision.



Testing Extreme Programming
Testing Extreme Programming
ISBN: 0321113551
EAN: 2147483647
Year: 2005
Pages: 238

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net