The XP Tester as Quality Assurance Engineer

The XP books say that the customer writes the test. In Extreme Programming Explained, Kent Beck says that customers need to ask themselves, "What would have to be checked before I would be confident this story was done?" This very question implies tests that check for intended functionality, or what my boss calls "happy path" testing.

Beck goes on to say that XP teams should have a dedicated tester who "uses the customer-inspired tests as the starting point for variations that are likely to break the software." This implies that the tester should guide the customer in defining tests that will really stress the application. He also mentions "stress" and "monkey" tests designed to zero in on unpredictable results.

In practice, when I neglected to negotiate quality with a customer, acceptance testing became as treacherous as the mud pit that currently surrounds the new wing of my house. I wrote and performed acceptance tests according to my own standard of quality. Naturally, the tests, particularly the load tests and "monkey" tests, uncovered issues. To the XP-naive customer, these just look like bugs, and they're upsetting. The customer starts to worry that their stories aren't really being completed.

The XP way to deal with any kind of issue or defect is to turn them into stories, estimate them, and let the customer choose them for subsequent iterations. We know that some defects and unexpected issues will always crop up. However, to minimize the pain of dealing with these, it's best to set the criteria for quality at the start of each iteration.