How XP Solves Testing and Quality Assurance Problems

That's XP in a nutshell: a collection of simple practices, taken to an extreme. Perhaps the most extreme thing of all about XP teams is the discipline with which they follow the practices. For instance, a 100% pass rate on unit tests doesn't mean 90% when the project is behind schedule. It means 100%, all the time.

So how can something as seemingly simple as this solve testing and quality assurance problems? Well, if you look closely, you can see that some of these practices relate directly to themes that testing and quality assurance professionals have been preaching for years.

Take code inspections, for instance, which many QA departments have tried to get into general practice as a more cost-effective way to identify defects. On an XP project, pair programming of production code means that 100% of all code is inspected when written. Or take the idea of "test first" reviewing the test cases before writing the code, which testing professionals have promoted for years as a way of avoiding defects in the first place. Programmers on XP teams write their unit tests first, before they write the code.

These ideas really work, just as the quality assurance and testing professionals said they would. Unlike most traditional software projects, XP projects actually apply these ideas diligently and consistently, instead of just paying lip service. Here are some specific problems the XP approach can solve:

System and Acceptance Testing Resources Wasted on Unit- and Integration-Level Bugs

One common problem testers experience on traditional software projects arises when they try to run system-level tests on code that's never been tested at the unit or integration level. This is an expensive and inefficient way to locate unit- and integration-level bugs and can easily consume all the time allocated for system and acceptance testing. Either the software is delivered late or the system- and acceptance-level testing is never completed, and the software suffers significant quality problems in operation.

This is especially frustrating to testers, because they really can't do anything about it. No matter how thorough, efficient, and/or automated are the tests they develop, they can't put either functionality or quality into the system; they can only detect its presence or absence.

XP avoids this scenario completely through the practices of 100% unit-test automation and continuous integration. Programmer pairs detect and correct unit and integration bugs during the coding sessions. By the time the code gets to acceptance testing, it's doing what the programmers intend, and the acceptance tests can focus on determining how well that intent matches the customer's expectations.

Missing and Out-of-Date Requirements

Moving upstream in the process, another common problem in developing tests occurs when requirements are missing or out of date. The heavyweight, detailed requirement documents favored by traditional software development are ideal for use in developing tests but frequently turn out to be too expensive to create and maintain, and they leave the tester with no alternative when they inevitably become obsolete.

Testers find this frustrating, because they know the tests developed from the out-of-date requirements are inaccurate, but they have nothing else to go on except nonbinding verbal communication. Even getting that can be difficult, because those in the know will resent having to spend more time on questions after having already spent way too much time on the heavyweight documents in the first place.

XP solves this problem by replacing the massive requirements document with lightweight user stories that are easily maintained, augmented by the onsite customer, who functions as the oracle for all details the stories don't cover.

Huge Gaps between the System and User Expectations

Finally, one of the most painful things for a tester or quality assurance person to experience is when a significant gap develops between what a project is delivering and what the customer is expecting. This can easily happen in traditional software development, where the customer is out of the loop after the requirements phase, and months or even years can go by before any functioning code is delivered. Pointing out the gap can cause the tester to be viewed as having a negative attitude, not being a team player, and, eventually, as "the enemy." Not pointing it out leads to a significantly unhappy customer and also reflects badly on the tester (e.g., "Why wasn't this caught in testing?").

XP eliminates this scenario by producing frequent, small releases that provide exactly those features the customer has most recently prioritized as the most important. If a gap starts to develop, it naturally becomes visible within the space of an iteration (two to three weeks at the most) and is dealt with by the entire team.

These three problems lack of adequate unit and integration testing, missing and out-of-date requirements, and huge gaps between customer expectations and delivered product are often symptoms of more deeply seated problems in the enterprise. We're not saying XP is a solution for those root causes. We're saying that if you're a testing/QA professional and can get involved with a team practicing XP, these problems will be minimized. We've found that these are major factors in determining the quality of life for professional testers.