Test Phases and Practices

Writing, automating, and performing acceptance tests during the iteration are fundamental practices of XP. Are you going to throw all your existing testing process away when you start using Extreme Programming? You need to work hard to master all the XP practices as soon as possible, but at the same time, scrapping all your existing testing process may not be the best idea.

Consider each process and the reason for it. If you still need a particular testing phase or practice, keep it. Here are a couple of examples. Your development team should be able to test in an environment that looks just like production, but if the reality is that company resources are limited and several projects must share test machines, you may be forced to test in a more complete environment after the end of development. Or say you're releasing a new version of a retail Web site and you want to allow a group of customers to give the site a dry run, to make sure order fulfillment works correctly. In cases like these, it doesn't make sense to drop additional test phases after development.

Even if you don't have a technical reason to preserve a practice such as post-development testing, if your test organization or upper management is worried about the radical change XP will bring, it might be a good idea to ease gradually into XP from the testing perspective. Do have the development team (which should include a tester) complete acceptance testing before the end of each iteration. Then, if you usually spend two weeks in system test and a week in operational readiness test before each release, keep doing that, at least for the near future.

Post-development test phases are a two-edged sword. They may increase stakeholders' comfort level that the software package is ready for launch. On the other hand, they often become ominous regression spirals, where defects appear every few weeks, get fixed, and reappear later. In XP, we know unit tests must always pass 100%. We believe that acceptance tests, once they pass, must forever after pass 100%. This way, the regression death spiral can't occur. After a few releases, your customers may feel confident enough about the quality of the software as delivered by the development team to drop the extra test phases.

Another reason you might have to maintain some traditional practices, such as extra test cycles, is if your XP project produces software that has to interact with other systems delivered by non-XP teams. If your team's software is closely tied to that of a team using other development practices, establish a good working relationship with this other team at the start of your project.

As the tester, you can contribute lots of value here. Meet with the testers, programmers, and project managers of the other team. Explain your own testing practices and have them explain theirs. Set milestones for each team to produce code needed by the other, test records you may need for their system, and any other deliverables needed for development and testing. Define processes for dealing with defects whose source is hard to identify. Find out in advance what to do about a problem that appears to come from the other team's code.

XP is all about people. Developing personal relationships with the other team, whether they practice XP or not, will increase your project's chances for success and make everyone's life more pleasant.