Testing with iterative lifecycle models, such as the Rational Unified Process, occurs during, and especially near the end of, each iteration. As explained in Chapter 2, each iteration produces a specific, tested, executable release that addresses specific risks and requirements. These releases are then baselined in a configuration management tool. As testing progresses, defects are entered into a defect-tracking tool. The defects are prioritized and, in most cases, corrected and retested by the end of that iteration. Some lower-priority defects can be deferred to subsequent iterations. Figure 11-2 illustrates this process. Coding begins in iteration 1. Testing begins during the iteration and concludes by the end of the iteration. Defect correction begins near the end of the iteration. The process continues for each subsequent iteration.
Figure 11-2. Testing and defect correction progress during iterations
Advantages of Testing with Iterative Development
Numerous improvements to the testing process are made possible through the use of iterative development. The following list explains each of these advantages:
Prerequisites for Testing with Iterative Lifecycle Models
Before implementing a testing process on projects that use iterative lifecycle models such as the RUP, the practitioner should be aware of the testing team's needs. These needs do not differ significantly from testing teams on Waterfall lifecycle projects, but some needs take on an increased level of urgency.
For each iteration, only a subset of the complete set of functional requirements are implemented and tested. Furthermore, at the conclusion of each iteration, while planning the next iteration, the requirements assigned to subsequent iterations may change. Finally, requirements themselves can change. In Waterfall-based processes, from a testing perspective, this is not a problem, because all functional requirements are considered ready for testing all at onceafter they are all implemented and integrated to create the complete system. For an iterative lifecycle model, this requires careful tracking of requirements. Because testing begins earlier in the project lifecycle, requirements are more likely to change. Also, it's important to know for each given iteration exactly which requirements have been implemented and which ones have not.
This means that iterative lifecycle models require some sort of automated requirements management tool. Testing personnel need to quickly determine which requirements are implemented for each iteration. They need to know this information as soon as it is available so that test plans and scripts can be finalized. They need to be informed quickly if any requirements change so that they can review any test artifacts to determine how the change affects the artifacts and make any adjustments to the test artifacts if needed.
I also like to use the requirements management tool to track, for each functional requirement, which requirements have been implemented and which have passed testing. As the iterations progress, I often find it helpful to post graphs and charts of the progress in some common area where project members can see them. They can even be used as input to calculate Earned Value if your project uses it.
It's also vital to have robust configuration management tools and processes in place for testing with iterative lifecycles. This includes workspace management. Because testing and development occur simultaneously, the configuration management tools and processes must do the following easily and intuitively:
A defect tracking system that is easily accessed by testing personnel and developers must be in place. It should track all information related to the defect, including how it is reproduced. In particular, it should identify the baselined build in which the defect was discovered, along with an estimate, or rating, of the defect's severity.
It's important for the testing personnel to thoroughly document the exact steps needed to reproduce the problem. Screenshots of any error messages or screen settings are also useful. When the testers enter a defect into a defect tracking system, they should word the defect's headline or title carefully. The tester should use the headline or title to capture the essence of the problem and also get the team's attention. You don't want the developers to be unable to reproduce the defect because the description is too vague. This wastes the time of both testing personnel and developers.
Testing personnel are needed much earlier on projects that use iterative lifecycle models. However, unlike the "big bang" of the Integration and Test phase of the Waterfall lifecycle, staffing can ramp up more slowly, starting with senior test personnel for planning the testing process. As more iterations are completed, more test personnel can be added as the amount of code to be tested increases. This may be alleviated somewhat through the use of automated testing tools, which are discussed a little later in this chapter.
Because testing begins much earlier in an iterative lifecycle, testing personnel and developers work together closely for much of the project. It is vital that these two groups work well together. It's not uncommon for testing personnel to need assistance from the developers while they are testing an iteration. This informal communication is good and should be encouraged. However, the testing personnel need to remember that the developers are on fairly tight deadlines as they work to produce the next iteration.
Feedback for the Iteration Planning Process
As the testing results become available for a given iteration, the project manager should examine the defect reports that are available from the testing. As part of planning the iterations, the project manager should allow some time to correct bugs as well as test and implement another subset of the requirements. It may be unrealistic to expect to have enough resources to fix every bug. In that case, the severity rating on each defect report can be used to prioritize which ones will receive attention in the next iteration. Maintaining a "queue" of the remainder of the defects can be useful if a developer finishes his work for an iteration early, or an extra person becomes available. This keeps everyone busy and uses staff more efficiently. It also reduces or eliminates an extensive "cleanup" or defect correction activity later in the project lifecycle. Any defects that did not receive attention due to time constraints should be documented in the product's release notes. Although defects are undesirable, the fact that they are documented tells the customer that you've already tested the product and are at least aware of them.
To illustrate this process, consider Figure 11-2, shown earlier. During iteration 1, a build becomes available containing functionality that can be tested. The testing begins, and defect reports are submitted into a tracking tool. Toward the end of iteration 1, defects are prioritized, and correction and retesting begin. Any defects not corrected are examined, prioritized, and scheduled for correction in the next iteration. In the next and subsequent iterations, the process repeats until the end of the project.