Now let's discuss the quality planning of our case study, the ACIC project. (Other examples, including a different case study, can be found in my earlier book.9) To set its goals and defect estimates, the ACIC project manager used the data from the Synergy project, a similar project done earlier for the same client. The defect injection rate in Synergy was 0.036 defects per person-hour (obtained by dividing the total number of defects by total effort both available in the Synergy process database). The ACIC project manager wanted to do better than Synergy and expected to have a 10% reduction in the defect injection rate, considerably better than the organization norms. At the projected rate, the number of defects injected was expected to be around 501 * 8.75 * 0.033 (the product of effort in person-days, the total hours in one person-day, and the expected defect injection rate). That is, the estimate of the total number of defects injected during the entire life cycle was around 145 defects.
Quality is defined as the defect density during acceptance testing, or the overall defect removal efficiency before acceptance testing. Synergy found 5% of the defects during acceptance testing. The ACIC project aimed to reduce it to 3%, giving the number of defects expected to be found during acceptance testing as 145 * 0.03 = 5 (approximately).
The productivity goal was a slight improvement over what was achieved in Synergy. The goal for the schedule was to deliver on time, and the expected cost of quality was 32%, which was the same as in Synergy and the organization capability baseline. Table 5.1 shows all these goals for the ACIC project.
For the purposes of monitoring and controlling the project, the ACIC project manager wanted estimates of the number of defects detected in the various stages. He could then compare these estimates to the actual number of defects found and use them to monitor the progress of the project. With the estimate of the total number of defects injected, he could obtain these per-stage estimates using defect distribution.
Table 5.1. Goals for the ACIC Project | |||
Goals | Value | Basis for Setting Goals | Organization-wide Norms |
Total number of defects injected | 145 | 0.033 defects/person-hour. This is 10% better than Synergy, which was 0.036 defects/person-hour | 0.052 defects/person-hour |
Quality (acceptance defects) | 5 | 3% or less of total estimated number of defects | 6% of estimated number of defects |
Productivity (in FP/person-month) | 57 | 3.4% productivity improvement over Synergy | 50 |
Schedule | Delivery on time |
| 10% |
Cost of quality | 32% | 31.5% | 32% |
To obtain the defect distribution, he had the choice of using the capability baseline data or the Synergy data. Because Synergy did not have a requirements phase, he modified its distribution to suit the current life cycle. Essentially, he reduced the percentage of defects found in unit testing from 45% to 40%, reduced the percentage of defects in acceptance testing to 3% (because that was the quality goal), and increased the percentage of defects in requirements and design review to 20%. These percentages were also consistent with the distribution given in the capability baseline. Table 5.2 shows the estimates of defects to be detected in the various stages.
Because the quality goal was higher than that achieved in Synergy as well as the organization-wide norms, and because the productivity goal was also somewhat higher, the ACIC project manager had to devise a strategy to achieve these goals because following the standard process would not help achieve them. The basic strategy was threefold:
To employ defect prevention
To conduct group reviews of specifications and the first program written by programmers
To use the RUP methodology
Table 5.2. Estimates of Defects to Be Detected | |||
Review/Testing Stage | Estimated Number of Defects to Be Detected | % of Defects to Be Detected | Basis for Estimation |
Requirements and design review | 29 | 20% | Similar project (Synergy) and PCB |
Code review | 29 | 20% | Similar project (Synergy) and PCB |
Unit testing | 57 | 40% | Similar project (Synergy) and PCB |
Integration and regression testing | 25 | 17% | Similar project (Synergy) and PCB |
Acceptance testing | 5 | 3% | Similar project (Synergy) and PCB |
Total estimated number of defects to be detected | 143 | 100% |
|
Table 5.3. Strategy for Achieving the Higher Goals of the ACIC Project | |
Strategy | Expected Benefits |
Prevent defects using the standard defect prevention guidelines and process; use standards developed in Synergy for coding. | 10% 20% reduction in defect injection rate and about 2% improvement in productivity. |
Group review program specs for first few and the logically complex use cases. Group review design docs and first-time-generated code. | Improvement in quality because of improvement in overall defect removal efficiency; some benefits in productivity because defects will be detected early. |
Introduce RUP methodology and implement the project in iterations. Conduct milestone analysis and defect prevention exercise after each Iteration. | Approximately 5% reduction in defect injection rate and 1% improvement in overall productivity. |
That is, as compared with the process used in Synergy, the ACIC process was changed to achieve the higher goals.
Based on data from other projects, the project manager expected defect prevention to reduce the defects by about 10% to 20%. This would reduce the rework effort after testing and reviews, giving approximately a 2% improvement in productivity (based on the rework effort percentage of Synergy). He expected that group review of the program specifications and of the first module coded by programmers would improve the overall defect removal efficiency and also provide some benefits on the productivity front. Based on literature and anecdotes, he expected the use of RUP to benefit quality and productivity. Table 5.3 shows the strategy and expected benefits. Note that although the expected benefits of each strategy item are mentioned separately, it is hard to monitor the effects separately.
Because reviews are a key aspect of the quality process, they were mentioned separately in the quality plan. The plan specified the points in the development life cycle when the review was to be done, the work product to be reviewed, and the nature of the review. Table 5.4 shows these reviews in the ACIC project.