ACTIVITY-LEVEL ANALYSIS USING SPC

11.3 ACTIVITY-LEVEL ANALYSIS USING SPC

Milestone analysis uses metrics to monitor the state of the project at defined milestones and to suggest corrective actions if needed. However, with milestone-based analysis, it is hard to determine which of the many activities performed since the last milestone may be the cause of performance degradation. This situation is similar to performing system testing; debugging is much harder when the system comprises many units. It is far easier to debug during unit testing. This difficulty in pinning down the cause of poor performance limits the corrective actions that can be taken.

To provide finer control, activity-level analysis is also done at Infosys. In activity-level analysis, some metrics are analyzed immediately after a task is performed. The analysis is used to evaluate the effectiveness of the task performance. If the task has not been performed satisfactorily, immediate action can be taken to correct it and to prevent a repetition.

Activity-level analysis is usually done using statistical process control (SPC). At Infosys, reviews and unit testing have been identified as the two tasks for which SPC is applied. Using control charts, the project manager immediately evaluates the effectiveness of a review or a unit test. If the performance is not satisfactory, necessary action can be taken.

Note that the broad activities of design, detailed design, system testing, and so on are already evaluated individually through milestone analysis because they typically start and finish at milestones. Hence, the focus of activity-level monitoring is on the coding-related activities of reviews and unit testing.

Chapter 7 discusses the basics of statistical process control, and Chapter 10 explains the monitoring and control of reviews using SPC. Here we discuss activity-level monitoring and control of unit testing the other activity for which SPC is applied. The approach for unit testing is similar to that used for monitoring reviews. The main performance characteristic that is monitored is the density of defects detected in unit testing. From the past data on unit testing, the control limits (that is, the range of acceptable density) are obtained. At the end of each unit test, the defect density is checked against the range. If it falls outside the limits, corrective or preventive actions may have to be taken. The guidelines given earlier for testing can be used to decide which actions are needed (for analysis, project managers can use supporting information about effort spent on the task, effort expended on previous tasks, and defects found). Figure 11.4 shows the control chart for defect density for unit testing of one programming language. The defect density can be plotted in terms of the number of defects per LOC or the number of defects per person-hour. Table 11.5 shows the capability baseline for unit testing, giving defect rates for a few languages.

Figure 11.4. Control chart for unit testing of Visual Basic programs

Table 11.5. Unit Testing Defect Rates for Some Languages
Language	Defect Rate Range (Average)
PB	0.0003 0.0266 defects/LOC (Avg: 0.008)
C	0.0004 0.0206 defects/LOC (Avg: 0.0052)
C++	0.00009 0.0067 defects/LOC (Avg: 0.0017)
RPG	0.0006 0.0075 defects/LOC (Avg: 0.0025)

As with reviews, project managers can use the SPC tool to implement activity-level control of unit testing. This tool contains relevant past performance data such as the defect injection rate, the defect removal percentage, and so on. Using this data, it determines the control limits as well as expected limits. When the performance data of a unit test is entered, the tool immediately tells whether or not the performance is outside the limits. If it is outside the limits, further analysis must be done to determine what action, if any, should be taken.