Start Early to Level 4 | Interpreting the CMMI (R): A Process Improvement Approach, Second Edition

Process Performance Model

A closely related concept to the PPB is the process performance model (PPM). The PPM describes the relationships among attributes of a process and its work products, and is used to estimate or predict a critical value that cannot be measured until later in the project's life ” for example, predicting the number of delivered defects or predicting the total effort. Attributes of a process include productivity, effort, defects produced, defects detected , and rework . Attributes of a product include size , stability, defects contained, response time, and mean time between failures. PPMs are built on historical data and are often built from PPBs. PPMs can be developed for a wide range of project objectives. Example PPMs include reliability models, defect models, and productivity models.

Exhibit 3 shows an example PPM for predicting effort based on both the PPB shown in Exhibit 1 and organization historical effort distribution. The PPB elements at the top of the second column (e.g., Number of Complex Reqs, Number of Design Pages, Number of Components, etc.) come from the Unit of Measure column in Exhibit 1. The Estimated Number of Elements column in Exhibit 3 is derived from project information gathered and from the organization's metrics database. The mean value for each PPB element by phase comes from the Mean column in Exhibit 1. The Historical Effort Distribution on line 12 of Exhibit 3 comes from project actuals stored in the organization's metrics database.

Exhibit 3: Example Process Performance Model for Effort (New Development)

Line Number	Estimated Number of Elements	Req Phase	Design Phase	Implement Phase	Integration Phase	System Test Phase	Total Effort
PPB Elements:		Mean	Mean	Mean	Mean	Mean
Number of Complex Reqs	75	35	49.8
Number of Nominal Reqs	100	21	31.7
Number of Simple Reqs	200	8.6	13.3
Number of Interfaces	TBD			8.6
Number of Design Pages	TBD			17.7
Number of Objects	TBD			4.31
Number of Subsystems	TBD				153.5
Number of Components	TBD				16.8
Number of Test Scenarios	TBD					12.4

Historical Effort Distribution (percent)		20	30	20	15	15	100

Estimates:
Based on PPB Elements		6445	9565
Based on Effort Distribution		6445	9668	6445	4834	4834	32,225

Actuals by Phase		6752

Prediction Based on Actual When Available or Best Estimate		6752	9565	6445	4834	4834	32,430

The purpose of the example PPM for effort shown in Exhibit 3 is to predict the total effort for the new development project. This model is designed to predict the total project effort throughout the life cycle, using more and better estimates and data as they become available. The example is divided into the following five sections

PPB Elements (Lines 1 through 10)

For each of the PPB elements the model contains an estimated number of elements and the mean effort in hours for each applicable phase from the example PPB shown in Exhibit 1. For example, line 2 shows that 75 complex requirements are estimated, that the mean number of effort hours in requirements phase for a complex requirement is 35, and that the mean number of effort hours in the design phase for a complex requirement is 49.8. At this time, only the number of complex, nominal, and simple requirement elements have been estimated; therefore, lines 5 through 10 show the remaining estimated number of elements as TBD (to be determined). Later in the project's life cycle, as estimates for these other elements become available, they would be added to the model.

Historical Effort Distribution (Line 12)

This shows that, historically, projects have taken 20 percent of total staff effort in the requirements phase, 30 percent in the design phase, 20 percent in the implementation phase, 15 percent in the integration phase, and 15 percent in the Systems Test phase. (We have simplified the effort distribution percentages to make the example easier to follow.)

Estimates

The model has two estimates:

The first estimate on line 15 is based on the estimated number of PPB elements that most affect each phase. You can see how this is computed by looking at the value under the requirement phase. This value is computed by multiplying the estimated number of requirements by the mean number of effort hours for that type of requirement:
- Number of complex reqs — mean effort req phase for complex reqs = 75 — 35 = 2625
- Number of nominal reqs — mean effort req phase for nominal reqs = 100 — 21 = 2100
- Number of simple reqs — mean effort req phase for simple reqs = 200 — 8.6 = 1720
- Summing those three values, we obtain 6445.
The total estimate based on PPB elements is calculated by summing the value for all phases. (Because only the first two phases are estimated, we do not have a total effort estimate on line 15.)
The second estimate on line 16 is based on the requirements phase estimate (6445 from line 15) proportionally propagated across the life cycle. We use the ratio of historical distribution for requirement effort to the historical distribution for effort of the other phases. Using the historical effort distribution from line 12 results in the following values:
- Design effort = 6445 — (0.30/0.20) = 9668
- Implementation effort = 6445 — (0.20/0.20) = 6445
- Integration effort = 6445 — (0.15/0.20) = 4834
- System test phase effort = 6445 — (0.15/0.20) = 4834
- Total effort is then the sum of all phases = 32,225

Actuals by Phase (Line 18)

This captures the actual hours per phase. This example is shown at the completion of the Requirements phase where 6752 actual hours were used. No other actuals are yet available.

Prediction Based on Actual When Available or Best Estimate (Line 20)

This is the real purpose of this model. The values on this line are selected from Actuals by Phase (line 18) if available. If Actuals by Phase is not available, then the value from Estimate Based on PPB Element (line 15) is used, if available. Use the Estimate Based on Historical Effort Distribution (line 16) if no other values are available. The prediction is then 32,430, which is the sum of all phases. (Not to be confused with the Sum of All Fears , which is a book by Tom Clancy.)

So how do you use a PPM with your projects? You use PPMs to estimate or predict a critical value that cannot be measured until later in the project's life. Our example showed a model to predict total effort throughout the project life cycle. No one really knows what the total effort will be until the project is finished.

We have seen a model similar to our example used to accurately predict total effort. The PPM used came within 5 percent of the total effort following the completion of the Requirements phase. We have also seen models built that successfully predict the number of delivered defects, mean time between failures, and number of system failures during integration testing. The exciting thing about the use of the models is that the focus of project management is on trying to find defects and trying to accurately measure performance, and not on dysfunctional interoffice politics.