Measuring the Value of Process Improvement

Oscar Wilde said, "Nowadays, people know the cost of everything and the value of nothing." Unfortunately this is often true for organizations focusing on process improvement. Most organizations can tell you their process improvement budget for the year and how much they have spent to date, but few can relate hard data regarding the benefits they've derived from that investment.

When asked about the benefits, management is likely to tell you things like: "Projects seem to be running a lot smoother now." "We don't seem to have as many late projects as we used to." "Customers seem to be happier with the quality of the products that we're producing." "We've achieved CMMI maturity level 3, so we know we're doing well!" If a process capability baseline was established before the project began , you should be able to demonstrate improvements in the list of desired outcomes .

Soft, qualitative data is comforting, but it is not likely to sustain the improvement program through the next budgeting cycle, let alone the next economic downturn. When times get tough, programs that cannot support the benefits they provide with hard data are likely to be thanked for their contribution as their people are redeployed to work on "real" projects that can.

So how does an SEPG go about measuring the benefits derived from process improvement? Clearly the alignment principle provides some guidance in this regard. Using the sample alignment principle, the organization must measure:

• Field- reported defects
• Cost variance
• Schedule variance
• Functional variance

The success (or lack thereof) of the process improvement program can be determined objectively by comparing the results of the projects with those proclaimed in the alignment principle. An SEPG that can demonstrate sustainable benefit in quantifiable terms is much more likely to have the opportunity to repeat this success.

To further establish the "measurement mentality " for process improvement, the SEPG should be encouraged (or is it "expected?") to hypothesize the value of each improvement in measurable terms prior to piloting it on a real project. The pilot should be conducted in such a way that it confirms or denies the realization of the hypothesized value, and deployment decisions should be based on this objective analysis. Deploying the process element as is, modifying the process element and running additional pilots, or abandoning the proposed changes are likely outcomes of such postpilot analyses. Remember that pilots are simply data-gathering activities; a successful pilot is one that contributes to organizational knowledge, not necessarily one that demonstrates a proposed process change is ready to be inflicted on all projects.