The Process Areas for Maturity Level 4: Quantitatively Managed

This chapter is designed to help the reader understand the basic tenets of Maturity Level 4 in the staged representation of the CMMI . However, because this chapter consists of summaries of the process areas, anyone wishing to get a better idea of the model, no matter which representation is to be used, can benefit from this chapter. Once again, we are not attempting to teach the CMMI; we simply offer a condensed version of the various areas and key points to consider.

Moving from Level 3 to Level 4

Maturity Level 4 is all about numbers. The projects are managed "by the numbers." Organizational decisions are made "by the numbers." Processes, services, and product quality are all measured "by the numbers ." At Level 4, the organization has achieved all of the goals of Levels 2 and 3. Processes, while qualitatively stable and predictable at Level 3, can be proved to be quantitatively stable and predictable at Level 4. The major difference between Level 4 and the next level (Level 5) is that Level 4 analyzes the data collected, determines special causes of variation from the norm, and supports quantitative management and control. You do this to make your processes predictable. Level 5 addresses common causes of variation. So, for Level 4, an organization needs data that are stable and consistent. The major preoccupation of assessors when reviewing Level 4 is: "Did this organization mix apples and oranges? Are the data really accurate? Did this organization collect the right data and did they collect the data right?" To get "good" data, an organization usually has to collect data for several years , or at least through several projects and several life cycles of the projects. And when you first begin collecting data, they will not be consistent data.

Measurement data are collected beginning with Level 2 in the staged representation and, in most organizations, actually begin being collected at Level 1. The problem is that the data are not clean and consistent because the processes used on the projects (where the data are collected and used) are not yet stable and consistent. Data in and of itself are not magical : they simply reflect what is going on in the projects. The point is that an organization cannot go to Level 4 overnight, and the focus is on the data.

What problems do we see in organizations when they decide to move from Level 3 to 4? At Level 3, measures are collected and preliminary thresholds are established, usually relating to size and effort. If the thresholds are exceeded, some sort of corrective action is undertaken. At Level 4, the control limits are based on years of historical data and trend analyses done on those data. More data are collected, and therefore more limits are established, monitored , and refined as necessary. At Level 3, the data may be somewhat inconsistent and "dirty." Although in a perfect world we would like to see "clean" data at Level 3, the focus in Level 3 is on organizational process, not necessarily on stabilized, normalized, statistically accurate data ” which is exactly what Level 4 expects. One problem that we see in some organizations that have barely met the criteria for Level 3 are that the processes are not always followed consistently across the organization. Now, one author of this book has stated that, in that case, this organization is not Level 3 and should never have been awarded Level 3. The other author of this book, after many arguments and nights spent sleeping on the couch which he definitely deserved for having such a stupid opinion, finally agreed ” with the following caveat: it all depends on what the organization does to enforce consistency, how important the consistency issue is to the appraisal team, what type of assessment is done to award the level, and who is on the team. So those of you new to process improvement, expecting black-and-white answers and no "wiggle room" in interpreting, implementing, and assessing this effort, are in for a letdown. Review Appendix B, entitled "Myths and Legends of the CMMI" and Chapter 11, entitled "Appraisals Using the CMMI," for more information.

At Level 3, we have also seen that tailoring the organizational process can get out of hand. That is, there is so much tailoring of the process that it cannot be judged to be a "standard" process. In addition, the measurement culture at Level 3 is not often well understood . People may not really understand why they are collecting the metrics they collect ” only that they are "required" by the model, and so that is why they are collected. The metrics cannot be studied for consistency and clarity, or may not be closely analyzed to determine why the numbers are inconsistent, and whether that represents a potential problem or not. Because the numbers can be "wild," management decisions made using the numbers can be pretty much off-base. And with inconsistent data, you really cannot compare the performance of several projects against each other to truly analyze trends.

At Level 4, managers and analysts must use the data, and apply statistical and quantitative techniques to help monitor activities, identify potential problems, and note areas that need attention. The instructions for the CMM for Software at Level 4 basically said to keep it simple. The measurements here in the CMMI, and suggested techniques, are very sophisticated and are more difficult to implement.

Level 4 is about making processes stable and predictable. Level 5 is about making improvements to stable processes so as to improve the functioning of the organization.

Why do all this measurement stuff? Because it supports a proactive approach toward managing projects.

There are two process areas for Level 4:

1. Organizational Process Performance

2. Quantitative Project Management

Note that there are no additions to the list of generic goals at Level 4 from Level 3. What makes this Maturity Level different is the two process areas. The generic goals are listed below. We use the abbreviations GG for Generic Goal and GP for Generic Practice. The Common Features (Commitment to Perform ” CO, Ability to Perform ” AB, Directing Implementation ” DI, and Verifying Implementation ” VE) are also abbreviated.

• GG3: Institutionalize a defined process

  • GP2.1 (CO1): Establish an organizational policy

  • GP3.1 (AB1): Establish a defined process

  • GP2.2 (AB2): Plan the process

  • GP2.3 (AB3): Provide resources

  • GP2.4 (AB4): Assign responsibility

  • GP2.5 (AB5): Train people

  • GP2.6 (DI1): Manage configurations

  • GP2.7 (DI2): Identify and involve relevant stakeholders

  • GP2.8 (DI3): Monitor and control the process

  • GP3.2 (DI4): Collect improvement information

  • GP2.9 (VE1): Objectively evaluate adherence

  • GP2.10 (VE2): Review status with higher-level management

To satisfy the goals for Level 4, the goals for Levels 2 and 3 must be satisfied as well. This mandate holds true for both the specific goals and the generic goals listed above.

The following pages discuss each process area for Level 4. We use the abbreviations SG for Specific Goal and SP for the corresponding Specific Practices.