Transitioning to Level 5

Transitioning to Level 4

Level 4, either Capability Level or Maturity Level, is about managing quantitatively. It includes both organizational- and project-level quantitative management. When reading the CMMI, you realize that it is written from the perspective of an organization and its projects executing at a steady state. The Organizational Process Performance process area describes an organization that has in place both the PPBs and PPMs that will be useful to the projects. The Quantitative Project Management process area describes projects that have selected processes that will satisfy the project's objectives, that are executing stable processes, and that are managing process performance to best satisfy the project's quality and process-performance objectives using PPBs and PPMs.

OK, so how do you transition to Level 4? Here are some steps that we recommend. You will most likely need to perform multiple iterations through these steps as you become more experienced and your data become better.

Select Measures that Will be the Basis for Your PPBs and PPMs

An organization needs to develop the PPBs and PPMs that will be used by the projects. These baselines and models are built around what is important to the goals of the organization and the projects. We mentioned in previous chapters that the only goals some organizations have are "to make money." And so why is this a bad thing, you say? Well it is not, but we need to decompose that goal into something that the engineering, product development, and support groups can directly contribute to. Identify the critical components and attributes, and identify measures of the standard processes and their work products. Measures related to product quality, cost, and schedule are often your best bet.

In Chapter 17, "A Boatload of Metrics," we identified a list of nearly 200 measurements. We are not advocating that you collect anywhere near that number for your baselines and models, but you can use that list as a resource.

An example of using a single measure to build a PPM occurred in an organization we worked with that was able to do some impressive forecasting work based on only one measurement (the number of defects expected per thousand lines of code based on a similar project). The organization used that single number, its project's rate of defects found, and its project's rate of defects closed over the past four weeks to build an initial prediction model of when it would finish testing. The organization improved the model over the project's life cycle and got real customer buy-in and support. Its simple prediction model allowed the organization to better manage both the project schedule and customer expectations, turning a "problem project" into a "success story." Before you ask, No we are not saying that this one success story makes the organization Level 4; However, it did create an initial success that the organization was able to build upon and demonstrate the value of Quantitative Management to the organization.

Be sure to select measures that cover the life cycle for the types of projects that you want to quantitatively manage. For example, if you have maintenance projects, do not just measure the new product development life cycle. You might measure the number of problem reports, time to review problem reports, time to implement problem reports , time to test, number of retests, etc.

You need to consider both "breadth and depth." "Breadth" means the measures across the entire life cycle (e.g., cost, schedule, defects, and effort), and "depth" means the details in areas that are critical to your organization (i.e., the number of customer returns by release and the number of subcontractor failures by vendor).

Collect the Measures Identified from the Projects

Often, the most difficult part of measurement is to get good data. Again, do not be disheartened if you find problems with the data. Even if your standard process requires that your projects collect certain metrics, you may find that the data are missing or are incorrect. You may need to "mine the data" this means dig them up. The data may be in lots of different forms in paper records, Excel spreadsheets, Word documents, or PowerPoint slides. They may only be in project status reports. They may be in many separate systems in many separate departments finance, engineering, human resources, etc. Do not be surprised if you need to invest a lot of time and money to uncover old data.

Work with the projects early and often to make the data better. You may decide to take more time to collect new and better data.

Analyze the Data from the Projects

Chapter 18, "Statistical Process Control," shows a number of tools that you can use to analyze the data. In some organizations, this will be the first time anyone outside the project has really looked closely at the data. You are most likely going to find missing, incorrect, or dirty data; and the data are likely to show that not all projects are really following the standard process. Your analysis activities should include the following steps:

1. Review and fix the data. Investigate and correct missing data, zeros (for things such as the number of hours to perform a task), and really large or really small numbers .

2. Look for ways to normalize data across projects. For example, use "defects by size ," not just "defects."

3. Plot the data. Use a histogram to see if you have a normal or Poisson distribution. Use a scatter diagram to see any correlations . Use a run chart to view any trends. Use control charts once you find data that show promise.

4. Investigate the out of control points for their root cause. It is best to get project team members involved with this analysis because the recorded data are likely to be incomplete. Put corrective actions in place to address the root causes. (This is a good place to use the approach described in the Causal Analysis and Resolution process area.)

Establish Organizational PPBs and PPMs from the Project Data

Calculate the current and predicted process performance and capture that in a PPB. Be sure to establish baselines that cover the life cycle for the types of projects that you want to quantitatively manage. You will most likely want baselines that cover cost, schedule, and product quality. Be sure to identify the profile of the projects used to create the baseline; for example, large development projects using a waterfall life cycle and formal peer reviews, medium development projects using iterative life-cycle and informal peer reviews, or small maintenance projects using quality assurance-led reviews.

From the baselines, produce predictive models that will allow your projects to manage the goals that are important to your organization. You will most likely want models that predict productivity, defect insertion and detection, and schedule.

Derive Project Goals

The steps up until now have been organizational; and while that may be important, we need to remember that, as defined by the CMMI, the projects within the organization produce and deliver the products and services.

Thus, it is important to realize that the project has some goals to satisfy. These often include on-time delivery, implementing all the technical requirements of the project, and staying within cost and schedule. Goals can come from the customers; and goals may also come from your organization if you are reading this book, it is likely that your organization wants to improve or at least wants to get a rating. Those are organizational goals.

You need to derive the project goals so that you know what to quantitatively manage to get to those goals.

Select Critical Subprocesses to be Managed by the Project

Ensure that you have a set of subprocesses that cover the life cycle (breadth) of the project, and ensure that the critical subprocesses go into enough detail (depth) on the things truly critical to the project.

Select the measures and define what needs to be done to collect and store the measures for the project. OK, here is where you might have a problem if the project goals are not covered as part of your organizational goals. Investigate adding these project goals to the organizational goals, but only if appropriate. Some projects truly are unique. You may need to identify unique measures, baselines, and models to satisfy these project goals. However, we have found that most projects are not truly unique from other projects; therefore, the goals of the projects should have been incorporated into organizational goals.

Select the Process Performance Baselines to be Used by the Project

You need to compare the profile of your project to the profile of projects used to create the baselines, and select a PPB or set of PPBs to use. Here is where you may have another problem if the projects used to create the PPB are nothing like your project. However, do not give up just because things appear different. Here is an example. We were recently working with a large enterprise with multiple groups throughout the United States. This company had been comparing the average number of hours to fix problem reports. It found maintenance data in three separate organizations of widely separated geographic locations, using different processes and separate collection activities. To the company's surprise, the values came back as 12 hours per problem report, 13 hours per problem report, and 15 hours per problem report, respectively. These organizations had not compared notes, yet they came within 20 percent of each other. While you may want much closer ranges, these data proved useful in initial quantitative management of the maintenance projects.

Select the Process Performance Model(s) to be Used by the Project

Having selected the project's goals, measures, subprocesses, and PPBs, you can select the PPMs to use on your project. These may include:

• Schedule and cost models

• Progress models

• Reliability models

• Defect models (defect identification and removal rates, defect removal effectiveness, latent defect estimation)

• Productivity models

You will need to calibrate the models to match your project's performance. For example, your project will have its own defect and identification and removal rates that may be different from what is in the defect model. Be sure you understand the basis of the models before you do any calibration.

Manage the Project Quantitatively

The project can now use the PPBs and PPMs to manage process performance and to work toward achieving the organization's and projects' goals. This work includes:

• Maintaining control charts of the subprocesses and identifying the root cause of out of control points

• Adding the actual performance data to the prediction models

• Taking corrective actions throughout all phases of the project's life cycle to best achieve the project's quality and process performance objectives