Monitoring the Process Improvement Effort | Interpreting the CMMI (R): A Process Improvement Approach, Second Edition

The Pilot Phase

After the Design phase has completed, the processes developed by the PATs should be piloted across two or more projects. The Pilot phase consists of the following steps:

Select pilot projects.
Document success criteria and measurement techniques.
Orient and train pilot project members in CMMI concepts.
Orient and train pilot project members in the processes and procedures developed.
Perform the pilots.
Monitor the pilots.
Analyze results from the pilots.
Measure success.
Provide lessons learned.
Update procedures and re-pilot as needed.

As described below, the piloting effort consists of a structured approach to implementing a selected number of procedures in project(s) throughout the organization, and then evaluating the procedures throughout the project(s), as well as at the end of the pilot.

Select the Pilot Project(s)

The project or projects targeted for the pilots selected must be in the appropriate phase of the process, and must be in the appropriate phase of the life cycle. For example, you cannot adequately pilot your test procedures if none of the selected pilot projects have reached the test phase yet. To adequately test the procedures during the pilot process, the pilots have to be of sufficient duration and size , and have an appropriate number of staff members. The pilots also must last long enough to test all of the procedures piloted. For example, if the procedures being piloted were written for regular projects that were made up of 50 people, lasted ten months, and consisted of over a million lines of code, and involved systems engineering, software engineering, and software acquisition personnel, one would expect the pilot to take at least three months and involve members of those organizations mentioned. One would also expect the pilot projects to not be small, Web-based design projects, but match the type of projects for which the procedures were written. Typically, we request that the pilot project last between two and six months, and that at least five process improvement staff members be involved with the pilot on an ongoing basis.

Document the Success Criteria

To determine whether the pilots performed are going to be successful, the organization needs a clear-cut vision of what success really means. The following are some of the criteria that can be used to determine the success of a pilot project:

How long will the pilots last?
How many pilots will there be?
How will the use of the procedures be monitored ?
How will needed changes to the procedures be identified and implemented?

Train Pilot Participants

Project staff from the pilots must be trained in the policies and procedures to be followed and the purpose of the intended pilot. Training does not consist of simply handing the pilot participants the written procedures and telling them to read them. The EPG may assist in the training. Process Action Team (PAT) members may also assist.

Monitor the Pilot Efforts

The EPG provides an ongoing program for monitoring and analyzing the pilot progress by monitoring and answering questions posed by the pilot project teams .

Refine Policies and Procedures Based on Pilot Results

The PATs must be prepared to make changes to the policies and procedures, both on an ongoing basis as the pilot progresses and afterwards as the pilot is evaluated.

After the Pilots

The pilot projects will last as long as there are process areas to be piloted, or significant changes to processes. The overall plan should be to use these initial pilot projects on as many process areas as possible. If the originally selected projects for piloting cannot be used again, then new pilot projects may need to be selected.

More information concerning checklists for the pilot effort can be found in the appendices.