Lessons Learned

Several points could have been addressed better during the CMMI case study:

1. As a result of interviewing the newer (software) project, the team discovered how much more meaningful some of the PAs (e.g., Requirements Development, Project Planning, Project Monitoring and Control, Integrated Project Management, and Risk Management) would have been had they been evaluated at the program element level, which joins the software and systems engineering efforts.

2. The use of defined criteria worked well for selecting the PAs that would be examined for each project. This allowed sampling of PAs to get a sense of the model without performing a comprehensive or exhaustive formal assessment.

   However, in retrospect, the selection of PAs fell short in a few places.

   • Process and Product Quality Assurance

   • Integrated Project Management for IPPD

   • Integrated Teaming

   • Organizational Environment for Integration

3. The newer project (CAU) planned to review Process and Product Quality Assurance but could not because attempts to fill the Software Quality Assurance role had not been successful at that time.

4. The team decided early on to use the systems engineering/software version of CMMI without the three IPPD-related PAs. It was initially understood that the IPPD PAs and practices would not be applicable to USA projects based on preliminary feedback. Later interviews with the newer project uncovered that much of the structure in place relies on the use of product development teams, and that a network of product development teams exists. The IPPD practices would have been applicable even though the project did not relate to the use of the IPPD term. The team should have probed a little deeper here during planning to more fully understand the applicability of the IPPD practices.

5. The case study mini-teams used any techniques they were comfortable using to elicit information from projects for writing observations. Some of the mini-teams elected to use the SEI questionnaires to solicit data and information on current project practices from participants. In some cases, the team amplified the questions with more explanatory text, making for long surveys. A lengthy survey and busy project participants did not make a good combination; only one respondent chose to complete the instrument. There was minimal return on investment for the effort expended to augment the existing instrument.

The informal techniques of conducting meetings, interviews, and distributing e-mail worked well. This placed the burden on the team member and not the project to collect, format, and report results rather than to depend on the project member to record all of the information on an instrument.