Software Process Maturity Assessment and Software Project Assessment

The scope of a software process assessment can cover all processes in the organization, a selected subset of the software processes, or a specific project. For most process assessments that are based on the maturity concept, the target unit of analysis and rating is normally at the organizational level. In fact, most of the standard-based process assessment approaches are invariably based on the concept of process maturity. This is the case for the SEI (Software Engineering Institute at Carnegie Mellon University) capability maturity model (CMM), the SPR (Software Productivity Research, Inc.) approach, the Trillium model (a CMM-based model developed by a consortium of telecommunications companies, headed by Bell Canada), as well as the recently developed BOOTSTRAP methodology (the result of a European Community project) and the ISO/IEC 15504 draft standard (Zahran, 1997).

When the assessment target is the organization, the results of a process assessment may differ , even on successive applications of the same method. Paulk and colleagues (1995) explain the two reasons for the different results. First, the organization being investigated must be determined. For a large company, several definitions of organization are possible and therefore the actual scope of appraisal may differ in successive assessments. Second, even in what appears to be the same organization, the sample of projects selected to represent the organization may affect the scope and outcome. This project sampling effect can be substantial for large organizations with a variety of projects.

When the target unit of assessment is at the project level, the potential problems associated with organizational-level assessments just discussed are not relevant. The ambiguities and vagueness with regard to assessment results do not exist. Furthermore, some process dimensions of a standard process assessment method may not apply to a specific project. On the other hand, a software project assessment should include all meaningful factors that contribute to the success or failure of the project. It should not be limited by established dimensions of a given process maturity model. One should assess not only the processes of the project, but also the degree of implementation and their effectiveness as substantiated by project data. Project assessments address " hows " and "whys" with sufficient depth, in addition to the " whats ." Therefore, exploratory and in-depth probing are key characteristics of software project assessments. In this regard, the standard questionnaires used by the maturity assessment models may not be sufficient. It is well known that the standard questionnaires address the "whats" but not the "hows" by design so that each organization can optimize its own approach to process maturity. Because of this inherent limitation of standard questionnaires, standard-based process assessment models also rely on other data gathering methods such as document reviews and extensive interviewing.

In addition to the difference in the unit of analysis, the very concept of process maturity may not be applicable to the project level. What matters is the success or failure of the project, as measured in field performance and development effectiveness and efficiency. If the projects achieve measurable improvement, whether or not a certain set of process activities is being practiced, or a certain maturity level is achieved, is not relevant. If a project fails, the remedial actions have to aim directly at the causes of failure. Process maturity becomes relevant when an organization intends to embark on an overall long- term improvement strategy. Even then, the additional value derived from the implementation of additional process elements needs to be monitored and verified at the project level.

Software project assessments, informal or formal, must be independent assessments in order to be objective. The assessment team may be in the same organization but must be under a different management chain from the project team. It may come from a different division of the company, it could be an external team, or it could be a combination of internal personnel and external consultants .

The necessity of and demand for project assessments exist regardless of whether the organization is pursuing a long-term process maturity improvement strategy. Within an organization of a specific maturity level, there are always variations among projects with regard to the state of practices of development methodologies, how they are implemented and why, and their correlation with the project outcome. The two types of assessment can be complementary: the process maturity assessments for overall improvement strategy for the organization and specific project assessments to drive immediate and specific improvement actions at the project level. With customization and a shift in the assessment focus (unit of analysis), standard process assessment methods might be applied to project assessments. For small organizations with a few projects, the distinction between a process maturity assessment and a project assessment may be blurred.