SEI CMM, SEI CMMI, ISOIEC, DOD-STD, MIL-STD: High Ceremony Striving for Higher Predictability

SEI CMM, SEI CMMI, ISO/IEC, DOD-STD, MIL-STD: High Ceremony Striving for Higher Predictability

Parallel to the Agile Movement is a strong trend toward high-ceremony approaches to software development. This trend includes software assessment frameworks such as SEI CMM, SEI CMMI, and ISO/IEC and processes such as DOD-STD and MIL-STD. Companies increasingly see software as strategic business investments and can no longer afford unpredictable results in terms of cost overruns and quality issues. As a result, many organizations have become aware of the importance of using a well-defined and well-documented software development process to facilitate success of their software projects.

Characteristics of a high-ceremony project include the following:

• Thorough plans are carefully documented.

• Many artifacts related to management, requirements, design, and testing are produced.

• Management, requirements, design, and testing artifacts are detailed, documented, and put under version control.

• Traceability links between requirements, design elements, and test artifacts are created and maintained .

• Change Control Board approval is required for changes.

• Inspection results are carefully recorded and tracked.

High-ceremony approaches are appropriate for complex systems development, large teams, and distributed teams . Such approaches often produce quality systems that are easy to maintain, but the cost of producing them is typically higher, and time to market is slower than for low-ceremony approaches. If time-to-market is a primary concern, high-ceremony approaches may actually lead to lower quality, because there just isn't enough time to "do the job" prescribed by the process properly.

SEI CMM: Process Assessment Framework

In the quest for better predictability and higher quality, organizations first need to understand how good their current process is, and what they should look for in a future process to reach a higher level of process maturity. The Capability Maturity Model, ^[11] developed by the Software Engineering Institute, is designed to do exactly this. The CMM is not a process ”it is an assessment framework used to determine an organization's process maturity on a scale of 1 to 5. CMM does not tell you how to develop software ”for that you need a process, such as the RUP, XP, or MIL-STD-498.

^[11] See http://www.sei.cmu.edu/cmm.

The CMM provides a very detailed outline of everything that could (not necessarily "should" ”that would depend on project and organization specifics) be part of a software development or acquisition process for a mature organization. Unfortunately , many organizations and CMM assessors interpret this to mean that the more artifacts and activities used (equating to high ceremony), the better the process. But adding unnecessary artifacts and activities to your process, just to reach a higher process maturity rating on the SEI CMM scale, results in an overweight process ”one that is cumbersome and ineffective . By having an overly strong focus on peer reviews, inspections, traditional quality assurance activities, and detailed planning, CMM also has the undesired effect of encouraging waterfall development rather than iterative development ^[12] by not forcing the identification of issues through early and continuous integration and testing.

^[12] See Royce 2002.

SEI CMMI: Process Assessment Framework

To address this issue, SEI introduced SEI CMMI, ^[13] which more effectively accommodates modern best practices such as risk-driven and iterative development. Rather than encouraging an organization to create more ceremony (as CMM does), CMMI encourages users to focus selectively on the individual areas for improvements that best meet the organization's business objectives and mitigates the organization's areas of risk. Figure 3.3 shows where CMM and CMMI fit onto our process map.

^[13] See http://www.sei.cmu.edu/cmmi/general/genl.html.

Figure 3.3. SEI CMM and CMMI on the Process Map. SEI's CMM and CMMI both advocate high-ceremony approaches to software development. CMM has a slight bias toward waterfall development, while the newer CMMI has a slight bias toward iterative development.

ISO/IEC 15504: Process Assessment Framework

Similar to CMMI, ISO/IEC 15504 is a framework that assesses the maturity of software processes on a scale of 1 to 6; it was derived from the Software Process Improvement and Capability Determination (SPICE) project. ISO/IEC 15504 has many similarities with SEI CMMI, including selective assessment of only the process areas that are the most valuable for your organization. ISO/IEC 15504 can be placed roughly at the same place on our process map as CMMI.

DOD-STD and MIL-STD: High-Ceremony Processes

Standards that provide a high-ceremony software development approach in an attempt to minimize cost and schedule overruns are DOD-STD-2167, DOD-STD-2167A, MIL-STD-1521B, and MIL-STD-498 ”all developed by the U.S. Department of Defense (DOD). These standards drive users toward a high-ceremony approach, with the exception of MIL-STD-498, which stresses that users should produce only artifacts that are necessary for their specific project. The authors of 2167 and 2167A went through some trouble explicitly to avoid a default waterfall interpretation, but unfortunately these standards were normally combined with MIL-STD-1521B, which prescribed a sequence of high-ceremony reviews (requirements review, preliminary design review, critical design review, and so on) that were so expensive to perform they made anything but a waterfall approach impractical . MIL-STD-498 is slightly more open to using iterative development, but it still does not go far enough (see Figure 3.4). Interestingly, the DOD now encourages the use of iterative development.

Figure 3.4. Various Military Standards for Software Development on the Process Map. DOD-STD-2167 and 2167A, combined with MIL-STD-1521B, provide a very high-ceremony, waterfall approach to software development. MIL-STD-498 promotes a slightly more iterative approach to software development.