APPRAISAL MYTHS

In the first part of this chapter, I ve attempted to debunk some myths surrounding CMMI implementation in organizations, but wait, there s more! Since first becoming a SEI-authorized lead assessor in 1998 and then later an authorized SCAMPI lead, I think I ve heard as many myths regarding maturity appraisals as I have about CMM or CMMI implementation. Some of them are downright bizarre.

Remember, as a change agent in your organization, you need to first understand why the CMM and the appraisal myths are being perpetuated. The myths may be serving a vital purpose to someone in power in the organization. There are often better (read: safer) ways to get the emperor to put some clothes on than publicly stating that he s naked.

Take a minute to read through the following list of appraisal- related myths. Decide which ones you think are myths and which are fact-based beliefs. Then read the following sections to find out if what you believed about maturity appraisals is true.

Note: In some organizations, the word audit is used to refer to maturity appraisals or evaluations. Not only is audit technically inaccurate, it is also a negative, fear- inspiring word. If this is the word used in your organization, find out how that came to be, and try to get people to change the language. It s not beneficial to your process improvement efforts to have people trembling in fear of appraisals.

• When an organization passes an appraisal, it gets certified by SEI at a CMM maturity level.

• For an appraisal to be official, it needs to be led by someone from outside the company.

• You re not allowed to have people from your own organization on the appraisal team.

• If we can get an easy lead appraiser, we ll pass the assessment.

• We can pass the appraisal if we just get our people to give the right answers in the interviews.

Myth or Methodology: When an Organization Passes an Appraisal, It Gets Certified by SEI at a CMM Maturity Level

This one always amuses me. There have been individuals who I ve told (and shown in print) at least three times that this is pure myth and yet, somehow, the next time those people mentioned appraisals, they talked about being certified. They just simply want to believe.

But before we begin debunking the getting certified aspect of a maturity appraisal, let s first get rid of the pass or fail concept of assessments. Yes, ultimately, you may want an appraisal team to determine whether or not your organization has reached a certain maturity level. But if that s all the organization is after, then it s not mature, irrespective of what level rating comes out of the appraisal. The primary goal and the reason why an organization spends so much effort and money on an appraisal should be to determine its process strengths and weaknesses, which hopefully turn into action plans for continued improvement.

Now, back to the myth. So here it is again, try to get it this time ” nobody certifies anybody at maturity levels. Not only does SEI not certify organizations at maturity levels, SEI emphatically and publicly states that it doesn t even validate assessment results that are reported to it. Here s SEI s statements that you ll find on its Web site with regard to assessment results that get reported to it: ^[49]

The terms SEI certified and CMM certification are simply incorrect since there is no such thing.

The SEI does not certify organizations.

The SEI only licenses and authorizes lead appraisers to conduct appraisals.

The SEI or any other organization is a certifying authority of the results from an appraisal.

The SEI did not confirm the accuracy of the maturity levels reported in this list and has no intention of doing so.

Here s what actually happens: An appraisal team spends a week or more in your organization evaluating your software or system engineering practices against the practices and goals in CMMI. At the end of the appraisal, the appraisal team delivers a findings presentation. If it was documented in the approved appraisal plan that the appraisal would yield a maturity level rating, then the final findings presentation will include a statement indicating whether or not your organization has satisfied the goals of a particular maturity level. That s it. There s no plaque, no certificate from SEI, nothing of the sort . If you really want a certificate, drop me an e-mail and I ll print you one up for oh, let s say $50 dollars.

Myth or Methodology: For an Assessment to Be Official, It Needs to Be Led by Someone from outside the Company

This is myth. With regard to appraisals, you should probably eliminate the word official from your vocabulary, since you re not likely to find a consensus definition or meaning for it. At any rate, if you or your management really need to be able to call your appraisal official, then just ensure that you plan and conduct an appraisal that is compliant with SEI s Appraisal Requirements for CMMI (ARC). In a SCAMPI Class A ” which is the only class of CMMI appraisal that can yield a maturity level rating ” the lead appraiser should not be from the organizational unit being appraised because this would represent a risk to objectivity. So, it really depends on how your organization has defined organization or organizational unit for the purposes of SCAMPI. If you re working in a large company, you may have process improvement or CMMI efforts going on in many subunits or organizations within the company. However, it is not likely that those separate efforts are coordinated at the corporate level to the extent you could perform a SCAMPI for the entire company in a single appraisal. Besides, the logistics of doing so would be a nightmare.

The reason many companies invest in sending people to SEI s Lead Appraiser training is so that they have internal resources to use to lead appraisals. Having internal people lead appraisals can save the organization anywhere from $20,000 to $40,000 per SCAMPI Class A appraisal. Sometimes, even though an organization may have access to internal lead appraisers, it may still hire someone from the outside because the management perceives a need for a higher level of experience and expertise than that possessed by its internal people.

Myth or Methodology: You re Not Allowed to Have People from Your Own Organization on the Assessment Team

Not true. In fact, the SCAMPI methodology requires that some portion of your appraisal team membership be people from within the organization being assessed. These internal team members serve a vital purpose in the appraisal. They serve as guides by helping the entire team understand how the organization s work maps to CMMI. Without internal people on the team, appraisals would take much more time and cost more money on the average than they do.

^[49] From SEI s Web site. Go to http://seir.sei.cmu.edu/pml/for more information.