DETERMINE THE STARTING POINT FOR CMMI PROCESS IMPROVEMENT


The starting point for a CMMI process improvement project consists of two parts :

  1. Things the organization already has in place.

  2. Things the organization does not have in place and needs or wants.

The organization must have information on both aspects to form an accurate picture of the starting point for process improvement and an accurate starting point is critical to developing realistic plans. Finding out what the organization already has in place is addressed in the following section, What the Organization Already Has. Figuring out what it needs follows that in the section, What the Organization Needs or Wants (but Does Not Have).

Here s some bad news: Understanding an organization s true, real, or exact starting point for process improvement is not possible. The accuracy of the starting point is relative to the best information that can be gathered at a point in time. The accuracy of the information you gather is a function of how much money and time the organization can spend in gathering it and the skills and expertise applied to that discovery. Your characterization of the organization s process capability or organizational maturity changes with time, not only because the processes and their implementation are constantly changing, but also because you are perpetually discovering things about the organization that change the original characterization. Inferences or assumptions made based on things observed last year turn out to be false in the light of newly discovered information this year. (The information was there last year, you just didn t see it.) As more information about an organization surfaces, the context of original observations also changes. What once was fact is no more. Something else you need to know but may not like to hear is that organizations process capability or organizational maturity can diminish more easily than it can improve and few people will even notice.

From a process improvement planning and implementation perspective, it s important to understand or at least acknowledge the relativity of the starting point. Remember that your process improvement plans are realistic based on an accurate picture of the then-current state. So if the organization s now-better-characterized-current state differs significantly from the then-current state, the plans must be revised accordingly . This means that the process improvement plans are fluid and constantly changing, which again is not all that different from the iterative planning for software and systems projects.

If a moving starting point and a moving target drive you mad, you might consider a different line of work. At least that has been my thought several times as a consultant. In our TARIF client, we conducted a fairly thorough baseline appraisal [at the time, equivalent to the Standard CMMI Appraisal Method for Process Improvement (SCAMPI Class B)] [3] to ascertain the organization s process implementation. One of the areas in which the organization initially appeared to be strong was project monitoring and control. Direct evidence was corroborated by indirect evidence and affirmed via interviews. The story of monitoring and control in TARIF looked and sounded coherent and consistent and we based our initial judgments and plans accordingly.

In the ensuing year that followed the baseline appraisal, we would periodically witness behaviors or artifacts which should have made us challenge our initial characterization of PMC in TARIF but did not because ” like everyone ” we wanted to be right. But when enough facts fly in the face of a theory, the theory loses unless you re insane. We finally had to reset our view of PMC in the organization. Practices that we had deemed implemented (colored green in charts) turned to not implemented (colored red). You ll always have some explaining to do when charts turn from green to red. We had to go to the client and admit that we knew now what we didn t know then and that the plans had to change. We had to admit to being smarter today than we were yesterday and that we might be smarter tomorrow than today. If a moving starting point and a moving target drive you mad, you might consider a different line of work.

Using Appraisals to Determine the Current State

One of the more useful models to come from SEI is the life cycle for process improvement known as Initiating, Diagnosing, Establishing, Acting, and Leveraging (or Learning), which is commonly referenced by its acronym, IDEAL SM . [21] Figure 3.2 shows a picture of the IDEAL Model.

click to expand
Figure 3.2: IDEAL Model

Many organizations use appraisals as the means to implementing the Diagnosing phase of IDEAL. Appraisals, particularly those that conform to a published methodology such as SCAMPI, are usually effective at yielding an accurate characterization of an organization s current process capability or organizational maturity. However, appraisals, particularly SCAMPI Class As, can be very expensive and highly disruptive to the organization s core operations.

SCAMPI appraisals can be conducted in one of two modes: (1) Discovery or (2) Verification. In Discovery mode, the primary objective of the appraisal is to find (hence, discover ) in the organization, processes and evidence of the use of those processes, and map that evidence to the practices in CMMI. A Verification form of SCAMPI assumes that the organization has already gone through discovery and has already mapped evidence of the use of processes to CMMI. It also assumes the organization possesses indigenous knowledge of CMMI and how to implement CMMI-based processes.

The default mode seems to be Verification and most process improvement consultants I know, such as SEI-authorized lead appraisers, will try to talk you into a Verification form of a SCAMPI appraisal. The cynical side of me wants to believe that it s because Verification appraisals are the quickest and most profitable in this business, but let s pretend there s no basis for such cynicism.

There are many important factors that should be considered when making a business decision as important as conducting appraisals. Like most other decisions in process improvement, the only right answer is that which is right for your organization. Remember that, besides a SCAMPI appraisal, there are other ways of accomplishing the goal of gathering factual process information from your organization. The other ways are described in the following sections. Also, before you hire a consultant or vendor to advise you, read Chapter 6 ” Acquiring Process Expertise and Tools.

What the Organization Already Has

Let s just accept for the sake of discussion that, even though you ve tried to convince your management to set business goals for the process improvement effort, what your bosses really want you to do is implement CMMI. Oh, what to do, what to do so many practices to implement and so little time! You still need to know where to start. Are you simply going to assume that your organization isn t doing anything that relates to CMMI and that you re going to have to plan and implement all of the practices? If you do, you could be wasting a great deal of your time and effort and the organization s money.

start sidebar
Natural Change

It would be extraordinary to find a software or systems organization that has been around for any amount of time that isn t already doing things that could be mapped to CMMI practices. Good software and systems management practices and business practices naturally evolve in most software and systems organizations. So, before you begin planning tasks , before you spin people up to start writing lots of policies and procedures, sit quietly and consider the possibility that the journey has already begun, and that you re simply not yet aware of that fact. Read Chapter 1 ” News Flash! There Is a Level 1! That information will help you understand the nature of determining the real starting point for process improvement in your organization.

end sidebar
 

Call it what you will ” gap analysis, evaluation, assessment, appraisal, etc. ” you need to perform some kind of discovery and analysis to find out what things your organization is already doing and how those things map to components of CMMI. The analysis of where your organization stands against CMMI can be rigorous or less rigorous , expensive or relatively cheap.

One of the first things you should do is find out if the organization has recently (within the past three to six months) conducted some type of method-based appraisal such as a Software Capability Evaluation (SCE SM ), [48] a CMM-Based Appraisal for Internal Process Improvement (CBA IPI), [5] or a SCAMPI. If you do find that some type of appraisal was recently conducted, and if the organization has not changed dramatically ( reorganized) since that appraisal, you can reasonably assume that the results of that appraisal are still valid and can be used for process improvement planning. If more than six months have lapsed since your organization s last appraisal, if it has never conducted one, or if it has recently undergone significant changes, you reduce the risks to your project by assuming that you need to get a fact-based understanding of the organization s current state with regard to process implementation. An appraisal to determine the organization s current strengths and weaknesses in the context of CMMI is typically referred to as a baseline appraisal because it establishes the baseline or starting point for process improvement work.

If the organization can afford the cost and interruption of planning and performing a SCAMPI appraisal, then that is what it should do to get the most comprehensive characterization of the current state. Alternatively, you and perhaps a few other people can appraise your organization as a routine, ongoing activity that we call in this chapter, Everyday Appraisals.

People have often asked me, when should our organization get appraised? My answer is always the same, every day! I encourage organizations to do what I did as an employee of organizations involved in process improvement: devote a little time each day or each week to collecting evidence of process instantiations and mapping them to the CMMI practices.

In almost any role in any company in the postmodern world, you will receive from 10 to 60 e-mails each day. Inevitably, some percentage of these e- mails or their attachments will be either direct or indirect evidence of the instantiation of a process either at the project level or an organization level. But most people don t hang on to such items. Instead, they start collecting and organizing such evidence about three months prior to a scheduled appraisal and such activity is often hectic and spastic because the organization is in a crunch to prepare for the appraisal. It doesn t have to be that way.

I recall one year in CSC during which I was the project manager for the process improvement project of an organizational unit. Starting about two weeks before a CBA IPI for CMM Level 3, senior managers would frequently pop into my cube and ask why I wasn t frantically preparing for the assessment. Weren t there any emergencies? Weren t there any fires to put out? Why were people not scurrying about doing things to get ready? Why weren t people pulling all-nighters? Why was there no panic? My calm ” the lack of quivering and knee-jerking in my demeanor ” made the bosses fret even more.

Well, my manager at the time (who later became my business partner) and I had decided long ago that we weren t adrenaline junkies and that crises were just one of the many manifestations of poor planning, poor organizational skills, and lack of discipline; in other words, low maturity behavior. The appraisal had been planned, the evidence had been prepared, and all commitments had been secured from all participants and stakeholders weeks before the start of on-site appraisal activities. In our view, panicking was worse than pointless; it would have been an indicator of a low maturity organization.

This section reveals some techniques you can use to conduct your own informal, relatively inexpensive analysis of the organization s processes against CMMI practices. These techniques can only be implemented by people who possess a very good, in-depth understanding of CMMI and a deep knowledge of their organization. If you or the organization does not have this knowledge, you should consider hiring external expertise to plan and lead the baseline appraisal. Read Chapter 6 ” Acquiring Process Expertise and Tools ” for more information on hiring and using process and CMMI consultants.

If you already know the current state of your organization in terms of CMMI-based processes, you can scan the rest of this section or skip it and go to What the Organization Needs or Wants (but Does Not Have) in this chapter to learn how to determine the other half of the true starting point.

Everyday Appraisals

As pointed out earlier, every day in your job people exchange information, much of which indicates the use of organizational, group , or team processes. Everyday Appraisal is the systematic monitoring of that information stream and, using CMMI knowledge and experience, netting those pieces of information that indicate process instantiation. Everyday Appraisal accomplishes three goals simultaneously with the same effort:

  1. It builds a picture (over time) of the organization s state of process implementation.

  2. It uncovers and reveals the undefined processes that people are following to perform their work; processes which should be the basis for the defined processes that come later (see Make the Process What People Do in Chapter 5 ” Five Critical Factors in Successful Process Definition).

  3. It contributes to the collection of Practice Implementation Indicators (PII) that the organization will eventually need to conduct an appraisal that is compliant with the Appraisal Requirements for CMMI (ARC) and SCAMPI.

Everyday Appraisal uses many of the same skills and techniques that are used in an ARC-compliant appraisal, but is less structured, less intrusive , and is performed continuously by one or a few people over time. When appraising or measuring an organization against CMMI, the mistake people make most often is they start looking at documentation for terms used in the model and they talk to people and ask them questions, again in CMMI terms, about their work. The reason this approach doesn t work is because project managers, software developers, system engineers , configuration specialists, architects , and other people in managerial or technical roles don t naturally work, speak, or write in CMMI vernacular. Nor should they; that is not their job.

Listen, Don t Talk So Much

In the course of normal work, we all have conversations with people every day. One practice I have found personally useful to learning, especially when I was new to an organization, was to engage people in conversations about their roles and their work. In Everyday Appraisal, one of the routine activities is engaging people in casual conversations and doing more asking and listening than talking.

The following dialogue represents a typical information-gathering interview conducted by someone who is trying to measure his organization s implementation of the REQM PA in unnatural terms. (PI = Person in process improvement role.)

PI PERSON: Do you develop an understanding with the requirements providers on the meaning of the requirements?

DEVELOPER: Yes. [She doesn t even understand the question. It s not likely that she knows who the requirements providers are or that she even knows what is meant by requirement, but she instinctively figures out that the desired answer is yes. ]

PI PERSON: (not much wiser) How is bidirectional traceability maintained between the requirements and the project plans and work products?

DEVELOPER: We use our configuration management tool for traceability and we write software we don t have any work products. [She really has no idea what she is being asked, but she s not about to show that she doesn t understand.]

PI PERSON: (clueless) How do you obtain commitment to the requirements from the relevant stakeholders?

DEVELOPER: Oh, we re totally committed to developing the highest quality software. Uhm I m late for a meeting and got to run. Bye.

It s not too difficult to see what went wrong in the above conversation. It may seem like an exaggeration, but it s unfortunately close to dialogs I have witnessed. The process person, who s obviously somewhat of a novice at his craft, is asking his questions by reading almost verbatim the REQM practices in CMMI. The dialog was a failure from two perspectives:

  1. The PI person failed to obtain any useful information about how the organization manages requirements.

  2. The PI person most likely alienated the developer. His unnatural, model-based questions probably made her feel inept and she will avoid interaction with the process geek in the future. One

  3. opportunity is lost and future opportunities are now at risk.

In process work, always try to avoid asking questions in terms that solicit a binary yes or no response. People are smart; it will take them no time at all to realize that the best answer is almost always yes. And once you ve received a yes answer, it becomes more difficult for you to ask how because then it is almost as if you don t believe their yes.

In the second question, the PI person continues his questioning in the language of CMMI. The developer guesses at the meaning of terms such as bidirectional traceability and work products, but her definitions undoubtedly differ from those of the process person. Her pride gets the better of her (as it often does with most of us), so she doesn t admit that she doesn t understand these terms. Instead, she makes assumptions about their meaning and context and answers accordingly. The process person will go away with the impression that the organization is not managing requirements when, in reality, he has no factual information supporting that impression ; he didn t ask good questions and he didn t ask them well.

Now, let s take a look at the same interview scenario with the process improvement person trying to get the same information as before, only this time he s asking the questions in a way that will yield a more accurate picture of what s going on in the organization.

PI PERSON: In your own words, tell me how you know what you should be working on how do you know what it is you re supposed to be coding?

DEVELOPER: Oh, our team leader e-mails us these ACRs ” Application Change Requests ” that get assigned to our department by the Change Board. The ACR describes what needs to be changed in the product, you know, like adding some type of feature or functionality.

PI PERSON: When you get these ACRs, does anything else change besides the code?

DEVELOPER: Yep before we can begin working on an ACR, we first have to do an analysis and tell our team lead if the change affects other components or if it affects the original design or architecture of the application. We also have to tell our lead how long it will take to implement the ACR.

PI PERSON: How do you know that it s okay to actually implement the ACR?

DEVELOPER: Well, we have to write up our analysis of the change and the estimate on the ACR form. Our lead then takes that back to the Change Board, which has to approve it before we can start working on it.

PI PERSON: Who is on the Change Board?

DEVELOPER: Oh, lots of people representing different groups like the customers, senior management, and companies we subcontract work to.

Notice how the information gathered by this interview paints a totally different picture of requirements management in the organization than the previous interview? It also uncovers clues about processes in other areas such as CM and PP and the involvement of stakeholders (GP 2.7). The results are different not because the second interview is going after different information, but because the second interview follows a different approach, one that uses language that is more natural to the respondent, the developer.

In the first question, the PI person doesn t make the mistake of assuming that the developer knows anything at all about CMMI, nor does he pose the question to solicit a yes or no answer. Instead, he simply asks an open -ended question to get the developer to talk about how she knows what to work on. The question is asked in a very natural, conversational style. You could even put the person being questioned more at ease by empathetic techniques: I m kind of new here and I can t figure out what it is we re supposed to work on; where does that information come from?

By the second question, our rapidly learning PI person has picked up on the developer s own language by asking the question using the acronym ACR; I m now one of you because we speak the same language. The PI person also makes the reasonable assumption that the developer uses ACRs to change code and uses that to lead the developer down the path of thinking and talking about what else gets changed by the ACR. A wealth of information comes back from the developer in response, including information related to process areas other than REQM mentioned earlier. Finally, in the third question, the PI person is asking about requirements review and approval, but structures the question in language that is both conversational and nonthreatening.

Two conversations between the same two characters yield dramatically different results all because of the approach, the style, and the language. The second dialog is an enormous success because the PI person comes away with a treasure of information and clues to more information. He has also established a collaborative, trusting relationship and has banked some emotional capital. Does CMM or CMMI teach a person how to do this: how to be a natural conversationalist or how to become a trusted advisor? It does not. So you might put some thought into the inventory of skills possessed by you and other change agents in your organization.

Reviewing Documents with a Natural Eye and an Open Mind

Another way to measure where your organization stands in terms of CMMI is to review documentation. As it is with talking to people, one of the common mistakes made by process people is that they tend to view the software development world through a CMMI-colored lens. Always remember, the model is an abstraction of the real world, not the other way around (unless, of course, you re insane).

In terms of process, there are essentially two types of documents (often called artifacts ). The first type of artifact is one that provides people with some form of instruction or direction, either at the organizational or project level. Such artifacts are often generically called processes. The second major type of artifact is often called implementation, instantiation, or evidence-of-use. These are the things that get produced when people follow the processes. One analogy that might help you distinguish the two types of artifacts is baking. When you bake a cake, the recipe (process) and the cake pan or mold (template) constitute the process artifacts. The result ” the completed cake ” is the implementation artifact; it is evidence-of-use that you followed the processes. The recipe and the mold are not unique to any single cake, rather they can be used to produce many cakes. The cakes are the work products; the outputs of baking.

Within the category of implementation or instantiation artifacts, there are two subcategories which are particularly useful in SCAMPI appraisals: direct and indirect evidence. Direct evidence is the direct result or output of performing a process or procedure. For example, if an organization has a process for developing a project management plan, the direct evidence of that process being followed would be a project management plan built and used by a software or systems project. Indirect evidence includes artifacts that are natural byproducts of people performing a process, but not the direct or intended result. In the example of the project management plan, indirect evidence would include items such as meeting minutes or action items that reflect people discussing, building, or using the project s plan.

Now, let s use an example to illustrate an approach to reviewing a naturally occurring document in an organization. The example we will look at is an e-mail note (see Figure 3.3) from a project manager (Adam) to his senior manager (Tricia).

click to expand
Figure 3.3: Sample E-Mail Note Containing Use of Process Evidence

Notice the numeric tags (in bold type) inserted next to words and phrases in the note. These tags correspond to items listed in Table 3.1, which provides an analysis of how those references relate to CMMI practices.

Table 3.1: Mapping a Sample E-Mail Note to CMMI

Tag

Analysis and Clues

Possible Related CMMI PAs and Practices

1

The mention of training needs and going to a class is a clue that training plans and records may exist. Find out the type of class being attended to have an indirect evidence artifact for GP 2.5.

OT: SP 1.1, SP 1.2, SP 2.1, SP 2.2

GP 2.5

2

This e-mail note itself constitutes a project status and indicates that people communicate the status of their work to their managers. An analysis of the full distribution of the e-mail note might lead to an understanding of how relevant stakeholders are kept informed of project status and work.

PMC: SP 1.6

GP 2.10: REQM, PP, PMC, OT

3

Locked down could be local vernacular for something being put under change control or configuration management. The mention of a functional design document should be investigated for its applicability to practices in Requirements Development and Technical Solution.

CM: SP 1.2, SP 2.2

RD: SP 3.2

TS: SP 2.1, SP 3.1

GP 2.6: RD, TS

4

The reference to a customer rep implies the organization has a sales or marketing unit. This sentence implies coordination with that marketing unit and warrants further investigation for links to Integrated Teaming, Integrated Project Management, and stakeholder involvement. It s also a clue of where to look for more information on how requirements come into the project.

REQM: SP 1.1, SP 1.2, SP 1.3

GP 2.7: REQM, PP, PMC

5

This statement indicates the project manager is cognizant of a change to the requirements.

REQM: SP 1.3

6

The fact that the project manager recognizes that the new fields affect a change to other work products or system functionality (accounting module) indicates that some level of requirements traceability may be performed.

REQM: SP 1.4, SP 1.5

RD: SP 1.1, SP 2.1, SP 2.3

PP: SP 2.6

7

The coordination with Finance provides a clue that there may be native processes or protocols in place that indicate Integrated Teaming, Integrated Project Management, and stakeholder involvement.

PMC: SP 1.2, SP 1.5 GP 2.7: REQM, PP, PMC, RD

8

As with Tag 1, the mention of training needs and going to a class is a clue that training plans and records may exist. In this context, it is reasonable to presume that Mike may be acquiring training for his role on the PCCB (configuration management or change control).

OT: SP 1.1, SP 1.2, SP 2.1, SP 2.2

GP 2.5: CM?

9

It would be an informed guess that PCCB in the context of this note is an acronym for Project (or Product) Change Control Board, which indicates a change control or configuration management function is in place.

CM: SP 1.2, SP 1.3, SP 2.1, SP 2.2

GP 2.3: CM

GP 2.4: CM

10

Sheila has somehow confirmed that the new feature descriptions meet some acceptance criteria. The possible existence of acceptance criteria should lead you to look for an organizational standard for requirements, which then should lead you to see if Sheila s confirmation was performed via some type of peer review.

REQM SP 1.1, SP 1.3

RD SP 1.2

VER SP 1.3, SG 2

11

Doug and Nicole have looked at the new feature descriptions (read: new or changed requirements) and have developed some type of estimates, presumably to understand the impact of the changes.

REQM SP 1.3

PP SP 1.2, SP 1.4

PMC SP 1.1

12

Another clue of the existence of a change control or configuration management process and resources. If the PCCB comprises cross-functional units and relevant stakeholders, you re also looking at stakeholder involvement (GP 2.7) for both REQM and PMC.

REQM SP 1.2, SP 1.3

CM: SP 1.2, SP 1.3, SP 2.1, SP 2.2

GP 2.3: CM

GP 2.4: CM

GP 2.7: REQM, PMC

13

Versioning of the project plan indicates a project plan exists and that changes to the plan are managed and controlled.

PP SG 2

PMC SG 1, SG 2

GP 2.6: PP, PMC

14

Project risks are at least being considered. This statement should lead you to look for a project risk management plan and updates to that plan resulting from changes to requirements or project parameters.

PP SP 2.2

PMC SP 1.3

RSKM SG 3 (maybe)

15

Having the team take a look at the changes for a risk analysis indicates there may be a risk management plan and that relevant stakeholders are involved in risk management.

PP SP 2.2

PMC SP 1.3

RSKM SG 3 (maybe)

GP 2.7: PMC, RSKM

16

Having some team members take a look at the risks and cost it out gives us a picture that changes to the project are being somewhat managed and controlled, even if not in accordance with a defined process (GP 3.1).

PP SP 2.2

PMC SP 1.3, SP 1.4, SP 2.1, SP 2.2

RSKM SG 3 (maybe)

GP 2.7: PMC, RSKM

17

It would be difficult to imagine that this project could so accurately manage changes to cost and schedule if those two artifacts didn t exist in some form. Given the context of the statement that changes to cost or schedule would be negotiated with Terry should be a strong clue that project commitments were established with relevant stakeholders and that they are monitored .

PP SP 2.1, SP 2.3, SP 2.6, SP 3.3

PMC SP 1.2, SP 1.5, SP 2.2, SP 2.3

GP 2.7: PP, PMC

18, 19

Obtaining customer sign-off on changes to the plan and then communicating those changes are strong indicators of stakeholder involvement in the project.

GP 2.7: REQM, PP, PMC, RSKM

20

Jackpot! This sentence smacks of objective verification of plans and processes. If that beautiful smell doesn t lead you to look into the existence of PPQA practices and processes, I don t know what will.

PPQA

GP 2.9: PP, PMC, RSKM

21

Mention of a baseline of anything is a clue that configurations have been planned and are being managed. In this case, the product s feature set was apparently base-lined and probably so too were other things such as the requirements and the plans.

CM SP 1.1, SP 1.3, SP 2.1, SP 2.2

GP 2.6: REQM

22

Churn rate is a term classically associated with measuring the rate of change (volatility) to requirements. This is a good sign that the project (and maybe the organizational unit) has defined and planned measures to quantitatively understand the efficacy of their processes.

MA SP 1.2, SP 1.4, SP 2.1, SP 2.2, SP 2.4

GP 3.2: REQM

23

The observation that Adam knows that the churn rate is still below acceptable thresholds might even indicate that, at least in some areas, they are quantitatively managing their processes.

OPP: SP 1.2, SP 1.3, SP 1.4

The e-mail note in Figure 3.3 is, on the surface, ordinary and mundane, not unlike hundreds of e-mail notes, voice mails, and other types of communication that pass between people in any organization every day. However, such routine artifacts can be rich sources of information about process implementation in the organization. Even though Adam never mentions a CMMI process area or practice by name , his seemingly innocuous note is virtually dripping with clues to the use of processes (perhaps undocumented) in the organization.

This sample e-mail note is just one example of the treasures of information that float by you every day. Look at the information harvest just from this one note. A 291-word document yields at least 23 indicators or evidence of process implementation in 13 CMMI process areas and dozens of specific and generic practices.

Granted, not all of the hundreds or thousands of pieces of information that cross your desk or desktop every year will be as replete with process clues as our example, but many of them will. So now ask yourself again: are you going to wait until a month or two before your organization s appraisal to start collecting evidence of processes and mapping them to CMMI?

And again: does CMMI teach you how to view a document with an open and curious mind? It does not, yet that is a skill you will need for Everyday Appraisals.

Mapping What You Discover to CMMI

When you capture information from Everyday Appraisals, you will want to record it to begin building the profile of your organization as measured by CMMI. Tracking and recording what the organization does and correlating those process instantiations with CMMI practices is usually called mapping. There is almost an infinite number of ways to perform CMMI mapping, ranging from using databases and automated document management systems that cost hundreds of thousands of dollars to simple Microsoft (MS) Word or Excel tables that you can build yourself.

Continual mapping of your organization s process implementation results in two big benefits:

  1. You and others in the organization will always know, at any given time, the extent to which you have implemented processes that are consistent with or correlate to CMMI practices. I don t particularly care for the phrase CMMI compliance, but essentially a current CMMI map will tell you how compliant your organization is with CMMI. Thus, some people refer to their CMMI mapping document as a compliance matrix.

  2. When the time comes to begin planning and preparing for an appraisal, you won t have to scramble to find the direct and indirect evidence the appraisal team will need to examine. You will know the names of the artifacts, where they are located, and to which CMMI practices they apply.

Table 3.2 shows a very simple form of CMMI mapping for the Requirements Management (REQM) process area.

Table 3.2: Example Simple Mapping of Artifacts to CMMI Practices

Goal

Practice

Practice Description

Direct Evidence

Indirect Evidence

SG 1

SP 1.1

Develop an understanding with the requirements providers on the meaning of the requirements.

  1. RTM showing original requirements for project Sandstorm and derived requirements

  2. Project Bluebook MOA

Minutes and action items from 6/26/03 JAD session with customer reps
 

SP 1.2

Obtain commitment to the requirements from the project participants.

  1. Sales Data Migration project requirements signoff sheet

  2. COLDD conversion SRS

3. Meeting minutes from requirements review with Marketing
 

SP1.3

Manage changes to the requirements as they evolve during the project.

  1. SDM Requirements Document revision history

  2. RMS database change audit report

  3. ECR database records

  1. CCB meeting minutes and AI log

  2. RMS database change audit report and CCB approval records

  3. COLDD conversion SRS updates

 

SP1.4

Maintain bidirectional traceability among the requirements and the project plans and work products.

  1. RMS database (all projects)

  2. COLDD RTM

  3. IT sustaining engineering work order log

  1. CCB meeting minutes and AI log

  2. COLDD project review meeting minutes (Requirements section)

  3. Project status reports (all projects except COLDD)

 

SP1.5

Identify inconsistencies between the project plans and work products and the requirements.

  1. SDM requirements peer review defect log

  2. COLDD SRS QA audit report

 

As stated, CMMI maps such as the example in Table 3.2 can be very simple, inexpensive, and easy to use. They can also be very complex, expensive, and require a learning curve to become a proficient user . The trade-off between the two ends of that spectrum is usually the utility of the tool and its ability to yield a variety of information for different needs. The MS Word table shown in Table 3.2 is cheap and easy to build and populate, but it is very limited in its ability to yield summarized management information that indicates the organization s overall level of process improvement.

Somewhere between the simple table and the $150,000 database will be the form of CMMI mapping that is right for your organization.

Figure 3.4, Figure 3.5, and Figure 3.6 show examples from a sophisticated MS Excel-based CMMI mapping tool that I developed (Natural SPI CMMI EZTrak) and have used with success in many client organizations. As with most such tools, it has evolved over many years to incorporate experience and lessons learned from its use.

The matrix shown in Figure 3.4 is just a portion of one of many worksheets in the EZTrak tool (there is one worksheet for each PA). As you can see there are columns that provide a count of the number of artifacts (direct and indirect evidence) and the number of organizational process assets. The final column uses a proprietary, tailorable algorithm to determine the extent to which a particular practice is implemented (I), partially implemented (P), or not implemented (N).

click to expand
Figure 3.4: CMMI Mapping (1 of 3)

The picture in Figure 3.5 shows what is frequently called a Quilt Chart. Essentially, this worksheet automatically extracts data from other worksheets in the tool to build the grid you see, which provides a view of the organization s implementation of CMMI practices in one view.

click to expand
Figure 3.5: CMMI Mapping (2 of 3): Quilt

The chart shown in Figure 3.6 is automatically generated from the data contained in the Quilt Chart shown in Figure 3.5. This view of CMMI progress and status is very popular with senior and executive managers.

click to expand
Figure 3.6: CMMI Mapping (3 of 3): Graphs

A Word of Caution about CMMI Mapping

No one is born with the ability to extrapolate the content of documents and conversations to CMMI process areas and practices. Proficiency in this skill only comes with a very strong understanding of CMMI, extensive experience using this model to implement process improvement in a variety of system engineering and software environments, and reading comprehension skills. Many artifacts will serve as direct evidence for a subset of practices and simultaneously serve as indirect evidence for other practices. Knowing the difference is not intuitive. For the novice, it becomes too easy to look at something and accept on faith that it maps to a component of CMMI. To do so is dangerous in that it can give you a false positive with regard to the organization s level of CMMI compliance.

If your organization is relatively new to CMMI, mapping your organization s process implementation to the model may be a skill that is more cost-effective to outsource than to try to grow internally. For more guidance, read Chapter 6 ” Acquiring Process Expertise and Tools.

What the Organization Needs or Wants (but Does Not Have)

Let s assume you do have a pretty good understanding of how the processes and practices in your organization compare with the practices in CMMI. However, the information you probably don t have at your fingertips is what the organization needs or wants, but doesn t have. Here is a bad thought you really want to stay away from: Let s go do the CMMI practices that we know we re not doing. Why is that a bad thought? How do you know that implementing the CMMI practices the organization is not currently doing would be good for the organization s business? How do you know those practices would benefit anyone in any way? How do you know that implementing those additional process areas or practices won t be a useless waste of time and money?

Here s the good news: it is not solely the responsibility of the people with process responsibility (i.e., the SEPG) to determine the organization s needs and wants. This also happens to be the bad news. Knowing what improvements, process or otherwise , the organization needs to make is based on the organization s goals and strategy. By definition, establishing the organization s goals and strategy must involve the organization s leadership such as executives and senior managers.

Now you have a great opportunity right in front of you! This is a golden opportunity to get the organization s leadership involved in the process improvement effort by asking for their help in understanding the strategy and goals. For more information on the criticality of starting with strategy and goals, read Everything Starts with the Strategy and It Starts at the Top in Chapter 1 ” News Flash! There Is a Level 1!

GQM Lite: A Quick-Start Approach for Process Improvement

One technique I ve used very successfully with clients is a variation of the Goal-Question-Metric (GQM), [19] which Natural SPI has dubbed GQM Lite. GQM Lite is a relatively low-cost methodology for quickly determining the organization s needs and how those needs can be satisfied with process improvement tasks.

In its most simple form, GQM Lite is a four-step process:

  1. Establish the organization s business goals. Goals are statements of desired states in different business perspectives such as customer focus, employee growth and development, financial results, market share, operational excellence, and innovation. The organization s leadership must be personally involved in the establishment of the goals.

  2. Define the questions you would need to ask to know if the goals are being met.

  3. Define the measures that will answer the questions defined in Step 2.

  4. Define the tasks and activities that need to be performed that will yield the measures defined in Step 3.

Establishing the Goals

There are almost an infinite number of ways an organization s leadership can set the strategy and establish the goals for the organization and, with any luck, you won t have to be involved in this task because it is absolutely the hardest thing to do. If the leadership of an organization has not defined or cannot articulate a vision, mission, strategy, or strategic goals, then that leadership doesn t know why its organization exists. And if the organization doesn t have a reason for existing, its potential market and customers surely won t know either. And if the organization doesn t have a vision or mission or strategy or goals, CMMI won t help because process improvement is a means to an end, not the end itself. Leading executives through the job of establishing goals for the organization can be done, but that process warrants its own book and is not discussed herein.

Something that does seem to occur frequently in organizations, especially in government and defense contractor organizations, is that a vision or mission statement will get written but go no further. That s actually good news because you and other stakeholders can at least use a vision or mission statement as a starting point, deconstruct it, and come up with some reasonable business goals.

Such was the case with one client, the Range Instrumentation Technical Support (RITS) organization located at Eglin AFB, Florida. They had a mission statement, which is shown in Figure 3.7. In the figure, key words are circled, which gave us clues as to what was important to the people who wrote the mission statement.

click to expand
Figure 3.7: Mission Statement Deconstruction

We then led a group of stakeholders to assign meaning and relevance to the four words or phrases that we culled from the mission statement, which led to the following thoughts:

  • Enhance: Improve (make better) capabilities, systems, skills, processes, and quality (also see Superior Instrumentation below). We cheated and just looked this one up in a few dictionaries. Reuse.

  • Development: Enhancement proposals (only?); we left the definition of this term TBD for the time being.

  • Sustainment: Incremental defect correction and (minor enhancements?).

  • Superior Instrumentation: We knew what instrumentation means, but what did superior mean? We didn t know what it meant to someone else (but then probably neither did they), so we decided that we get to define it!

Focusing on these words and phrases in the mission statement, the client organization was then able to define some long-term business goals, the achievement of which would fulfill the mission.

Getting from Goals to Measures

Okay, so either by miracle or a stroke of fortune or genius, let s say the organization has business goals, and you and the other process improvement people know what those goals are. Now it s time to put some heads together and answer two questions:

  1. What questions would we have to ask ourselves in the future (or what indicators would we have to see) that would tell us that the goals are being achieved?

  2. What measures would answer the questions that we re asking to find out if the goals are achieved?

To accomplish this step, we have evolved and implemented a one-page form that walks people through the process of deconstructing organizational goals (G) to questions (Q) and then to measures (M). The form, along with a 45-minute tutorial on GQM Lite, enables people to easily define information in these areas:

What questions should be asked to determine if goals are being achieved?

Which indicators (measurable concepts) answer the questions? What derived measures can be used to form the indicators? Which base measures are used to calculate the derived measures? Who collects the measures?

How are the measures collected and how often? To whom are the measures reported ?

How are the measures used? What methods are used to analyze measures and indicators?

As experience indicates, GQM Lite is both effective and efficient. Following the GQM Lite tutorial, the seven member RITS Engineering Process Group used the GQM Lite form to define all their process measures in a total of about 14 effort hours. I ve witnessed process improvement teams or process action teams in other organizations agonize over a measurement program for months and still come up empty-handed.

Figure 3.8 shows one of the interim outputs from using GQM Lite to derive process improvement plans from organizational business goals; it is what success using GQM Lite looks like.

click to expand
Figure 3.8: TYBRIN Procurement Process Improvement GQM Lite Mapping

In TYBRIN Corporation, a fast-growing, successful defense contractor, we used GQM Lite with a group of purchasing agents to define the goals, questions, and measures for a revamping of their purchasing and subcontracting processes (SAM and ISM). Figure 3.8 shows the mapping between the procurement goals, questions (measurable concepts), and measures. The measures were then used to determine the tasks and activities for improving the procurement processes.

Getting from Measures to Process Improvement Actions: Managing Process Improvement Requirements

In software and systems delivery projects, the work between customer goals or high level requirements is the analysis, clarification , development, and management of requirements. Requirements management is also the work that needs to occur between establishing the goals and measures and then planning the process improvement tasks.

There is no reason I can think of why an organization shouldn t gather and document its requirements for undertaking an effort as costly as implementing model-based software process improvement. Having defined the business goals is the first step toward process improvement requirements, but the organization s senior leaders are not the only stakeholders in the process improvement project. For CMMI-based improvements to take hold in the organization and thrive, your project will need the support of many other stakeholders who will want to know what CMMI and process improvement are going to do for them. If you re trying to solve software/system development or integration problems with CMMI, you need to understand those problems because they are very likely to be barriers to the achievement of the goals.

The process for collecting requirements for process improvement is basically the same as collecting requirements for a software or systems project. You talk to the people in your organization ” let s call them customers just for discussion purposes ” who theoretically are supposed to benefit from process improvement. Ask them what they think they should get in return for investing time and energy into process work. By the way, just because you already know all the standard answers ” increased productivity, increased predictability, blah, blah, blah, don t bait or ask people leading questions. Don t ask, Wouldn t you really like to increase the productivity of your software developers? Duh. What do you think their answer would be?

In fact, forget about all the clich reasons for doing process improvement. When you talk to managers, project managers, and engineers and you ve convinced them that you really care about their opinion, you ll get some refreshingly creative responses to the question, What do you want out of process improvement?

I ve heard everything from, I m doing it because my manager wants it, (which, by the way, is rarely true), to, I m very frustrated with working 40 hours and not knowing where my time went I want to know why I can t get more done. As you gather requirements for process improvement, keep in mind that people will often say what they think is expected of them instead of giving you a candid answer. Try not to be na ve. The only real interest is self-interest. Be skeptical of responses to your questions that sound something like I want to do what benefits the organization. Yeah, right. When you start to get these kinds of answers, modify your approach to get people to trust you enough to tell you what they re really thinking and feeling.

One really good reason for collecting requirements for your process improvement project is that it helps ensure support and buy-in on the improvement activities. If people feel that there s some personal benefit in it for them, they re far more likely to support the initiative. Remember, people do not take action for no reason; everything we do is to either avoid pain or get pleasure or both ” process improvement is no different. So find out the problems that plague your organization or find out what would make work easier for people and then try to figure out if a CMMI-based process improvement is going to address any of those things.

However, when all is said and done, here s the very best reason for gathering requirements for process improvement: If you talk to lots of people and no one ” not even the leadership ” can come up with a single requirement for implementing CMMI, maybe you and your management should ask yourselves, So, why is it we re doing this? One of Natural SPI s principals, Donna Voight, would give you this harsh , but sound advice: if you don t have a client, go home. In other words, if your organization s process improvement efforts are not being expended to achieve a business goal or solve a problem, then you don t have a real job. Quit what you re doing and go find one.

In a typical CMMI process improvement project, the people responsible for process definition and implementation will generate lots of documents. As the CMMI project moves forward, you and others will want some assurance that what is being built is satisfying the process improvement requirements and goals.

With your process improvement project s goals and requirements documented, you now know the end-point for your project. You have a deliverable for which you can now do some estimating and planning. If your project is simply process improvement with no clearly defined goals or requirements, you d better be prepared to be in the job for a long time because you won t know when you re finished. Conversely, you might be in the job for a short time because, at some point, management will grow tired of spending money with no clear deliverables in sight and no measurable progress.

[3] Standard CMMISM Appraisal Method for Process Improvement (SCAMPISM), Version 1.1: Method Definition Document (MDD), CMU/SEI-2001-HB-001, Carnegie Mellon University, December 2001.

[21] McFeeley, B., IDEALSM: A User s Guide for Software Process Improvement, Technical Report CMU/SEI-96-HB-001, February 1996, Carnegie Mellon University.

[48] Byrnes, P. and Phillips, M., Software Capability Evaluation (SCE), Version 3.0, CMU/SEI-96-TR-002 , Software Engineering Institute, Carnegie Mellon University, April 1996.

[5] Dunaway, D. and Masters, S., CMM-Based Appraisal for Internal Process Improvement (CBA IPI): Method Description (CMU/SEI-96-TR-007) , Carnegie Mellon University, April 1996.

[19] Basili, Victor R., Software Modeling and Measurement: The Goal/Question/ Metric Paradigm, Technical Report, CS-TR-2956, Department of Computer Science, University of Maryland, College Park, MD 20742, September 1992.




Real Process Improvement Using the CMMI
Real Process Improvement Using the CMMI
ISBN: 0849321093
EAN: 2147483647
Year: 2004
Pages: 110
Authors: Michael West

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net