Case for Project Review

In an article titled "Escaping the IT Abyss," Dempsey et al. tell us the role in which project reviews play in the most effective IT organizations:

Reviewing overall IT performance helps you to distill lessons that will help you do better next time. Most companies claim to perform reviews, but few do so effectively. They may complain that they can't review big projects properly because "Most of the original people are gone or have moved on to different efforts," as one CIO told us, or because "The scope of the project has changed many times over the years," as another explained. Both these objections can be addressed if projects are kept short, given clear objectives, required to produce regular deliverables, and monitored continuously.

In the best IT companies, business people drive the projects and are on the hook for the results. Hard benefits are budgeted for, and results are measured at every stage. Post-implementation review is easier, since all the milestone results are available to help managers weigh outcomes against objectives. Lessons learned are built into subsequent projects.

As the quoted passage above implies, only by examining the past can organizations adequately prepare for the future. For example, an individual interviewing for a job claims to have twenty years of experience in a particular field. Following the interview, one of the interviewers asks the candidate if the individual indeed has twenty years of experience, or only one year of experience twenty times. If organizations do not contemplate and learn from both successes and failures, progress will be slow, or even non-existent.

An initial post-project review, or postmortem, is recommended as a starting point for the entire project review process. A project review as a whole is a post-milestone review meeting that formalizes the process of learning from past experience. A thorough project review should carefully analyze the project to identify strengths and weaknesses of the each development phases. Post-project reviews begin the process of incorporating best practices into an organization's development process so that the project team learns to identify and confront potential risks in future projects.

Benefits of Project Reviews

Numerous benefits, both individual and corporate, arise from conducting project reviews. As mentioned earlier, the most important benefits are self-examination, reflection, and resulting growth through the incorporation of best practices. These benefits ideally occur for both the individual and the organization. Additional benefits of project reviews are that they:

• Provide project closure Closure is important if team members have spent time and energy on the completed project, and will begin another project immediately. Project closure is particularly important if the project team will soon dissolve. Official project closure will also help the team members to move on to something new.
• Provide a final outlet for team communication A project review can be cathartic for team members who have strong feelings about particular aspects of a project. If not expressed in a controlled manner and setting, team members may express such feelings later in less desirable circumstances, places, or means. In sharing their thoughts, team members should focus on the team's actions rather than on the individuals; furthermore, such a communication outlet should never be used to place blame or point fingers at any individuals or groups.
• Address the team's morale Project reviews may actually enhance team morale by allowing teammates to share positive as well as negative feelings, and also to offer praise throughout the team. Traditionally, software development has often been somewhat of a solitary effort; the software development team approach addressed in this book lends itself to shared responsibilities as well as shared praise for a successful project.
• Set best-practice baselines Future teams may also benefit from project reviews by accessing the current team's perception of the project's strengths and weaknesses. Such perceptions can provide the basis for creating best-practice baselines for software development within an organization. Project teams should remember the framework concept, which suggests that organizations must modify their practices to find the best methods and executions for their particular projects. In the early stages of developing a best-practice baseline, a good exercise for all development team members is simply to read project reviews from their organization's previous projects. To simplify access, these baselines should be gathered into a single location, often referred to as a best-practice guide or process library.
• Establish feedback loops All the project review output can be combined into a best-practice baseline and should be used as input for subsequent projects. Organizations should apply project review insights to improve future projects. Such a feedback loop marks an organization as being dedicated to improvement. Feedback loops are key features of the Capability Maturity Model for Software.

Capability Maturity Models

Dr. Joyce Statz of TeraQuest provided the Capability Maturity Model for Software information upon which this section is based. Since the mid-1980s, Carnegie Mellon University has provided several studies and models that describe organizations' progressions from "infancy" to "maturity." The Capability Maturity Model (CMM) for Software is one of several such models. CMMs support evolving capability for developing software, managing people, acquiring software, personal software process, and systems engineering. Many of these models share common features, although content and intended audiences vary.

Each model provides a structured view of its focus area, generally in a five-layer progression of increasingly sophisticated practices. Most CMMs intend to incrementally improve an organization's overall capability. Each layer of the CMM provides a baseline for improvement in established practices, as well as a basis for the next layer of mature practices. The CMM for Software comprises five layers, which are described in Table 14.1.

Table 14.1 Five levels of CMM for Software

Level	Focus	Key process areas
5: Optimizing	Continuous process improvement	Defect prevention Technology change management Process change management
4: Managed	Product and process quality	Quantitative process management Software quality management
3: Defined	Defined engineering process	Organizational process focus Organizational process definition Integrated software management Software product engineering Inter-group coordination Training program Peer reviews
2: Repeatable	Project management and commitment process	Requirements management Project planning Project tracking and oversight Software subcontract management Software quality assurance Software configuration management
1: Initial	Heroes	Extraordinary effort

A complete discussion of the CMM is outside the scope of this book. Nevertheless, it is worthwhile to note that the Microsoft Solutions Framework is an excellent guide for organizations in following the progress from the initial stages of Level 1 to the higher maturity levels. For additional information on the CMM for Software, please refer to Mark C. Paulk, Charles V. Weber, Bill Curtis, and Mary Beth Chrissis' The Capability Maturity Model: Guidelines for Improving the Software Process (Addison-Wesley, 1995).

Our purpose in briefly discussing the CMM is to demonstrate the importance of project reviews in evolving an organization to a more efficient CMM maturity level. One characteristic of a CMM Level 1 organization is reinventing processes for each project. By conducting project reviews and implementing the reviews' results as standard practices, such an organization can avoid this inefficient, but common, problem.

Two key challenges for CMM Level 2 organizations are accurate project planning and effective project tracking. Using project reviews to compare plans with actual events and timelines enables such a team to hone its estimating and tracking skills. Over time, this organization can create an increasingly realistic sense of time and resources that project tasks require. Without a project review, it is likely that this team will repeatedly overestimate or underestimate project schedules and resources.

In the white paper Microsoft Solutions Framework and the Capability Maturity Model, Dr. Statz points out that project reviews are critical to reaching Level 3, as noted below:

Although most Level 2 organizational discipline is seen at a project level, members of an individual project can usually find good practices that could be useful in another project. Leveraging those best practices across the organization and defining processes that can be tailored to each project is the heart of Level 3 of the CMM.

Project teams generally accumulate a rich process history as they complete their project. They should keep this data in a repository for use by other project teams on future projects. The organization will develop an appreciation for common processes and advisors who can help tailor those processes.

At Levels 4 and 5, the fundamental purpose of the project review is to facilitate the feedback loop. As the team identifies problems, it can change its processes to eliminate or mitigate such problems in future projects.

The CMM for Software provides a solid measure of a development organization's effectiveness and maturity. The project review is a key tool in organizations' efforts to improve development processes and progress to more efficient levels of software development maturity.