In an article titled "Escaping the IT Abyss," Dempsey et al. tell us the role in which project reviews play in the most effective IT organizations:
Reviewing overall IT performance helps you to distill lessons that will help you do better next time. Most companies claim to perform reviews, but few do so effectively. They may complain that they can't review big projects properly because "Most of the original people are gone or have moved on to different efforts," as one CIO told us, or because "The scope of the project has changed many times over the years," as another explained. Both these objections can be addressed if projects are kept short, given clear objectives, required to produce regular deliverables, and monitored continuously.
In the best IT companies, business people drive the projects and are on the hook for the results. Hard benefits are budgeted for, and results are measured at every stage. Post-implementation review is easier, since all the milestone results are available to help managers weigh outcomes against objectives. Lessons learned are built into subsequent projects.
As the quoted passage above implies, only by examining the past can organizations adequately prepare for the future. For example, an individual interviewing for a job claims to have twenty years of experience in a particular field. Following the interview, one of the interviewers asks the candidate if the individual indeed has twenty years of experience, or only one year of experience twenty times. If organizations do not contemplate and learn from both successes and failures, progress will be slow, or even non-existent.
An initial post-project review, or postmortem, is recommended as a starting point for the entire project review process. A project review as a whole is a post-milestone review meeting that formalizes the process of learning from past experience. A thorough project review should carefully analyze the project to identify strengths and weaknesses of the each development phases. Post-project reviews begin the process of incorporating best practices into an organization's development process so that the project team learns to identify and confront potential risks in future projects.
Numerous benefits, both individual and corporate, arise from conducting project reviews. As mentioned earlier, the most important benefits are self-examination, reflection, and resulting growth through the incorporation of best practices. These benefits ideally occur for both the individual and the organization. Additional benefits of project reviews are that they:
Dr. Joyce Statz of TeraQuest provided the Capability Maturity Model for Software information upon which this section is based. Since the mid-1980s, Carnegie Mellon University has provided several studies and models that describe organizations' progressions from "infancy" to "maturity." The Capability Maturity Model (CMM) for Software is one of several such models. CMMs support evolving capability for developing software, managing people, acquiring software, personal software process, and systems engineering. Many of these models share common features, although content and intended audiences vary.
Each model provides a structured view of its focus area, generally in a five-layer progression of increasingly sophisticated practices. Most CMMs intend to incrementally improve an organization's overall capability. Each layer of the CMM provides a baseline for improvement in established practices, as well as a basis for the next layer of mature practices. The CMM for Software comprises five layers, which are described in Table 14.1.
Table 14.1 Five levels of CMM for Software
|Level||Focus||Key process areas|
|5: Optimizing||Continuous process improvement||Defect prevention|
Technology change management
Process change management
|4: Managed||Product and process quality||Quantitative process management|
Software quality management
|3: Defined||Defined engineering process||Organizational process focus|
Organizational process definition
Integrated software management
Software product engineering
|2: Repeatable||Project management and commitment process||Requirements management|
Project tracking and oversight
Software subcontract management
Software quality assurance
Software configuration management
|1: Initial||Heroes||Extraordinary effort|
A complete discussion of the CMM is outside the scope of this book. Nevertheless, it is worthwhile to note that the Microsoft Solutions Framework is an excellent guide for organizations in following the progress from the initial stages of Level 1 to the higher maturity levels. For additional information on the CMM for Software, please refer to Mark C. Paulk, Charles V. Weber, Bill Curtis, and Mary Beth Chrissis' The Capability Maturity Model: Guidelines for Improving the Software Process (Addison-Wesley, 1995).
Our purpose in briefly discussing the CMM is to demonstrate the importance of project reviews in evolving an organization to a more efficient CMM maturity level. One characteristic of a CMM Level 1 organization is reinventing processes for each project. By conducting project reviews and implementing the reviews' results as standard practices, such an organization can avoid this inefficient, but common, problem.
Two key challenges for CMM Level 2 organizations are accurate project planning and effective project tracking. Using project reviews to compare plans with actual events and timelines enables such a team to hone its estimating and tracking skills. Over time, this organization can create an increasingly realistic sense of time and resources that project tasks require. Without a project review, it is likely that this team will repeatedly overestimate or underestimate project schedules and resources.
In the white paper Microsoft Solutions Framework and the Capability Maturity Model, Dr. Statz points out that project reviews are critical to reaching Level 3, as noted below:
Although most Level 2 organizational discipline is seen at a project level, members of an individual project can usually find good practices that could be useful in another project. Leveraging those best practices across the organization and defining processes that can be tailored to each project is the heart of Level 3 of the CMM.
Project teams generally accumulate a rich process history as they complete their project. They should keep this data in a repository for use by other project teams on future projects. The organization will develop an appreciation for common processes and advisors who can help tailor those processes.
At Levels 4 and 5, the fundamental purpose of the project review is to facilitate the feedback loop. As the team identifies problems, it can change its processes to eliminate or mitigate such problems in future projects.
The CMM for Software provides a solid measure of a development organization's effectiveness and maturity. The project review is a key tool in organizations' efforts to improve development processes and progress to more efficient levels of software development maturity.