1.8 Quality Management

---

Quality management involves ensuring that the expectations and quality requirements of the project are understood and actively managed. Effective quality management contains six components:

I. Expectation management

II. Quality verification

III. Process management

IV. Metrics

V. Continuous improvement

VI. Rewards and recognition

The objectives of quality management are to ensure the delivery of a quality product that successfully realizes the intended benefits[4]. The product must meet all of the expectations of the project stakeholders, and it should uniformly apply the approved principles, measures, standards, and methods of software program management. The quality program should create an environment fostering continuous improvement and actively manage and continue to leverage the knowledge capital of the organization.

1.8.1 Benefits of Quality Management

The benefits of quality management are obtained from implementing a philosophy of 'build in quality' versus 'inspect for quality.' This really means one cannot rely on back-end inspections to achieve quality. The Project Manager is responsible for setting project expectations that ensure the project will deliver quality work. The expectation of 'get it right the first time' should be shared among all members of the Core Team. To achieve all of these benefits and meet all user expectations, the quality management team should establish Continuous Process Improvement (CPI) as a team mantra. The team should be ready and willing to enable customerdriven measurement. Quality is achieved when your customers state that a quality product has been delivered. A truly effective quality program must empower people across the organization. Everyone is responsible for the quality of the project (e.g., there are no 'other' people responsible for quality in this project). The quality program should maximize solutions by involving project members, the people closest to the problems, and it should provide ample opportunities for improvement within the project development environment.

Quality management is defined using two basic premises: (1) develop an explicit understanding of what quality means to the Sponsor and stakeholders, and (2) ensure that those requirements are deliberately and consciously built into the project at the start, rather than simply looked for at the end of the project. If you are dealing with vendors, make sure you come to a common understanding of quality measurements before starting work with them. In order for your expectations to be met, the vendor must know what is expected and what to deliver in terms of quality components.

1.8.2 Components of Quality Management

Having a solid understanding of the following key components of quality management can help to ensure these quality goals are always met:

• Managing expectations

• Quality verification

• Internal quality audit

• Process management

• Definition of metrics

• Reporting of quality metrics

• Continuous process improvement

• Establishing rewards and recognition

• Quality training and orientation

Managing Expectations. Managing expectations focuses on two key components: understanding stakeholder expectations and managing stakeholder expectations. In order to understand stakeholder expectations, you must understand who the project's key stakeholders are and be able to identify the best way to gather specific, relevant information about their expectations, including data collection, categorization, and prioritization of their expectations. It is a common mistake to assume that all stakeholders have the same expectations of a project.

A good-quality program will ensure that expectations remain realistic throughout the duration of the project effort. A good-quality program will periodically analyze progress on expectations and share expectations with the project team and stakeholders. It is important to address expectations that are not met and continuously monitor stakeholder perceptions regarding the project. The Project Manager must work with the quality team to keep expectations up to date and relevant. The Project Manager should strive to implement improvements that address expectations falling short of the desired level. Finally, the Project Manager must communicate expectation status and evaluate team progress on meeting those expectations.

Quality Verification. The most effective means of ensuring quality in a program is to verify it periodically as the project progresses from start to finish. This is done by identifying critical targets for quality verification and defining the criteria and processes for each of those targets. This will help you ensure that the desired quality levels are achieved. Some targets you should consider for your assessment are the deliverables, the processes, and the people.

For deliverables, you should include verification, validation, and testing. This would include the use of quality testing techniques such as stage containment and entry and exit criteria. Stage containment refers to a process of defining boundaries around which testing will occur. Entry and exit criteria are used to stipulate when testing begins and ends. When you are identifying such processes, consider evaluating your project compliance with such quality verification processes. You may also desire to benchmark against other known standards, such as the Project Realization Process (PRP), Capability Maturity Model (CMM), or ISO 900X audits. When assessing people, it is important to measure team satisfaction periodically. If all of the folks on your team are melancholy about their daily work, there is something wrong, you can be sure! You need to dig into this problem and work to resolve this matter quickly. Never underestimate the power of good team morale on a project. You should strive to address gaps in satisfaction by assigning responsibility, designating completion dates, and communicating the remediation plan to everyone.

Internal Quality Audit (IQAs). IQAs are performed to ensure that your business activities meet stated requirements and demonstrate the effectiveness of your quality systems. IQAs are basically plans with procedures for conducting internal assessments of your system. In conducting an audit, you should ensure that the audit is scheduled according to the importance of the activity and use objective independent auditors to carry out the assessment. They should document everything and follow up to ensure that any needed corrections are made. Communicate the results of the audit across the organization as needed, and document any and all corrective actions taken. Finally, verify the effectiveness of the corrections made and communicate that to the organization as well.

Process Management. The first step in process management is to clearly define and obtain agreement from all affected parties on what constitutes process boundaries. To accomplish this goal you must identify users and providers of the process, ensure the process is described at the appropriate level of detail, and identify how metrics and goals are tracked and collected for the process. You must provide for continuous evaluation and improvement of the process. You can use the following techniques to do this:

• Perform desk checks-look for gaps and redundant steps that add no value.

• Assess compliance with the process.

• Perform root-cause analysis on performance gaps.

• Discuss changes or improvements following the execution of a process.

Defining Metrics. For metrics to be of any value, they need to be measured against some criteria. It is the Core Team's responsibility to define the criteria for determining the appropriate metrics used in a project. The team should define the required measurement and reporting processes and identify the resources tasked to gather, analyze, and report metrics. They should next define a process for piloting the selected metrics, identify implementation requirements, and identify the requirements and the methods used to communicate these metrics and the corresponding results to the larger audience of the company. This process is sometimes referred to as determining performance measurements, which is our next topic.

1.8.3 Determine Performance Measurements

The requirements should be developed on the assumption that the system will provide a more efficient and/or more effective work process that supports the organization's mission. Performance measures for the work process should be developed to measure its progress toward accomplishing the mission. This is also a requirement of the Government Performance and Results Act of 1993 (GPRA)[5]. Ideally, some indicators that measure the impact of the project on the mission performance measures should be developed to measure the performance of a proposed system.

Performance measures must be developed for each proposed system, and a method for collecting that information must be established. Most of the performance measures should be indicators of how well the system is meeting the requirements defined for the system. Project team personnel who develop performance measures will likely require training. Performance measurement is sometimes defined as a group endeavor that seeks to improve the performance and accountability of an organization, process, program, product, or service. Some key steps to include in a performance measurement process are as follows:

• Agree on basic principles for mission, goals, and objectives.

• Brainstorm many ideas for measures.

• Select the best measures.

• Take action (i.e., develop a plan and monitor progress).

• Evaluate and calibrate the measures.

Performance measures that are sometimes used by private-sector firms to account for the value and impact of information technology are as follows:

• Process/product/service improvement

• Cycle-time reduction

• Customer satisfaction

• Cost-effectiveness

Some Information Management performance measures include the following:

• Percent change in life cycle costs

• Percent change in work process cycle time

• Percent change in acquisition time to deliver a product or service

• Percent change in product/service quality (e.g., fewer error rates in transactions)

• Percent change in customer satisfaction

• Percent change in information systems projects on schedule/within budget

• Percent change in systems that comply with architectures and standards

• Percentage of systems project management staff who meet acquisition and information management education and training requirements

Finally, over time, some valuable lessons have been learned through painful experience. These lessons include the following:

• Involve key stakeholders from the beginning.

• Place primary focus first on the most costly or troubled programs.

• Develop measures in the context of management plans and budgets.

• Choose measures that are results oriented and quantifiable and demonstrate value.

• Select a vital few projects to undertake at any time.

• Do not promise more than you can deliver.

• Educate and train stakeholders in performance measurements.

Report Quality Metrics. Provide reports that are easy to interpret and use. In order to understand and improve the quality of the development process, you should attempt to incorporate quality metrics for scheduled versus completed deliverables, the number of issues tracked per work group, and the average quality review turnaround time and duration. Compare the estimate at completion with the original baseline, and record the number of errors incurred during the test phase. All of these metrics provide benchmarks for future projects.

In addition to these metrics, it is often beneficial to conduct postmortem meetings. These are formal reviews at project completion that are used to identify any CPI opportunities. Solicit team-member feedback. This process is important to obtain and track feedback on all aspects of the project environment. To do this effectively, you should create Quality Action Teams (QATs). These teams are empowered to act as a virtual project team and are tasked to identify possible solutions to any CPI opportunity. Establish a CPI recognition program that provides incentives and rewards to encourage wider participation in CPI activities. Engage experts to provide accurate and objective assessments. This will help ensure quality services and delivery. Conduct stakeholder, team member, and user satisfaction surveys. These surveys will help the SPMO quantify the overall project satisfaction in combination with the postmortem documentation.

1.8.4 Continuous Process Improvement (CPI)

In order to implement CPI effectively, the SPMO must define an organizationwide CPI policy to ensure that improvement efforts are integrated into daily activities. One technique often used to increase project member involvement in CPI activities is to use corporatewide broadcast communications. This can be done with newsletters, project update sheets, e-mail, and so on. This type of communication helps build the level of participation by keeping a broader corporate audience informed. It also reminds the team that a larger audience than those in the conference room where they have been working has been following up on the progress of its efforts. It helps keep team members focused on the objectives and, in many instances, provides a form of positive recognition by association. If the project bombs for some reason, the reverse effect would likely hold true. Who would want his or her name associated with failure? It is another form of motivation that can be relied on to provide results. Many times in my career, I have heard people make the statement that a project would not fail on their account. People take pride in their work and want to do better. CPI simply enables this to occur.

1.8.5 Software Process Improvement Organizations

In a NASA Software Engineering Laboratory report[6], software process improvement researchers reported that software organizations historically exhibited significant shortcomings in their ability to capitalize on the experiences gained from completed projects. These researchers found that most of the insight gained through previous efforts had been passively obtained instead of having been aggressively pursued through specific planning and use of organizational infrastructures. Software developers and managers, although well-meaning and interested, generally do not have the time or resources to focus on building corporate knowledge or organizational process improvements. They have projects to run and software to deliver. Thus, collective learning and experience must become a corporate concern and be treated as a company asset.

Reuse of experience and collective learning should be supported by an organizational infrastructure dedicated to developing, updating, and supplying upon request synthesized experiences and competencies. This infrastructure should emphasize achieving continual sustained improvement. Software process improvement organizations are devoted to using lessons, data, and general experience from software projects to ensure that ongoing and future efforts use the experiences gained from previous activities to continually improve the associated organization's software products and processes. Most software process improvement organizations consist of the following elements:

• Developers- who design, implement, and maintain software. They also provide project documentation and data gathered during development and operations.

• Process analysts -who transform the data and information provided by the developers into reusable forms (e.g., standards, models, training) and supply them back to the developers. They provide specific support to the projects on the use of the analyzed and synthesized information, tailoring it to a format that is usable by and useful to a current software effort.

• Support staff -who provide services to the developers by supporting data collection and retrieval and to the analysts by managing the repository of information.

Although each element is separate and distinct from the others, these three components are intimately related to each other. Each component has its own goals, process, and plans, but when combined, all three components have the mission of providing software that is continually improving in quality and cost effectiveness. Figure 1.2 outlines the differences in focus among the three components constituting the software process improvement organization. It also provides you with a high-level picture of the software process improvement organization. It highlights the activities and information flows among its three components-namely developers, analysts, and support staff.

Figure 1.2: Focus of Software Process Improvement Organizational Components.

The developers produce and maintain software, but are not directly responsible for capturing the reusable experience. They provide the analysts with project and environment characteristics, development data, resourceusage information, quality records, and process information. The developers also provide feedback on the actual performance of the models produced by the analysts and used by the project. Therefore, with respect to software process improvement, the developers have global responsibility for using, in the most effective way, the packaged experiences to deliver high-quality software.

The analysts, by processing the information received from the developers, produce models of products and processes and return direct feedback to each project. They also produce and provide baselines, tools, lessons learned, and data, which are given parameters in some form in order to be adapted to the characteristics of a project.

The support staff sustain and facilitate the interaction between developersand analysts by saving and maintaining the information, making it efficiently retrievable, and controlling and monitoring access to it. They use tools that assist in collecting, validating, and redistributing data and reusable experience.

Remember, the ultimate goal of a software process improvement organization is to understand and repeat successes and to avoid failures. Therefore, the software process improvement organization's processes and operations must be based on solid and objective development experience. A measurement-based approach is needed for project management, evaluation, and decision making. Software measures are applied to process, product, and resources. Measurement is one of the basic tools available to the software process improvement organization for performing its tasks and to management for controlling and improving the efficiency of the whole infrastructure.

Establishing Rewards and Recognition. Establish a governing body for the rewards-and-recognition process within the project environment. The responsibilities of this team include identifying the types of behaviors to reward and providing initiatives that support the project's goals. Define all eligible participants by determining which groups or individuals are affected by the project deliverables. Establish award criteria, reward availability, frequency of award presentations, and responsibility for providing and bestowing the awards. Develop a communications plan for initial and ongoing initiatives that people will participate in and outline the process for how an individual or team participates in the rewards program.

---