11.1 Project Integrity

---

11.1.1 Practice 1. Adopt Continuous Program Risk Management

Practice Essentials

A. Risk management is a continuous process beginning with the definition of the concept and ending with system retirement.

B. Risk management is a program responsibility impacting on and supported by all organizational elements.

C. All programs need to assign a Risk Officer as a focal point for risk management and maintain a reserve to enable and fund risk mitigation.

D. Risk needs to be identified and managed across the life of the program.

E. All risks identified should be analyzed, prioritized—by impact and likelihood of occurrence—and tracked through an automated risk management tool.

F. High-priority risks need to be reported to management on a frequent and regular basis.

Implementation guidelines

A. Risk management should commence prior to contract award and shall be a factor in the award process.

B. The DEVELOPER needs to establish and implement a project Risk Management Plan that, at a minimum, defines how best practices 3 through 8 will be implemented. The plan and infrastructure (tools, organizational assignments, and management procedures) will be agreed to by the ACQUIRER and the DEVELOPER and need to be placed under Configuration Management (CM).

C. DEVELOPER and ACQUIRER senior management should establish reporting mechanisms and employee incentives in which all members of the project staff are encouraged to identify risks and potential problems and are rewarded when risks and potential problems are identified early. The ACQUIRER needs to address risk management explicitly in its contract award fee plan, and the DEVELOPER needs to provide for the direct distribution to all employees in furtherance of establishing and maintaining a risk culture.

D. Risk identification should be accomplished in facilitated meetings attended by project personnel most familiar with the area for which risks are being identified. A person familiar with problems from similar projects in this area in the past should participate in these meetings when possible. Risk identification should include risks throughout the life cycle in at least the areas of cost, schedule, technical, staffing, external dependencies, supportability, and maintainability and should include organizational and programmatic political risks. Risk identification needs to be updated at least monthly. Identified risks should be characterized in terms of their likelihood of occurrence and the impact of their occurrence. Risk mitigation activities need to be included in the project’s task activity network.

E. Both the DEVELOPER and the ACQUIRER should designate and assign senior members of the technical staff as Risk Officers to report directly to their respective Program Managers and should charter this role with independent identification and management of risks across the program and grant the authority needed to carry out this responsibility.

F. Each medium-impact and high-impact risk should be described by a complete Risk Control Profile.

G. Periodically updated estimates of the cost and schedule at completion should include probable costs and schedule impact due to risk items that have not yet been resolved.

H. The DEVELOPER and ACQUIRER Risk Officers need to update the risk data and database on the schedule defined in the Risk Management Plan. All risks intended for mitigation and any others that are on the critical path and their status against the mitigation strategy should be summarized. Newly identified risks should go through the same processes as the originally identified risks.

11.1.2 Practice 2. Estimate Cost and Schedule Empirically

Practice Essentials

A. Initial software estimates and schedules should be looked on as high risk due to the lack of definitive information available at the time they are defined.

B. The estimates and schedules should be refined as more information becomes available.

C. At every major program review, costs-to-complete and rescheduling should be presented to identify deviations from the original cost and schedule baselines and to anticipate the likelihood of cost and schedule risks occurring.

D. All estimates should be validated using a cost model, a sanity check should be conducted comparing projected resource requirements, and schedule commitments should be made.

E. Every task within a work breakdown structure (WBS) level needs to have an associated cost estimate and schedule. These tasks should be tracked using earned value.

F. All costs estimates and schedules need to be approved prior to the start of any work.

Implementation Guidelines

A. Estimate the cost, effort, and schedule for a project for planning purposes and as a yardstick for measuring performance (tracking). Software size and cost need to be estimated prior to beginning work on any incremental release.

B. Software cost estimation should be a reconciliation between a top-down estimate (based on an empirical model [e.g., parametric, cost]) and a bottom-up engineering estimate.

C. Software cost estimation should also be subjected to a sanity check by comparing it with industry norms and specifically with the DEVELOPER’s past performance in areas such as productivity and percentage of total cost in various functions and project phases.

D. All of the software costs need to be associated with the appropriate lower-level software tasks in the project activity network. Allocate the estimated total project labor effort among all the tasks in the activity network.

11.1.3 Practice 3. Use Metrics to Manage

Practice Essentials

A. All programs should have in place a metrics program to monitor issues and determine the likelihood of risks occurring.

B. Metrics should be defined as part of the definition of process, identification of risks or issues, or determination of project success factors.

C. All metrics definitions need to include description, quantitative bounds, and expected areas of application.

D. All programs need to assign an organizational responsibility for identification, collection, analysis, and reporting of metrics throughout the program’s life.

E. Metrics information should be used as one of the primary inputs for program decisions.

F. The metrics program needs to be continuous.

Implementation Guidelines

A. Every project should have a project plan with a detail activity network that defines the process the team will follow, organizes and coordinates the work, and estimates and allocates costs and schedules among tasks. The plan should be broad enough to include each subprocess/phase. The project plan needs to include adequate measurement in each of these five categories:

1. Early indications of problems

2. Quality of the products

3. Effectiveness of the processes

4. Conformance to the processes

5. Provision of a basis for future estimation of cost, quality, and schedule

B. Metrics should be sufficiently broad based. Data should be collected for each process/phase to provide insight into the above five categories.

C. To use these metrics effectively, thresholds need to be established for these metrics. These thresholds should be estimated initially using suggested industry norms for various project classes. Local thresholds will evolve over time, based upon experience (see Practice 1.E above). Violation of a threshold value should trigger further analysis and decision making.

D. Examples of data, initial thresholds, and analysis of size, defect, schedule, and effort metrics can be found at www.qsm.com.

E. Continuous data on schedule, risks, libraries, effort expenditures, and other measures of progress should be available to all project personnel along with the latest revision of project plans.

11.1.4 Practice 4. Track Earned Value

Practice Essentials

A. Earned value project management requires a work breakdown structure, work packages, activity networks at every WBS level, accurate estimates, and implementation of a consistent and planned process.

B. Earned value requires each task to have both entry and exit criteria and a step to validate that these criteria have been met prior to the award of the credit.

C. Earned value credit is binary, with 0 percent being given before task completion and 100 percent when completion is validated.

D. Earned value metrics need to be collected on a frequent and regular basis consistent with the reporting cycle required with the WBS level. (At the lowest level of the work package, the earned value reporting should never occur less frequently than every two weeks.)

E. Earned value and the associated budgets schedules and WBS elements need to be replanned whenever material changes to the program structure are required (e.g., requirements, growth, budget changes, schedule issues, organizational change).

F. Earned value is an essential indicator and should be used as an essential metric by the risk management process.

Implementation Guidelines

A. Progress toward producing the products should be measured within the designated cost and schedule allocations.

B. THE DEVELOPER should develop and maintain a hierarchical task activity network based on allocated requirements that includes the tasks for all effort that will be charged to the program. All level of effort (LOE) tasks need to have measurable milestones. All tasks that are not LOE should explicitly identify the products produced by the task and have explicit and measurable exit criteria based on these products.

C. No task should have a budget or planned calendar time duration that is greater than the cost and schedule uncertainty that is acceptable for the program. The goal for task duration is no longer than two calendar weeks of effort.

D. Each task that consumes resources needs to have a cost budget allocated to it and the corresponding staff and other resources that will consume this budget. Staff resources should be defined by person-hours or -days for each labor category working on the task.

E. For each identified significant risk item, a specific risk mitigation/ resolution task should be defined and inserted into the activity network.

F. The cost reporting system for the total project needs to segregate the software effort into software tasks so that the software effort can be tracked separately from the nonsoftware tasks.

G. Milestones for all external dependencies should be included in the activity network.

H. Earned value metrics need to be collected for each schedule level and be made available to all members of the DEVELOPER and government project teams monthly. These metrics are: a comparison of Budgeted Cost of Work Scheduled (BCWS), Budgeted Cost of Work Performed (BCWP), Actual Cost of Work Performed (ACWP), a comparison of BCWP and ACWP, a Cost Performance Index, a Schedule Performance Index, and a To-Complete Cost Performance Index.

I. The lowest-level schedules should be statused weekly.

J. The high-level schedules should be statused at least monthly.

K. Earned value reports should be based on data that is no more than two weeks old.

11.1.5 Practice 5. Track Defects against Quality Targets

Practice Essentials

A. All programs need to have prenegotiated quality targets, which is an absolute requirement to be met prior to acceptance by the customer.

B. Programs should implement practices to find defects early in the process and as close in time to the creation of the defect as possible and should manage this defect rate against the quality target.

C. Metrics need to be collected as a result of the practices used to monitor defects, which will indicate the number of defects, defect leakage, and defect removal efficiency.

D. Quality targets need to be redefined and renegotiated as essential program conditions change or customer requirements are modified.

E. Compliance with quality targets should be reported to customers on a frequent and regular basis, along with an identification of the risk associated with meeting these targets at delivery.

F. Meeting quality targets should be a topic at every major program review.

Implementation Guidelines

A. The ACQUIRER and the DEVELOPER need to establish quality targets for subsystem software depending on its requirements for high integrity. A mission-critical/safety-critical system may have different quality targets for each subsystem component.

System Quality Assurance needs to monitor quality targets and report defects as per the Quality Plan.

B. Quality targets can be under change control and established at the design, coding, integration, test, and operational levels.

C. Quality targets should address the number of defects by priority and by their fix rate.

D. Actual quality or defects detected and removed should be tracked against the quality targets.

E. Periodic estimates of the cost and schedule at completion should be based on the actual versus targeted quality.

11.1.6 Practice 6. Treat People As the Most Important Resource

Practice Essentials

A. A primary program focus should be staffing positions with qualified personnel and retaining this staff throughout the life of the project.

B. The program should not implement practices (e.g., excessive unpaid overtime) that will force voluntary staff turnover.

C. The staff should be rewarded for performance against expectations and program requirements.

D. Professional growth opportunities such as training should be made available to the staff.

E. All staff members need to be provided facilities, tools, and work areas adequate to allow efficient and productive performance of their responsibilities.

F. The effectiveness and morale of the staff should be a factor in rewarding management.

Implementation Guidelines

A. DEVELOPER senior management needs to work to ensure that all projects maintain a high degree of personnel satisfaction and team cohesion and should identify and implement practices designed to achieve high levels of staff retention as measured by industry standards. The DEVELOPER should employ focus groups and surveys to assess employee perceptions and suggestions for change.

B. DEVELOPER senior management should provide the project with adequate staff, supported by facilities and tools to develop the software system efficiently. Employee focus groups and surveys should be used to assess this adequacy.

C. The training of DEVELOPER and ACQUIRER personnel should include training according to a project training plan in all the processes, development and management tools, and methods specified in the software development plan.

D. The DEVELOPER and the ACQUIRER should determine the existing skills of all systems, software, and management personnel and provide training, according to the needs of each role, in the processes, development and management tools, and methods specified in the Software Development Plan (SDP).

---