Project Management Process Areas

Project Planning

The CAU project was strategically populated with experienced personnel from the PASS project to allow rapid transfer of experience in the Project Planning process area. This permitted the champions who were placed into CAU to contribute their intense process culture, experience base, and history to this start-up project.

Projects develop software estimates based on experience with similar functions. Initial estimates are developed from the top down with historical costing factors applied. A bottom-up costing activity is then used for tuning. For the CAU project, following the generation of estimates, three independent assessments are used for a sanity check using industry tools. Independent estimates are generated by the customer, consultants, and an independent subcontractor.

The basic inputs to the estimates are source lines of code, projected error rates, and level of testing required. A basis of estimate is developed for each estimate. The basis for the estimate includes product size (e.g., lines of code, number of test cases) and the computations using that data to generate an estimate.

Project budgets are based on the estimates. These are developed against an Integrated Master Schedule, which takes into account task size, complexity, and dependencies.

Assessment of the growth in the work scope based on requirements creep is conducted on a schedule that has been accepted by the customer. Source lines of code estimates are used to determine the impact of requirements creep. This data is used to renegotiate the scope of the work. Requirements growth beyond a negotiated threshold will result in a customer-directed scrub of the requirements.

The staffing plan is developed based on the agreed-on product content, schedule, and budget. Equipment and system resources to support all project activities are estimated and funding is allocated. Actual plans for obtaining this equipment are documented in the Information Technology Plan. Task charge numbers are opened for each activity, decomposed from the work breakdown structure, with funding loaded using the earned-value system. Allocation of facilities for housing the staff and performing testing and integration are included in the plan. Funding is allocated for these facilities and they are now in place.

Three levels of schedule plans are covered by the project Integrated Management Schedule. Plans are also in place for verification and validation, integration, facilities, and software via the Software Development Management Plan (SDMP). The SDMP covers planning for the processes to be used in developing the software. Subordinate plans are a direct fallout of the plans documented in the Project Management Plan and SDMP. Subordinate plans are developed to be consistent with those upper-level documents.

Data management plans include the use of the project's intranet Web page the Integrated Collaborative Environment, which includes such project data as presentations, trade studies, product team agendas and minutes, and requirements documents.

A training plan is developed that identifies the skills required by the project team. For the CAU project, skills are required in three areas: the legacy system that the project interfaces with, the commercial off-the-shelf (COTS) real-time operating system, and the language and tools to be used.

Relevant stakeholders participate in planning through the various product teams, as defined in the Project Management Plan. Participation of relevant stakeholders is also defined in the process documents for each phase of the development cycle (e.g., requirements generation and evaluation, development process, and detailed test process). Relevant stakeholders participate through attendance at both formal and informal reviews and inspections.

Project Monitoring and Control

Performance progress is monitored through two specific tools. First, the earned-value system provides timely insight into the completion of project milestones. Milestones are of sufficient detail to allow close review of project progress.

Following customer acceptance, the project's budget is loaded into an earned-value schedule that is used to manage the execution of the project. Parameters developed during planning (specifically, effort to accomplish specific deliverables and milestones) are entered and are tracked closely for variances. Training is provided in the use of the earned-value system with guidance on the rules for resource-loaded schedules.

The program-level schedules include milestones at which project accomplishments are to be reviewed. For example, reviews occur at the System Design Review, the System Requirements Review, the Preliminary Design Review, and the Critical Design Review, among others.

Until the project begins to develop code, actual measures cannot be generated against the planning size parameters. Instead, size is reestimated based on iterative deliveries of requirements from the customer to determine the delta to original estimates so that scope can be renegotiated with the customer.

The second tool consists of a set of performance metrics that have been established to provide project management with more information to assess progress and address problem areas. Actual staffing is tracked against the staffing plan. Staffing and training are guided by the training plan to ensure that the necessary skills are available to the project. The Information Technology Plan is executed and closely tracked for variances.

Deliverable data is specifically identified in the decision packages. These documents are assigned control numbers and are maintained in a project library tracked by project librarians. For the CAU project, this library is managed at the program level, not the software project level.

The various product teams carry out monitoring of project commitments. Product team leads are responsible for ensuring that stakeholder attendance and participation is appropriate at each product team meeting.

Issues that require special corrective action normally result in development of unique plans with special schedules for issue resolution. Issues are identified, responsibility for the issue is assigned, and a schedule for resolution is developed. Impacts to project schedules are communicated to the appropriate product team or to the project manager so that appropriate action can be taken. For the CAU project, this is now being done for issues concerning the System Software, Software Requirements Specification.

Integrated Project Management

CAU project processes are patterned after the PASS processes and are tailored to meet the specific requirements of the project. (The PASS processes and historical data have influenced the USA organizational process and database that is being developed. See the section on Organizational Process Definition, pages 131 132.) The project processes and the integration of these processes are described in the Project Management Plan. The Software Development Management Plan further defines the software processes and how they are integrated. The integration of these processes is implemented through the Integrated Management Plan. An important aspect of the plans is the use of integrated product teams to ensure that the integration of project activities includes all relevant stakeholders.

Historical data from the PASS project is used in project estimating and planning. The estimates developed from this data are tested against industry models.

Also, the project uses the lessons learned by other projects, particularly PASS. As the project itself progresses, it will, in turn, contribute lessons learned to other projects that might benefit from its experiences.

Dependencies are documented in the Integrated Master Schedule. Responsibility to manage and track these dependencies is accomplished by the individual product teams. The Development Product Team addresses significant project dependency issues. Coordination issues are addressed through the individual product teams, with significant issues communicated to the Development Product Team. The Development Product Team calls meetings with relevant stakeholders to coordinate resolution of issues.

Integrated product teams are the focal point for managing project activities. A series of product teams (e.g., Flight Software Product Team, Hardware Product Team, Operations Product Team, and Project Product Team) are used to involve relevant stakeholders in planning and in technical decision making. All the product teams regularly monitor progress with overall status reviewed at the product team level. Product teams meet on a weekly basis to perform integrated planning/decision making.

Supplier Agreement Management

The CAU project forms a source selection team that will use criteria in accordance with the source selection process. The supplier is required to describe its management approach, technical approach, past performance on similar programs, and pricing. Detailed evaluation criteria are applied to the bids received. A scale is used to evaluate each criterion. The contract is awarded to the supplier with the highest score. Risks are identified and mitigation plans established for the contracting and performance phases. The selection process, evaluation criteria, rationale for candidate selection, and risk information are reviewed at multiple management levels.

The selected subcontractor provides a rough order of magnitude cost based on USA's initial estimate. The subcontractor revises the estimate by applying an appropriate estimation method (e.g., IBM's Function Point Analysis) to the revised Software Requirements Specification. USA performs a parallel effort.

The subcontractor typically participates in the project's requirements processes to obtain an understanding of and make a commitment to the software requirements. The subcontractor-developed software design, code, test plans, and results will be inspected in accordance with USA's practices. The subcontractor will support (not perform) USA's integration, and detailed verification testing efforts. For the CAU project, formal acceptance testing of the subcontractor-developed software will be done at least one year prior to the first flight with the software installed. The subcontractor will remain in a supporting role through the completion of the first shuttle flight with the software. USA will then assume control of the software.

The project works with USA's Contracting Office and Purchasing. They follow the processes documented in the company policies, functional procedures, and company acquisition procedures to establish supplier agreements and procure well-defined products.

USA employees receive computer-based training to learn the concepts and process for effective subcontract management. This includes compliance with government regulations, acquisition strategies, source selection, and management of subcontractors' performance.

Because there is no specific USA commercial off-the-shelf policy, this is handled as a make/buy decision. Nonstandard COTS products incorporated as a significant part of the developed software are evaluated against a documented set of criteria in a trade study. A decision package agreement reviewed by NASA lists the COTS products to be acquired. An internal Information Technology Plan specifies the timing of the COTS acquisitions.

Risk Management

Risk management is an integral part of USA program management. USA emphasizes that risk is goal-based. If you have a goal, you have the potential to miss that goal. USA integrates risk management into its day-to-day decision making by employing a collective set of activities for identifying, assessing, handling, monitoring, and communicating risk information (Figure 7.3).

Figure 7.3. USA Risk Management Process

Proactive risk management is designed to uncover potential issues before they can disrupt normal operations. USA targets two areas for proactive risk management:

• Change as a major source for introducing new risks into a system or process a risk assessment will be performed whenever changes to the baseline are proposed.

• Communication and monthly reviews of significant risks as proactive measures the presentation of risk information from one program element may help identify collateral risks in other areas.

The USA Risk Assessment Scorecard (Figure 7.4) is the primary tool for prioritizing risks against goals. It contains predefined descriptions for consequence and likelihood on an increasing scale from 1 to 5. The consequence descriptions are grouped in risk measures that represent risks to contract goals so that risks from different functional program elements can be expressed in the same "scorecard" language. The likelihood descriptions are scaled according to the expectation of occurrence for the undesirable event.

Figure 7.4. USA Risk Assessment Scorecard

The risk score is computed by multiplying highest consequence by the likelihood. Risks are ranked to make the best use of resources and to prioritize the handling actions. The risk matrix has three regions: low (green) or scores 1 6, medium (yellow) or scores 7 14, and high (red) or scores 15 25. Colors correspond to the USA stoplight status reporting method.

A green risk is one of either relatively low likelihood or low consequence or both, and does not normally require action. A yellow risk can occur when a risk has a high consequence, but a low likelihood; or a high likelihood, but a low consequence; or a medium score on both. For yellow risks, action and notification to appropriate management is recommended. A red risk is one that is of both high likelihood and undesirable consequence. Red risks require action and notification.

When a risk assessment is performed, multiple options for handling the risk are usually presented. Actions can fall into three categories: accept, mitigate, or prevent.

Risks are "accepted" when the action is totally outside of USA's control or the risk is low priority. Management is aware that the risk condition exists, but no further action will be taken. Risks are "mitigated" when control measures have been put in place to reduce the consequence if the event were to occur. Risks are "prevented" when some form of action has been taken to reduce the likelihood of an undesirable event.

A well-defined USA company risk management process is in place to identify, assess, and address program risks. It includes mechanisms to ensure communication of the risk to appropriate management and objective criteria defining when actions are required. Significant risks on the project are reviewed with top USA management and also reviewed with the NASA customer at regularly scheduled integrated product team meetings.

Quantitative Project Management

The primary quality objective set for process performance is the product error rate threshold (errors/KSLOC) defined in the contract. NASA and USA set the project process-performance objectives jointly through contract negotiations. The current threshold was set based on historical process performance. The processes examined for process performance data include the development, development test, design/code inspection, and verification test processes. Data from these processes are used to determine process performance.

Emphasis is placed on inspection results/performance and inserted errors or defect density. The design/code inspection process is statistically analyzed using statistical process control techniques to determine how current process performance compares to historical performance and to identify any special causes of variation (signals) in the current execution of the process. When special causes of variations are noted, they are investigated to determine if action is required.

The measurements are monitored throughout product development to determine if the actual results support achievement of the established objectives. Management is involved in reviewing the process performance through internal quality review meetings. The process owners are involved, as needed, to make changes to the process based on feedback from the statistical process control analysis (e.g., data collection changes, refresher training).

For most USA software projects, there are no defined subprocesses that are statistically managed and monitored for process capability. The USA standard software process has not yet been institutionalized in a way that it can collect historical process stability and capability data. Additionally, no USA measurement repository, in which to store such data, has been established yet.