Specific Practices by Goal

SG 1 Prepare for Validation

Preparation for validation is conducted.

Preparation activities include selecting products and product components for validation and establishing and maintaining the validation environment, procedures, and criteria. The items selected for validation may include only the product or it may include appropriate levels of the product components that are used to build the product. Any product or product component may be subject to validation, including replacement, maintenance, and training products, to name a few.

The environment required to validate the product or product component is prepared. The environment may be purchased or may be specified, designed, and built. The environments used for product integration and verification may be considered in collaboration with the validation environment to reduce cost and improve efficiency or productivity.

Table . Practice-to-Goal Relationship Table

Continuous Representation	Staged Representation
SG 1 Prepare for Validation	SG 1 Prepare for Validation
SP 1.1-1 Select Products for Validation	SP 1.1-1 Select Products for Validation
SP 1.2-2 Establish the Validation Environment	SP 1.2-2 Establish the Validation Environment
SP 1.3-3 Establish Validation Procedures and Criteria	SP 1.3-3 Establish Validation Procedures and Criteria
SG 2 Validate Product or Product Components	SG 2 Validate Product or Product Components
SP 2.1-1 Perform Validation	SP 2.1-1 Perform Validation
SP 2.2-1 Analyze Validation Results	SP 2.2-1 Analyze Validation Results
GG 1 Achieve Specific Goals
GP 1.1 Perform Base Practices
GG 2 Institutionalize a Managed Process	GG 3 Institutionalize a Defined Process
GP 2.1 Establish an Organizational Policy	GP 2.1 Establish an Organizational Policy
GP 2.2 Plan the Process	GP 2.2 Plan the Process
GP 2.3 Provide Resources	GP 2.3 Provide Resources
GP 2.4 Assign Responsibility	GP 2.4 Assign Responsibility
GP 2.5 Train People	GP 2.5 Train People
GP 2.6 Manage Configurations	GP 2.6 Manage Configurations
GP 2.7 Identify and Involve Relevant Stakeholders	GP 2.7 Identify and Involve Relevant Stakeholders
GP 2.8 Monitor and Control the Process	GP 2.8 Monitor and Control the Process
GP 2.9 Objectively Evaluate Adherence	GP 2.9 Objectively Evaluate Adherence
GP 2.10 Review Status with Higher Level Management	GP 2.10 Review Status with Higher Level Management
GG 3 Institutionalize a Defined Process
GP 3.1 Establish a Defined Process	GP 3.1 Establish a Defined Process
GP 3.2 Collect Improvement Information	GP 3.2 Collect Improvement Information
GG 4 Institutionalize a Quantitatively Managed Process
GP 4.1 Establish Quantitative Objectives for the Process
GP 4.2 Stabilize Subprocess Performance
GG 5 Institutionalize an Optimizing Process
GP 5.1 Ensure Continuous Process Improvement
GP 5.2 Correct Root Causes of Problems

SP 1.1-1 Select Products for Validation

Select products and product components to be validated and the validation methods that will be used for each.

Products and product components are selected for validation on the basis of their relationship to user needs. For each product component, the scope of the validation (e.g., operational behavior, maintenance, training, and user interface) should be determined.

The requirements and constraints for performing validation are collected. Then, validation methods are selected based on their ability to demonstrate that user needs are satisfied. The validation methods not only define the technical approach to product validation, but also drive the needs for the facilities, equipment, and environments. This may result in the generation of lower level product-component requirements that are handled by the requirements development processes. Derived requirements, such as interface requirements to test sets and test equipment, can be generated. These requirements are also passed to the requirements development processes to ensure that the product or product components can be validated in an environment that supports the methods.

Validation methods should be selected early in the life of the project so that they are clearly understood and agreed to by the relevant stakeholders.

The validation methods address the development, maintenance, support, and training for the product or product component as appropriate.

Typical Work Products

1. Lists of products and product components selected for validation

2. Validation methods for each product or product component

3. Requirements for performing validation for each product or product component

4. Validation constraints for each product or product component

Subpractices

1. Identify the key principles, features, and phases for product or product-component validation throughout the life of the project.

2. Determine which categories of user needs (operational, maintenance, training, or support) are to be validated.

   The product or product component must be maintainable and supportable in its intended operational environment. This specific practice also addresses the actual maintenance, training, and support services that may be delivered along with the product.

   An example of evaluation of maintenance concepts in the operational environment is a demonstration that maintenance tools are operating with the actual product.

3. Select the product and product components to be validated.

4. Select the evaluation methods for product or product-component validation.

5. Review the validation selection, constraints, and methods with relevant stakeholders.

SP 1.2-2 Establish the Validation Environment

Establish and maintain the environment needed to support validation.

The requirements for the validation environment are driven by the product or product components selected, by the type of the work products (e.g., design, prototype, final version), and by the methods of validation. These may yield requirements for the purchase or development of equipment, software, or other resources. These requirements are provided to the requirements development processes for development. The validation environment may include the reuse of existing resources. In this case, arrangements for the use of these resources must be made. Examples of the type of elements in a validation environment include the following:

• Test tools interfaced with the product being validated (e.g., scope, electronic devices, probes)

• Temporary embedded test software

• Recording tools for dump or further analysis and replay

• Simulated subsystems or components (by software, electronics, or mechanics)

• Simulated interfaced systems (e.g., a dummy warship for testing a naval radar)

• Real interfaced systems (e.g., aircraft for testing a radar with trajectory tracking facilities)

• Facilities and customer-supplied products

• The skilled people to operate or use all the preceding elements

• Dedicated computing or network test environment (e.g., pseudo-operational t

• Dedicated computing or network test environment (e.g., pseudo-operational telecommunications-network testbed or facility with actual trunks, switches, and systems established for realistic integration and validation trials)

Early selection of the products or product components to be validated, the work products to be used in the validation, and the validation methods is needed to ensure that the validation environment will be available when necessary.

The validation environment should be carefully controlled to provide for replication, analysis of results, and revalidation of problem areas.

Typical Work Products

1. Validation environment

Subpractices

1. Identify validation environment requirements.

2. Identify customer-supplied products.

3. Identify reuse items.

4. Identify test equipment and tools.

5. Identify validation resources that are available for reuse and modification.

6. Plan the availability of resources in detail.

SP 1.3-3 Establish Validation Procedures and Criteria

Establish and maintain procedures and criteria for validation.

Validation procedures and criteria are defined to ensure that the product or product component will fulfill its intended use when placed in its intended environment. Acceptance test cases and procedures may meet the need for validation procedures.

The validation procedures and criteria include test and evaluation of maintenance, training, and support services.

Examples of sources for validation criteria include the following:

• Product and product-component requirements

• Standards

• Customer acceptance criteria

• Environmental performance

• Thresholds of performance deviation

Typical Work Products

1. Validation procedures

2. Validation criteria

3. Test and evaluation procedures for maintenance, training, and support

Subpractices

1. Review the product requirements to ensure that issues affecting validation of the product or product component are identified and resolved.

2. Document the environment, operational scenario, procedures, inputs, outputs, and criteria for the validation of the selected product or product component.

3. Assess the design as it matures in the context of the validation environment to identify validation issues.

SG 2 Validate Product or Product Components

The product or product components are validated to ensure that they are suitable for use in their intended operating environment.

The validation methods, procedures, and criteria are used to validate the selected products and product components and any associated maintenance, training, and support services using the appropriate validation environment.

SP 2.1-1 Perform Validation

Perform validation on the selected products and product components.

To be acceptable to users, a product or product component must perform as expected in its intended operational environment.

Validation activities are performed and the resulting data are collected according to the established methods, procedures, and criteria.

The as-run validation procedures should be documented and the deviations occurring during the execution should be noted, as appropriate.

For users of the continuous representation, this is a capability level 1 specific practice. Validation processes at capability level 1 or 2 may not include procedures and criteria, which are created in the Establish Validation Procedures and Criteria specific practice at capability level 3. When there are no procedures or criteria established, use the methods established by the Select Products for Validation specific practice to accomplish capability level 1 performance.

Typical Work Products

1. Validation reports

2. Validation results

3. Validation cross-reference matrix

4. As-run procedures log

5. Operational demonstrations

SP 2.2-1 Analyze Validation Results

Analyze the results of the validation activities and identify issues.

The data resulting from validation tests, inspections, demonstrations, or evaluations are analyzed against the defined validation criteria. Analysis reports indicate whether the needs were met; in the case of deficiencies, these reports document the degree of success or failure and categorize probable cause of failure. The collected test, inspection, or review results are compared with established evaluation criteria to determine whether to proceed or to address requirements or design issues in the requirements development or technical solution processes.

Analysis reports or as-run validation documentation may also indicate that bad test results are due to a validation procedure problem or a validation environment problem.

Typical Work Products

1. Validation deficiency reports

2. Validation issues

3. Procedure change request

Subpractices

1. Compare actual results to expected results.

2. Based on the established validation criteria, identify products and product components that do not perform suitably in their intended operating environments, or identify problems with the methods, criteria, and/or environment.

3. Analyze the validation data for defects.

4. Record the results of the analysis and identify issues.

5. Use validation results to compare actual measurements and performance to intended use or operational need.