Testing

22 Verify that design and testing are taking place in a development/test environment and not in a production environment.

Failure to perform design and test work in dedicated environments could result in disruption of normal business activities.

How

View evidence that the environments being used during development and test are separate from the environment being used for production. View a layout of the architecture, and validate segregation of the environments. View project member logins to the various environments, and confirm that the servers being used for design, testing, and production match the architecture layout.

23 Review and evaluate the testing process. Ensure that the project has an adequate test plan and follows this test plan.

Testing of the system, software, or process will provide assurance that it works as intended.

How

Review the test plan for elements such as

• Determine whether the test plan includes

  • Unit testing (testing of individual system modules or units or groups of related units)

  • Integration testing (testing of multiple modules or units to ensure that they work together correctly)

  • System testing (testing of the overall system by the development team)

  • Acceptance testing (testing performed by the end users to validate that the system meets requirements and is acceptable)

  • Regression testing (retesting select areas to ensure that changes made to one part of the system did not cause problems in other parts of the system)

• Ensure that the test plan and related procedures and test cases are repeatable so that they can be used for regression testing and future releases.

• Ensure that test plans and cases go through a peer review to ensure quality.

• Determine if the test plan includes testing of bad/erroneous data, system error handling, and system recovery.

• Determine if the test plan includes testing of security and internal controls.

• Ensure that results of testing are fully documented.

• Ensure that gaps identified during testing are documented, tracked, resolved, and retested. Ensure that the gap/bug-tracking process is agreed to up front. This process needs to be baselined and a system of controlled change established quickly, or it can quickly become a mess with code being pulled in and out of production haphazardly.

• Ensure that the project team has agreed to metrics to be captured and reported during testing and that these metrics are reported in a timely fashion to the appropriate members of the project leadership.

• Ensure that the test plan includes the testing of performance requirements and thresholds.

• Ensure that each test case identifies the product, component, or module that it is testing.

• Evaluate that the process for ensuring all major functionality is tested and that all key logic paths are identified and tested.

• Ensure that test data have been created and that the customers agree that they are valid test data.

• Determine whether test steps define expected results and customer-acceptance criteria.

• Ensure that all test tasks are identified and assigned an owner and that the "who, what, where, and when" of testing have been clearly identified for all parties involved.

• Ensure that appropriate sign-offs have been obtained for the plan.

• Determine whether the test plan lists the sequence in which test steps should be performed.

• Ensure that test planning includes the identification of and plans for obtaining hardware and software needed for testing.

• If using a combination of vendor software and internally developed code, determine whether there is a process defined for ensuring that both parties' code will be merged together in a well-coordinated fashion.

Note 

This list should not be used as a mechanical checklist. The absence of one of these items should not automatically result in an audit issue. Instead, look at the testing process as a whole, and determine whether enough of the key elements are present to provide a reasonable comfort level that adequate and controlled testing is taking place.

24 Ensure that all requirements can be mapped to a test case.

A defined process for tracing requirements to the test plan will provide assurance that all requirements are addressed and tested.

How

If it exists, review the requirements trace map, and verify that all requirements are represented and map to a test case. If a trace map doesn't exist, review the process for ensuring that all requirements are tested.

25 Ensure that users are involved in testing and agree that the system meets requirements. This should include IT personnel who will be supporting the system and IT personnel who were involved in performing initial technical feasibility studies for the project.

The system, software, or process is being developed to meet a specific business need. The project cannot be a success if the key stakeholders are not satisfied. Therefore, they must be involved in testing and signing off on the system prior to implementation. Also, as mentioned in step 13, there are usually multiple organizations in the IT environment that will be involved in supporting any new system, including network support, operating system support, database support, data center personnel, IT security, and the help desk. If these organizations are not involved in testing and signing off on the system, they may not be prepared to support it, and/or the system may not be in compliance with applicable standards and policies.

How

Look for evidence of user acceptance testing. Ensure that key stakeholders who were involved in requesting and approving the project and in defining system requirements (including affected IT organizations) are also involved in testing and project sign-off.

26 Consider participating in user acceptance testing and validating that system security and internal controls are functioning as intended.

This is necessary for the same reasons outlined in step 15. By participating in testing, you will be able to independently validate these controls.

How

During earlier steps, you should have worked with the project team to identify the internal controls that should be built into the system, software, or process. Review the test plan to ensure that it encompasses testing of those internal controls. Participate as an acceptance tester of those test cases.