Quality Assurance

Quality assurance involves the auditing, monitoring, and management of all aspects of software quality throughout the entire project lifecycle. A rigorous QA process demands a range of testing types to ensure an acceptable level of quality for the system delivered to the customer.

Here are the different types of test an enterprise system commonly undergoes before being released into a production environment.

Unit tests.

The objective of unit testing is to validate a component's conformance to the design. These tests form the backbone of a test-driven approach and are the responsibility of the developer. They are fine-grained tests and typically exercise the functionality of a single class or component via its public and package-level methods and data.

Unit tests can be a combination of black-box and white-box testing methods. Under a black-box test, the unit test confirms the class under test meets its specified requirements. A white-box test looks at the internals of the class to verify how the requirements are met.

Unit testing is covered in Chapter 14.

Integration tests.

Unlike unit tests, which focus on a single class or component, integration testing takes a wider view, operating at a higher level. They confirm components are able to collaborate in order to deliver the required functionality.

Functional tests.

Functional tests verify the system's conformance to the end-user requirements. They usually align with individual use cases, which themselves provide a starting point for a test case. Larger projects tend to charge a dedicated QA team with the responsibility of producing and executing all functional tests.

Load tests.

Load, or performance, tests target the nonfunctional requirements and confirm the ability of the system to meet the specified performance criteria under a given load.

Stress tests.

Stress testing places the system under a load that exceeds its designed operational capacity. The purpose of the stress test is to observe the system's failsafe behavior under excessive load.

System tests.

A system test is carried out on a completely integrated system and looks to prove that the system meets all requirements, both functional, as defined in the use cases, and nonfunctional, such as meeting specific performance criteria.

Regression tests.

This is a combination of the different testing types that together measure the impact of a modification upon a system. Full regression testing can be a lengthy process, involving the rerunning of all unit, integration, functional, and system tests. Depending on the nature and extent of the change, a partial regression test may be preferable.

Acceptance tests.

An acceptance test confirms the system meets the acceptance criteria agreed with the stakeholders. It is common for the customers of a system to undertake this type of testing with their own QA team, as acceptance testing is closely linked to the terms of the contract under which the system was developed.

The focus of this discussion is on functional, load, and stress testing. Performing these tests correctly requires the establishment of a suitable QA environment. The next section examines the different environments necessary for developing and testing enterprise software.

The Project Environment

Enterprise projects commonly require three distinct working environments: development, testing, and production.

Figure 15-1 illustrates the three environments.

Figure 15-1. Environment setup.

In the development environment, software engineers write, unit-test, and integrate all source code for the system. This environment is likely to comprise developer workstations with additional machines for performing integration builds and housing source-control repositories.

The testing environment is where all formal testing takes place against regular, versioned releases of the system from the development environment. For tests to be meaningful, the test environment must closely resemble the production environment.

Note

The testing environment is sometimes called the staging area or preproduction environment.

Production is the target environment for the system, and the testing effort must validate the system is capable of meeting all functional and nonfunctional requirements when operating in this final environment. Testing is unlikely to be possible in production because live systems are involved, hence the need for the testing environment to closely mirror that of production.

The Testing Process

Testing as part of an iterative development methodology such as Extreme Programming (XP) or the IBM Rational Unified Process (RUP) is an ongoing process conducted throughout the entire software development lifecycle.

Chapter 3 covers iterative development processes, including XP and the RUP.

An iterative development process has a running version of the system available from the early stages of the project. This allows the project team to continually submit the system to a barrage of functional and nonfunctional tests. This constant and ongoing testing effort is a key strategy in reducing risk by confirming the system under development is able to meet customer requirements and design goals.

Formal testing should commence toward the end of each iteration. The process starts by delivering a versioned release into the test environment

Note

It is important the testing environment is reproducible between tests so the environment's hardware, configuration, and test data remains constant for each test cycle. This is essential for meaningful comparisons of test results between releases.

Each release is subjected to a full range of functional and nonfunctional tests, and any arising defects and issues are logged for the attention of the development team. The release and testing process continues until the system is of an acceptable standard. The final iterations of a project focus intensely on the testing effort and look to deliver the finished system into production.

It is common for systems with a high defect rate to spend a substantial amount of time bouncing between the developments and testing environments. A long and involved testing process causes lengthy delays in getting the application into production.

Testing for RAD Projects

Supporting rapid application development requires meeting two objectives:

• Improving the quality of software releases in order to reduce the number of testing cycles

• Reducing the time taken to complete the testing process

A test-driven approach to development results in higher quality software with fewer defects. This point has critical implications for rapid development, as the number of defects found during testing directly relates to the project's duration. Put bluntly, the more defects discovered, the longer the system takes to deliver to the customer. Moreover, a problem detected during the formal testing process is more expensive to rectify than if discovered during development. Rigorous testing during development pays dividends by enabling the detection and correction of defects close to their point of origin.

Reducing the time taken to execute all of a system's functional and nonfunctional test cases is achievable with test automation. This approach speeds up the testing process without comprising the quality of the test performed. The benefits of automated testing for rapid development make this the main topic for this chapter.