DIMENSIONS OF TESTING

To assess product quality, different kinds of tests, each one with a different focus, are needed. These tests can be categorized by several dimensions:

Quality dimension:

the major quality characteristic or attribute that is the focus of the test
Stage of testing:

the point in the lifecycle at which the test is executed
Type of test:

the specific test objective for an individual test, usually limited to a single quality dimension

Quality Dimension

There are patterns to quality problems; the same kinds of problems tend to recur in virtually every system. As a result, the following aspects need to be assessed for every product:

Reliability

The software is resistant to failure during execution: no crashing, hanging, memory leaks, and so on.
Functionality

The software executes the required use cases or desired behavior as intended.
Performance

The software and the system execute and respond in a timely manner and continue to perform acceptably when subjected to real-world operational characteristics (e.g., load, stress, and lengthy periods of operation). Performance testing focuses on ensuring that the functionality of the system can be provided while satisfying the nonfunctional requirements of the system.

For each of the quality dimensions, one or more types of tests should be executed during one or more of the test stages. In addition, there are other qualities to consider whose assessment may be more subjective : maintainability, extensibility, flexibility, and so on. Whenever possible, quantitative measures should be derived to assess these qualities.

Stages of Testing

Testing is not a single activity, executed all at once. Testing is executed against different types of targets (targets-of-test) in different stages of the software development. These test stages progress from testing small elements of the system, such as components (unit testing), to testing completed systems (system testing). The four stages have the following purposes:

Unit test

The smallest testable elements of the system are tested individually, typically at the same time those elements are implemented.
Integration test

The integrated units (or components or subsystems) are tested.
System test

The complete application and system (one or more applications) are tested.
Acceptance test

The complete application (or system) is tested by end users (or representatives) for the purpose of determining readiness for deployment.

Keep in mind that these stages occur throughout the lifecycle, with varying emphasis. An early conceptual prototype used in the inception phase to assess the viability of the product vision will be subjected to acceptance tests, informal though they may be; an architectural prototype developed during the elaboration phase will be subjected to integration and system tests to validate architectural integrity and performance of key architectural elements, even though most of the code in the system exists only as stubs. These stages are not "phases" that occur in sequence near the end of the project ”in the iterative lifecycle testing occurs early and often.

Types of Tests

There are many types of tests, each one focusing on a specific test objective and testing only one characteristic or attribute of the software. Because tests are executed throughout the lifecycle, the software being tested may be a single unit of code, integrated units, or a complete application or system. Here are some of the most common types of tests:

Benchmark test

Compares the performance of a target-of-test to a known standard such as existing software or measurement(s)
Configuration test

Verifies that the target-of-test functions is an acceptable manner on different configurations (hardware or software)
Function test

Verifies that the target-of-test functions properly, executing the required use case as intended
Installation test

Verifies that the target-of-test installs properly and can be installed successfully on different configurations or under different conditions, such as insufficient disk space
Integrity test

Verifies the target-of-test's reliability, robustness, and resistance to failure during execution
Load test

Verifies the acceptability of the target-of test's performance under varying operational conditions, such as number of users, number of transactions, and so on, while the configuration remains constant
Performance test

Verifies the acceptability of the target-of-test's performance using various configurations while the operational conditions remain constant
Stress test

Verifies the acceptability of the target-of-test's performance when abnormal or extreme conditions are encountered , such as diminished resources or an extremely high number of users

Regression Testing

Regression testing is a test strategy in which previously executed tests are re-executed against a new version of the target-of-test, to ensure that the quality of the target has not regressed ”moved backward ”while new capabilities have been added. The purpose of the regression test is to ensure that

the defects identified in the earlier execution of test have been addressed and
the changes made to the code have not introduced new defects or reintroduced old ones.

Regression testing can involve the re-execution of any of the test types. Typically, some regression testing is performed during each iteration, rerunning tests from prior iterations.