Workflow

Figure 12-3 illustrates the typical testing workflow, showing the major workflow details and their dependencies.

Figure 12-3. Typical testing workflow with details and their dependencies

Define Evaluation Mission

The purpose of this workflow detail is to identify the appropriate focus of the test effort for the iteration and to gain agreement with stakeholders on the corresponding goals that will direct the test effort.

For each iteration, this work focuses on:

• Identifying the objectives for, and deliverables of, the testing effort

• Identifying a good resource utilization strategy

• Defining the appropriate scope and boundary for the test effort

• Outlining the Test Strategy that will be used

• Defining how progress will be monitored and assessed

Verify Test Approach

The purpose here is to demonstrate that the various techniques outlined in the Test Strategy will facilitate the planned test effort. The intent is to verify by demonstration that the strategy will work, will produce accurate results, and will be appropriate for the available resources.

The objective is to gain an understanding of the constraints and limitations of each technique as it will be applied in the given project context and to either

• Find an appropriate implementation solution for each technique, or

• Find alternative techniques that can be used.

This helps to mitigate the risk of discovering too late in the project lifecycle that the approach to test is unworkable.

In each iteration, this work is focused on these tasks :

• Verify early that the intended test strategy will work and produce results of value.

• Establish the basic infrastructure to enable and support the test strategy.

• Obtain commitment from the development team to develop the software to meet the testability requirements to achieve the test strategy, and to provide continued support for those testability requirements.

• Identify the scope, boundaries, limitations, and constraints of each technique.

Validate Build Stability

Before entering a test cycle for a new Build, we validate that this Build is stable enough for detailed test and evaluation effort to begin. This work is also referred to as smoke test, build verification test, build regression test, sanity check, or acceptance into testing. This work prevents wasting resources on a futile and fruitless testing effort.

Test and Evaluate

Typically performed once per test cycle, this workflow detail involves the core tactical work of the test and evaluation effort, namely, the implementation, execution, and evaluation of specific tests and the corresponding reporting of incidents that are encountered .

For each test cycle, this work is focused on these tasks:

• Provide ongoing evaluation and assessment of the target of test.

• Record the information necessary to diagnose and resolve any identified defects or issues.

• Achieve suitable breadth and depth in the test and evaluation work to be able to provide feedback on the most likely areas of quality risk.

Achieve Acceptable Mission

The objective is to deliver a useful evaluation result to the stakeholders of the test effort, where useful evaluation result is assessed in terms of the Evaluation Mission set up at the beginning. In most cases that will mean focusing your efforts on helping the project team achieve the Iteration Plan objectives that apply to the current test cycle.

For each test cycle, this work is focused on these tasks:

• Actively prioritize the minimal set of necessary tests that must be conducted to achieve the Evaluation Mission.

• Advocate the resolution of important issues that have a significant negative impact on the Evaluation Mission.

• Advocate appropriate quality.

• Identify regressions in quality that have been introduced between test cycles.

• Where appropriate, revise the Evaluation Mission in light of the evaluation findings so as to provide useful evaluation information to the project team.

Improve Test Assets

The objective is to maintain and improve the various test assets: Test Ideas List, Test Cases, Test Data, Test Scripts, and so on. This is important especially if the intention is future reuse, in subsequent test cycles, of the assets developed in the current test cycle.

For each test cycle, this work is focused on these tasks:

• Add the minimal set of additional tests to validate the stability of subsequent Builds.

• Remove test assets that no longer serve a useful purpose or have become economic infeasible to maintain.

• Conduct general maintenance of and make improvements to the maintainability of test automation assets.

• Assemble Test Scripts into additional appropriate Test Suites.

• Explore opportunities for reuse and productivity improvements.

• Maintain Test Environment Configurations and Test Data sets.

• Document lessons learned ”both good and bad practices ” discovered during the test cycle.