Overview of Our Testing Approach

Our goal is to test software as thoroughly as possible, while recognizing that time and money constraints are real concerns. Our approach to testing object-oriented software is based on academic research as well as experience we have gained in working with clients in a variety of industries, such as telecommunications and finance.

Under our approach, testing is not an afterthought. Testing is a process separate from the development process, but intimately related to it. We have a motto: Test early. Test often. Test enough. We favor the following iterative development process:

Analyze a little.

Design a little.

Code a little.

Test what you can.

Testing what you can includes both what you can do technically and what you can do under time and resource constraints. A surprising amount of beneficial testing can be done within an iteration. Regular testing can detect failures early and save reworking in subsequent iterations. System testing and acceptance testing follow the last iteration. However, if you can develop a software system incrementally, then you can perform system testing at the end of each increment.

What kinds of testing do we promote for object-oriented software?

• Model testing

• Class testing, which replaces unit testing

• Interaction testing, which replaces integration testing

• System (and subsystem) testing

• Acceptance testing

• Deployment/self-testing

Each of these is covered in this book. Our testing process will define testing activities for every development activity.

We do not believe that you will or even should apply everything we describe in this book. There are seldom enough resources available to a development effort to do all the levels and kinds of testing we describe. We hope you will find a number of approaches and techniques that will prove applicable, useful, and affordable for your project.

We now provide a rationale for our motto of, "Test early. Test often. Test enough."

Test Early

Instead of engaging system testers toward the end of a project, start them testing at reasonable points during the analysis and design phases of a project. Testing analysis and design models not only can help to uncover problems early in the development process (where they are fixed more easily and more cheaply), but it can also help to scope the size of the effort needed to perform adequate system testing by determining what needs to be tested.

Testing early and often implies that the representations of the software are abstract or incomplete.

Test Often

We firmly believe that an iterative, incremental sometimes also referred to as iterative enhancement development process is best suited to the vast majority of projects. As iterations are completed on analysis, design, and implementation phases, the products should be tested. After completion of the first increment, some testing takes the form of regression testing.

Test Enough

Complete testing of every aspect of a software system is infeasible. Resources spent on testing should be directed where they provide the best payback. We favor techniques that are based on risk analysis, the reuse of test cases, and the statistical sampling of inputs for test cases.