Testing Incremental Projects

Fewer and fewer software development projects build a complete system from the raw products of a programming language and a comprehensive set of requirements. Many are "remodeling" projects in which a portion of an existing system is replaced with updated, often expanded, functionality. Commercial-off-the-shelf (COTS) software is used when possible to provide portions of the functionality of a system, whether it's completely new or renovated.

For example, one project we are working on as we write this is the incorporation of a voice-recognition engine; another is using a spreadsheet component comprised of several classes; and yet another is using a commercial application server as the middle layer in a three-tiered architecture. So does this, and should it, affect the test cases we use and the levels of coverage we require?

Any product that is purchased from a vendor to be included in a product or to be used to help produce a product should be subjected to an acceptance test by project personnel. Components that are purchased as a library from a vendor and components that come included in a product such as a development environment should be tested. We will discuss this in Chapter 10.

For projects that focus on a portion of a development effort, such as the renovation of a piece of legacy software, the basic testing phases remain but the relative importance of the phases shift. We have participated in a number of these projects over the years. Our experience has shown that each project should develop a test plan that clearly defines the amount of testing to be conducted at each phase. The amount is determined by the quality of the product being renovated, the depth of the renovation, and the quality demanded by the domain.

Legacy Projects

For legacy projects, in which a portion of the existing functionality is being replaced with new code, two types of testing are emphasized. First, guided inspection is conducted to ensure the quality of the interfaces that are designed to isolate the new portion of the system from the remaining legacy code. These interfaces are intended to represent at least the functionality that existed previously in the code that is being replaced. The interfaces to the legacy are complete if they provide all of the behaviors needed by the extension software to be able to implement their behavior. The interfaces are consistent if a behavior in the legacy code is accessed through a single entry point in the interfaces. The interfaces are correct if they accurately transmit the behavior of the legacy code.

The second type of testing that is emphasized is integration testing. This focuses attention on the points at which the new code actually interacts with the legacy code. Depending on how the modifications are implemented, you may find issues of compatibility among even primitive data types if the languages are different across the interfaces.

Protocols between the legacy and extended code should be identified, documented, and tested. This includes the usual messages going back and forth, and the exceptions that each throws that are caught by the other. To digress for a moment, a widely used design pattern, Facade, formalizes the protocol into an object. We use this approach so that if more of the legacy system is later converted to objects, the protocol objects will be thrown away and a new set will be created at the new interface between legacy and object code.

For both levels, the issues are relatively the same:

1. consistency of ranges of values used as parameters across legacy boundaries in both directions

2. consistency of new and existing state machines

3. completeness of the interfaces

4. correctness of the new computations

The system test plan should identify the interfaces being created in the project and should plan for extensive testing of those interfaces.