Frameworks

A large number of software projects that use object-oriented techniques also use application frameworks as a basis for some part of their system. For example, the Microsoft Foundation Classes (MFC) provide a framework for PC-based windowing programs. The framework provides an architecture for interactive, visual programs and a library of classes that support application development [Pros99]. We have used two frameworks in the C++ version of Brickles. We used the MFC as the basic implementation of the graphical front end. We also used our own game framework as the basic structure for the game aspect of Brickles.

There are several topics that we will consider in this section. The final application based on a framework must be tested. The classes in a framework's library should be tested as they are created. We will also consider differences between testing applications built with frameworks purchased from an outside vendor as opposed to those frameworks created in-house.

Basic Issues

Frameworks are intended to be partially completed applications. The developers intend that their framework will serve as the basic design and implementation for several applications a set of programs that have some measure of commonality in their behavior. This results in a better return on investment than one-off systems. The framework concept is perhaps the most successful approach for reuse to date; although we will see that the product line approach broadens the possibilities for reuse.

A framework provides three specific ingredients: a software architecture, the major control logic for an application, and a library of ready-to-use components. The places where the application developer can provide problem-specific code are termed hot spots. Frameworks operate on the "don't call us, we'll call you" principle. At a given hot spot, the application developer provides code that will be called when the framework code determines that it is appropriate. The framework provides a specification that components must satisfy in order to be inserted at a given hot spot.

A particular framework usually can be associated with a specific domain or a specific subsystem for an application. The MFC, as already mentioned, is associated predominantly with the user interface subsystem of an application. We have participated in the creation and use of domain-specific frameworks in the telecommunication industry, and we have built ground support systems for earth-orbiting satellites.

Framework Testing Processes

Projects that involve a framework may need as many as three different types of testing processes. Ultimately, the team must test the products that they create. The process for testing an application being developed from a framework is very similar to the one for any object-oriented application that is being developed iteratively with an emphasis on reuse. We have already presented a testing process in Chapter 3 so we will not repeat the details here.

The primary difference in the testing process is the degree to which the classes in a framework are abstract. The classes that comprise the framework are tested as they are developed. The PACT approach described in Chapter 7 can be utilized to support the reuse of the test cases; however, there is a limit to how specific the tests can be.

To test a framework that is being constructed within your development organization, one of the steps in the testing process has to be Build sample applications using the framework. Ideally, the set of samples should be sufficiently complete to use all of the hot spots. We have used different versions of our game framework to create a number of applications over several years. This is, however, an expensive process. One of the things that we used, and this should be done early in the life of a framework, was guided inspection. We will consider types of inspection test cases and some inspection criteria specific to framework style applications in the next section.

This same approach can be used to do an acceptance test on a framework that an organization is considering for purchase. One question that testing can be used to investigate is whether the framework has the flexibility to make the changes needed for proposed (or intended) applications. A second question is whether the quality of the individual components in the framework is sufficient. This is accomplished using the set of use cases, which is constructed to cover the points at which flexibility is needed.

Inspecting a Framework

Since a framework is an incomplete application, the earlier discussions of Guided Inspection test cases hold. Here we will provide those test cases that are different. Also we will add new criteria that are appropriate for a framework.

Completeness

The specification for a framework should provide a complete description of the scope of the framework. That is, it should be possible to take a set of use cases and determine whether the application specified by the use cases is within the scope of the framework.

A framework should be able to pass the same type of inspection that any application can with regard to completeness. It should be possible to see how every use case for an application will be handled. The difference is that with the framework, a use case may be satisfied by the functionality of a hot spot. It is assumed, during the inspection, that a hot spot will be implemented.

Consistency

The specification of hot spots should provide sufficient information to prevent an application developer from specializing one hot spot in a way that is inconsistent with the way another hot spot has been extended. That is, there are times when a set of hot spots are related and the documentation of a hot spot should indicate any constraints that exist among the hot spots.

Correctness

The correctness of a framework is at a different level of abstraction than an application. Correctness at this level means that the appropriate information is being passed from the framework to the application-specific code, and that the appropriate types of information are being passed back from the application.

Specificity

A framework can be too specific. That is, it can constrain the framework to a set of possible applications that is too narrow. It can also fix certain system attributes that should be variable. The Guided Inspection test cases that should be included are as follows:

1. Given the set of use cases for each of several anticipated products, examine each hot spot and determine how the application would extend the framework to achieve the required functionality. Note places where the framework does not allow sufficient flexibility and where it would need to be modified.

2. Ask an independent set of domain experts (who have not participated in the framework design) to conduct a variability analysis. They make a list of attributes that should change from one application to another. The inspection team compares that information to the list of hot spots to determine that it is possible to vary these attributes.

Structuring Test Cases to Support a Framework

It is usually expected that an application developer will utilize a framework by inheriting from framework classes. The PACT approach (see Chapter 7) complements this on the testing side. The framework development team constructs test cases using the PACT approach. Application developers are then able to extend these test case classes to build their own test cases. The result will be less effort to build the test cases for the application or, by expending the same effort, better test coverage.