Testing Off-the-Shelf Components

Increasingly, functionality is added to an application by purchasing "chunks" of software referred to as components.^[7] The quality of these components varies tremendously from one vendor to another. Until standardized measures are adopted or the marketplace forces improved quality, you should plan to do an acceptance test on any newly acquired component.

^[7] Chapter 10 will cover topics about components, but this is a natural place to talk about them.

An acceptance test should put the component into the context in which it will be used. The test cases should thoroughly investigate the limits of the specification.^[8] Creating the specification document will not be a wasted effort because developers will need it in order to properly use the component.

^[8] Create a formal specification if one does not exist.

We like to begin an acceptance test with extreme, even incorrect, values for example, running the mouse back and forth across the desk to generate a large number of mouse move events. A defective component may be overwhelmed by the large number of events and crash. Other stress tests include holding down a "repeat" key or making multiple menu selections before the program can respond and gray out certain selections on a menu. This is as much a test of the component manufacturer's attention to detail as it is a test of the software. If there are many failures here, you have to suspect that the quality is fairly poor.

If we continue beyond that set of tests, the testing of this component proceeds along the lines of any class. Even if the component is constructed from several classes, there is usually a main class that presents the component to the user. The tests are based on that class.

A Case Study in Component Acceptance Testing

Let's consider a commercially available Grid user-interface component and examine how we would test it before using it in an application.^[9] Figure 6.19 presents the test plan for the Grid component. Figure 6.20 presents a life-cycle scenario, which is a description of one specific use of the component that can be used to build certain types of test cases.

^[9] We use an actual product, but we have changed the name to avoid legal hassles.

Figure 6.19. A component test plan for the component Grid

Figure 6.20. A life-cycle scenario

A grid JavaBean displays information in row and column format. It allows users to select, manipulate, and store information presented. The product we will consider contains 4 interface definitions (implemented as abstract classes in C++) and 10 public class definitions. Additional classes are nested within these public classes. Of the 10 public classes, two GridMain and GridBigAdapter are the classes that developers use to integrate the component into their application. The documentation comprises standard JavaDoc HTML pages. These contain comments placed by the class developers, but nothing about the "component." Each method is presented in the form int compareTo(java.lang.Object anotherObject).

The compareTo() method returns a value of type int and requires one parameter, anotherObject, which is of type Object. There is no information about any constraints on anotherObject nor any indication of the range for the return value, even though our analysis discovered that it can only take on three different values.

The documentation does not provide a state diagram to use. Substitute scenarios in which a life cycle of uses is defined. A sample scenario is shown in Figure 6.20. These tests may be combinations of other specification-based tests.

GridMain has over one hundred methods. Many of them are simple accessor methods, but a large number are modifier methods that set specific attributes in the object. While testing could be a large job, a component will provide a large amount of functionality and, therefore, conducting a thorough test at one time will save much effort for the many developers whose objects will interact with the component.

An acceptance test combines elements of a class test and an interaction test. Therefore, we are interested in the patterns of interaction of this component with the rest of the system as well as the specification of the individual methods on the interface of the component. This component follows a standard Java user-interface design pattern. It uses an Adapter class to support the creation of the Listener objects needed to capture various types of events. An interaction test of this component should follow that pattern to achieve an effective interaction test.

First, let's analyze. Grid is primarily a collection class. It holds and displays data and has very little interaction with the objects that it holds. The major interactions that it has are with the event producers. The interaction that it has with its contents is to display them, store or retrieve them, and forward events on to them. Another type of interaction is when one object requests that a grid provide the object stored in a specific cell. Is the requesting object handed a clone? A reference? Does this action prevent the grid from being garbage collected?

A few simple interactions that the grid is intended to have include the following:

1. A mouse button click occurs in a certain cell of the grid.

2. A mouse button release occurs in a certain cell of the grid.

3. A mouse button is double clicked in a certain cell of the grid.

A test harness should be created that consists of a specialized Adapter that listens for these events and at least logs when the event has happened and has been handled. The tests should automatically create events for a variety of actions and pass the events to the grid. The resulting behavior of the grid should be evaluated.

A more complete interaction test would examine the complete life cycle of a grid. Create a few life-cycle scenarios that briefly describe typical uses of the component under test, as shown in Figure 6.20. The test harness would instantiate Grid with a variety of data types from the current application in the cells. The test harness should stimulate the grid to read its contents from storage and display it. The tester should perform a series of mouse actions. Another object should request and hold the contents of at least one cell in the table. The test harness should stimulate the grid to save the data and finally destroy the grid. Validating the test requires that the tester observe the visual behavior of the grid as the events are created, and check that the garbage collector can remove the grid while an object holds a reference to one of the content objects.

Tip

Use the basic logic of the manufacturer's sample programs as the basis for individual test cases. We use our standard test driver and then build test cases by beginning with the basic code from sample programs.

Incidentally, we found several problems during our acceptance test. These were submitted as bugs to the component company and were subsequently fixed in later releases.