4.4 Threshold Values and Equivalence Classes

One important heuristic for effective testing is to increasingly test along the boundaries of a permitted value range because these are the areas where most errors occur. This plays an essential role in many situations, such as when selecting input parameters. For example, when testing the factorial function of our mathematics library, the numbers 0, 1, 2, and MAXINTEGER are better input values than 5, 12, 69, and 101. The same heuristic suggests we also test a String input parameter with both an empty String and a very long character string.

However, the boundary rule does not apply only to input parameters. Boundary cases also exist with regard to the sizes of input files (e.g., length = 500MB), the number of a method's calls, the size of Collection objects, and many other factors. The more we think about this, the more points with potential problems in boundary areas we will identify. For example, although we tested DictionaryParser with an empty reader, we have not tested it with a large number of translation entries, which could lead to a buffer overflow or cause other problems. Let's try at this point to see how it works:

 public void test10000Lines() throws IOException {    StringBuffer buffer = new StringBuffer();    for (int i = 0; i < 10000; i++){       buffer.append("Wort"+i+"=word"+i+"\n");    }    parser = this.createParser(buffer.toString());    for (int i = 0; i < 10000; i++){       this.assertNextTranslation("Wort"+i, "word"+i);    }    assertFalse(parser.hasNextTranslation()); }

In addition, our tests have not yet paid any attention to the case where translation entries have empty strings. Empty strings do not represent a meaningful German word nor an English translation, so we postpone this issue to Chapter 4, Section 4.5, Error Cases and Exceptions.

Brian Marick offers a small catalog of test ideas at [URL:TestingCat], including more boundary cases interesting to test.

Finding boundary cases can be very easy at times, as when the maximum admissible number of lines is stated in the specification. In most cases, however, we need to have a close look at the implementation to discover real boundary cases. Knowing only that an index type int is used to access a specific object will enable us to see MAXINTEGER as a th-reshold value.

Another commonly used example is the following sort function:

 private final static int MIN_QUICKSORT = 15; public List sort(List unsorted) {    if (unsorted.size() < MIN_QUICKSORT) {       return bubbleSort(unsorted);    } else {       return quickSort(unsorted);    } }

Analyzing the code is the only way for us to find that 15 is a threshold value which leads to two different code execution paths. Full code coverage requires that both paths be tested, necessitating two additional test cases, with 14 and 15 elements, respectively. When the space of all possible test cases is split in this manner, the space is said to be composed of two equivalence classes.

Equivalence classes are an old but still central concept in testing theory. Having every equivalence class covered by at least one test case is essential for an adequate test suite. Cem Kaner [93] defines equivalence class as follows:

If you expect the same result ^[5] from two tests, you consider them equivalent. A group of tests forms an equivalence class, if you think that

They all test the same thing.

If one test catches a bug, the others probably will too.

If one test doesn't catch a bug, the others probably won't either. ^[6]

The reasons why we consider these tests equivalent are normally found in the implementation details: the test cases of an equivalence class use the same input variables, manipulate the same output variables, and have a similar internal control flow. Equivalence classes in test cases often result from exceeding threshold conditions with regard to our input values. An object-oriented particularity in this context is polymorphism, which leads to a situation where the class membership of an object used can change the control flow, as we will see in Chapter 7. Thinking about test equivalence classes often leads to the discovery of new and yet unconsidered test cases. For example, equivalence classes for invalid input are being neglected by many programmers.

^[5]Here, "the same" obviously does not mean identical.

^[6][Kaner93, p. 126].