4.5 Error Cases and Exceptions

One aspect of our program has not been explicitly dealt with either in the tests or in the implementation: error cases. In general, we have to distinguish between two different categories of error cases:

The first error category includes errors we expect and would like to catch within the application. Correct handling of such cases has to be considered in our tests.
The second error category includes errors we cannot foresee or those we can handle only with great effort. Such errors often point to programming errors and normally cause the application or a part of it to abort. In such a case, all that remains to be tested is to ensure that the application ends in a controlled way.

These two different error types can be represented in Java by checked exceptions or runtime exceptions, but this does not necessarily have to be the case. For example, expected faulty behavior is often marked by explicit return values, such as a result object that understands the isError() message or an implicit coding (e.g., -1 or null). On the other hand, it is absolutely common to transform runtime exceptions on one level into checked exceptions on another level, and vice versa.

Consistent handling of errors and exceptions is not trivial and requires frequent refactoring steps, especially in complex applications. A detailed discussion of this issue would go beyond the scope of this book; in fact, it could fill a book in itself. ^[7] Moreover, the test-first approach cannot solve this problem for us, but it forces us to think about consistency before implementing an error handling method of one type or another. The result of this consideration is more test cases with different test objectives:

The occurrence of expected errors causes the correct error object to be returned or the correct exception to be thrown.
The error object or the exception is correctly handled in the calling "client" object.

Let's try to transfer this finding to our example: So far, we have assumed that the Reader we passed to the DictionaryParser contains exclusively syntactically correct entries. But this assumption is more than naive, because the source we aspire to contains dictionary files written by humans. So we will first list a few errors that can potentially occur in the syntax:

A line is empty.
A line does not contain the "=" character.
The equal sign character (=) is preceded or followed by an empty string.
The word before or after the = character has blanks in its margins.

Note that this list does not include all conceivable error cases, but only a few typical ones. Our goal is to trace the most frequent problem cases rather than check for all potential errors. It is now a matter of finding the desired behavior of the parser. Cases 1 through 3 are obviously not suited for a meaningful parsing of the line, so we expect here the throwing of a DictionaryParserException. The following test results from error case 1:

 public void testEmptyLine() throws IOException,    DictionaryParserException {    String dictText = "Buch=book\n" +       "\n" +       "Auto=car";    parser = this.createParser(dictText);    this.assertNextTranslation("Buch", "book");    try {       parser.nextTranslation();       fail("DictionaryParserException expected");    } catch (DictionaryParserException expected) {}    this.assertNextTranslation("Auto", "car");    assertFalse(parser.hasNextTranslation()); }

Two things are unexpected in this test. First, we embedded an empty line between two correct lines. Especially when passing exceptions, it is meaningful to check our OUT for correct "continued functioning." The state of an object changes often before an error condition occurs, and the exception handler does not set it back afterwards. For this reason, the test ensures that everything continues functioning after the error. ^[8]

The second thing is the pattern used to check an expected exception:

 try {    parser.nextTranslation();    fail("DictionaryParserException expected"); } catch (DictionaryParserException expected) {}

I think that this check is simple and intuitive. As an alternative, JUnit offers a class, junit.extension.ExceptionTestCase, for the same purpose, but I find its use complicated, to say the least, because an (anonymous) subclass of ExceptionTestCase has to be created for each exception under test. In addition, the approach we selected expands easily in case we want to study the expected exception more closely, notably, with regard to its message string:

 try {    parser.nextTranslation();    fail("DictionaryParserException expected"); } catch (DictionaryParserException expected) {    assertEquals("message", expected.getMessage()); }

To induce the testEmptyLine() method to do faultless compiling, we have to create the DictionaryParserException class and add it to the throws clause of the nextTranslation() method. As a consequence, the majority of the methods in DictionaryParserTest cannot be recompiled until we give them DictionaryParserException. Whether we declare each single exception type or a simple throws Exception in the test methods is a matter of taste. Although the approach selected here documents the occurring exceptions much better, it also has a much higher adaptation cost.

The necessary change in the application code looks like this:

 public class DictionaryParser {    ...    public void nextTranslation()       throws IOException, DictionaryParserException {       if ("".equals(nextLine)) {          this.readNextLine();          throw new DictionaryParserException();       }       int index = nextLine.indexOf('=');       currentGermanWord = nextLine.substring(0, index);       currentTranslation = nextLine.substring(index + 1);       this.readNextLine();    } }

Accordingly, we can now define the test methods for cases 2 and 3:

 public void testLineWithoutEquals() throws Exception {    String dictText = "Buch=book\n" +       "Auto car\n" +       "Auto=car";    parser = this.createParser(dictText);    this.assertNextTranslation("Buch", "book");    try {       parser.nextTranslation();       fail("DictionaryParserException expected");    } catch (DictionaryParserException expected) {}    this.assertNextTranslation("Auto", "car");    assertFalse(parser.hasNextTranslation()); } public void testLinesWithEmptyWords() throws Exception {    String dictText = "Buch=book\n" +       "Auto=\n" +       "=car\n" +       "Auto=car";    parser = this.createParser(dictText);    this.assertNextTranslation("Buch", "book");    try {       parser.nextTranslation();       fail("DictionaryParserException expected");    } catch (DictionaryParserException expected) {}    try {       parser.nextTranslation();       fail("DictionaryParserException expected");    } catch (DictionaryParserException expected) {}    this.assertNextTranslation("Auto", "car");    assertFalse(parser.hasNextTranslation()); }

Once again, modification and refactoring of the nextTranslation() method is left to the reader.

Case 4 differs from the first three cases in that the words are meaningful, although they are enclosed in blanks. For this reason, we would like to ignore the blank at the beginning or end of a word:

 public void testSpacesInWords() throws Exception {    String dictText = "  Buch  =book\n" +       "Auto=  car  \n" +       " Buch=volume \n" +       "Modultest=unit test ";    parser = this.createParser(dictText);    this.assertNextTranslation("Buch", "book");    this.assertNextTranslation("Auto", "car");    this.assertNextTranslation("Buch", "volume");    this.assertNextTranslation("Modultest", "unit test");    assertFalse(parser.hasNextTranslation()); }

Again, we identified different cases, including one that should retain the blanks within a word. The questions we have to ask now—How many cases should be involved in the first attempt? How many should be added when reviewing the tests? and How many will eventually be added later when a problem occurs in practical operation?—can be answered based not least on experience and feedback. In the event that defects are found again and again in the shipped software, we have to invest more time in the test case creation. In contrast, if finding test cases devours 80% of available resources, we would do better to analyze the cost/benefit situation.

So far, our tests have checked whether a faulty Reader leads to the correct exceptions. Our next step involves correct handling of these "exceptions" in the "client," the Dictionary class:

 public void testInvalidTranslationsInReader() throws Exception {    String dictText = "Buch=book\n"+       "\n" +       "Buch volume\n" +       "Auto=car";    Reader reader= new StringReader(dictText);    dict = new Dictionary(reader);    assertEquals("dict size", 2, dict.size());    assertEquals("translation Buch", "book",       dict.getTranslation("Buch"));    assertEquals("translation Auto", "car",       dict.getTranslation("Auto")); }

Note that we had to deal with our actual test goal—check for correct handling of DictionaryParserException—by the back door. We created a Reader so that, based on our insider knowledge, it will throw the desired exceptions. This is a typical white-box test. Also, the correct handling— ignore faulty entries—was tested over a side effect, and we expanded the interface of the Dictionary class by the size() method for this purpose. This indirect approach is typical when a test includes several objects.

To prevent a sheer return of a constant from happening in the implementation of size(), we also add an assert statement verifying the size of the Dictionary object to all other tests, as in the next example:

    public void testTwoTranslations() {       dict.addTranslation("Buch", "book");       dict.addTranslation("Auto", "car");       assertFalse("dict not empty", dict.isEmpty());       assertEquals("dict size", 2, dict.size());       ...    }

And finally, a test that checks whether or not an IOException from the DictionaryParser also reaches the surface in the Dictionary constructor would definitely be useful:

    public void testIOExceptionFromReader() {       Reader reader = new StringReader("") {          public int read(char cbuf[], int off,             int len) throws IOException {             throw new IOException();          }       };       try {          dict = new Dictionary(reader);          fail("IOException expected");       } catch (IOException expected) {}    }

One disturbing thing that makes the test hard to understand is that generating the IOException requires knowledge about implementation details of the StringWriter class. At times, it is even impossible to generate the desired exception in the conventional way. Fortunately, in Chapter 6, Section 6.7, we learn a technique to avoid exotic test tricks in most cases.

^[7]To our knowledge, this book has yet to be written. Do you feel like writing it?

^[8]Keith Stobie [00] discusses this problem and similar ones during exception testing.