100 Go Code and 80 Exception Handling

100% Go Code and 80% Exception Handling

What do we mean by this, exactly? One of your project goals should be that line and branch code coverage should be at 100% so that you do not have any obvious coverage holes. You should also be checking to make sure that you have at least 80% exception handling in your code; otherwise exceptions might get lost and go out to the user directly, rather than being captured by the application and showing something usable.

Now that we know this is our goal, how does jcoverage assist us in achieving it, and what are some of the weaknesses of the methods it uses so that we are aware of them and can work around them?

Statement or Line Coverage

As we discussed earlier, statement or line coverage is the indication of the degree to which individual statements or lines are called during a test execution; you can see this when you look at an individual class report:

This report shows us how often a particular line was called during the test execution. jcoverage will show 0 counts and highlight the fact that the line has yet to be called. This is a very handy feature that enables you to quickly verify how your code is being called in tests.

One of the drawbacks of line coverage is that it doesn't indicate how much of your logic has been tested , only the particular lines that have been executed. This is where branch coverage steps in.

Branch Coverage

Branch coverage gives an indication of how our logic does or does not get called during tests. If we look at our example again, we see that by changing the addInt function to something like this:

 public int addInt(int newInt) {         if(newInt > 0) {             return myInt + newInt;         } else {             return myInt;         }     } 

that our test code will not sufficiently capture the less than 0 conditional. jcoverage indicates this by showing a 0 line count in our class coverage report, and by decreasing our branch coverage percentage to 85%.

As we can see from this report screen we have now reduced our overall coverage by missing this simple branch test. But, some of the drawbacks of Branch coverage are that it may not capture the conditional coverage properly. We see that here when we change the setProperty method to look at different string types.

 public void setProperty(String prop) {         if(prop == "mystring"  prop == "someotherstring") {             this.prop = prop;         } else {             this.prop = "mystring" + prop;         } } 

Here is our corresponding jcoverage output:

As you can see, this does not truly capture our possible condition coverage. We are not testing all three conditions, but jcoverage only counts the two as missing (one for the "mystring" check, and the other for everything else), not the third option of "someotherstring".

Exception Handling

Here are the two classes in questionyou will see the highlighting to indicate the Exception being thrown, and highlighting again where it is caught:

By changing the way our test runs, we can easily test whether or not an exception will get thrown, but it is the base code in Hello.java that needs to be changed in order to do something with the exception once it is caught. Another item that jcoverage doesn't completely pick up on is the branch coverage for the exception. Even though it shows 100% branch coverage, we have not exercised the two possible branches here to the greatest conclusion. jcoverage does pick up on the line coverage for the unexercised branch.

It can be very tricky to decipher what any ccoverage tool is telling you about exception handling. In our simple case it's rather straightforward. We are not exercising the Exception code either in the throwing stage or in catching. Unfortunately jcoverage does not have a built-in function to check that all exception code is exercised; it can only perform the more general line and branch coverage, but this it does well. Paying attention to when you should be doing something with an exception and when you aren't fully exercising all of your code will help you in determining whether or not your code is ready to go.

Diminishing Return

Now let's try and improve our percentage rating by testing our addInt() branch with a negative number.

 assertEquals(7, hello.addInt(-1)); 

So, what does this get us?

As you can see from this, jcoverage thinks that we are completely covering our addInt() method, increasing our percentage to 90%, however we haven't fully tested this method, we are missing the newInt == 0 condition.

 public int addInt(int newInt) {         if(newInt > 0) {             return myInt + newInt;         } else {             return myInt;         }     } 

Adding this method into our testing might increase our code coverage percentage, but jcoverage does not seem to catch this possible one-off error that we have run into. Our code coverage percentage stays at 90%. What to do? Do we stop writing tests simply because the coverage tool tells us we've reached our goal? My answer, of course, is no. By writing complete tests we again ensure that we are testing everything. The coverage tool is just there to help catch it before it becomes a big problem, or to put forward a goal for you to attain. And, if you aren't happy with one coverage tool, try anotherperhaps it will use a different algorithm that catches more.

Setting Goals

One of the nice things that you can do using jcoverage is to set absolute percentage goals using the check ant target.

 <check branch="95" line="95"/> 

By adding this into the report target, or the test target, we are able to send up a flare warning developers that their intended goals have not been met. jcoverage does this by failing the test class for whichever class falls below the defined percentages you have indicated are acceptable. You can also set goals on a per-package basis that look like this:

 <check branch="95" line="95">     <regex pattern="xptoolkit.jcoverage.*" branch="85" line="95"/> </check> 

This would verify that everything that wasn't covered by the regex statement would be at 95% for branch coverage and 95% for line coverage, and that the particular package would be at 85% for branch and 95% for line. This is particularly useful if you have an old package that is slowly getting tests added to ityou can ensure that your tests still pass, but you will be made aware of the deficiency if you suddenly stop writing new tests for it.