Practice2.Use Available Tools

Practice 2. Use Available Tools

A large number of tools are available to software developers that can help with defect prevention. Some of the available tools offer easily automated ways of detecting common coding errors. Other tools help a team communicate better through offering automated ways to document code and code structure; better communication helps prevent defects by making it easier for the team to understand code and its structure before attempting to change or enhance it.

There are some obvious tools, and some that aren't as obvious, as outlined in the following paragraphs. In most cases, these tools are very easy to set up and use, and many can be integrated into your product build for maximum effectiveness.

Teams that take the time to find the right tools for their projects and then integrate them into their daily workflow are definitely on the right track when it comes to defect prevention. Better tools are becoming available all the time, and it is important for a team to know what tools are available and how they could help.

Compiler

A compiler is the perfect example of a tool that virtually everyone uses, but that is not always as effectively as it could be. Compilers are really good at detecting common coding problems. Unfortunately, some potentially serious problems actually show up as warnings, and the problem with warnings is that they are too easy to ignore. You should understand the warnings that your compiler produces, prioritize the warnings, and, if at all possible, turn many of the warnings into errors. Then, make sure your code compiles without the warnings you care about. The extra flags won't add to your compile time; even if all the compiler does is catch what would have been a defect once a month, it's still worth the effort. That's one defect a month that is caught before your product reaches QA and your customers.

Source Code Analyzer

A source code analyzer is a tool that detects common coding errors that require more sophisticated analysis than that performed by a compiler. These tools are obviously language dependent and typically come with a default set of errors they look for, but they can also be customized so you can ignore certain errors or even add your own rules. Here is a very short list of common problems that a source code analyzer could catch for you.

• If you are using exception handling, it's almost always a bad idea to have an empty catch block, since it means that if the exception does occur, you have no control over how it is dealt with.

• Code that has probably been copied and pasted. It's obviously impossible to perform an exhaustive search for all duplicated code, but duplicated code is always bad because it means that bugs in the duplicated code need to be fixed in more than one place; what should be a single defect and fix almost always turns into many.

• Short global variable names. Global variables are rarely desirable, but if you have to use them, they should at least have meaningful names . . .!

• Switch statements that don't have a default case.

• Deeply nested if statements and method complexity. Complexity is usually a measure of how much nesting there is of statements.

• An object's constructor method calling a virtual method. The problem in this case is that the object could be partially initialized when the method is called, which could lead to an unknown state.

Source code analysis tools are a pretty recent development. One of the better tools that I have used is PMD for Java. PMD is open source (http://pmd.sourceforge.net) and requires less than a day to set up; most of the work involves turning off checking for undesirable rules. I sincerely hope that more open source projects are started to cover other common programming languages and development platforms and APIs, since every language and API has good and bad uses.

If you are not using Java, PMD also comes with a tool called CPD (copy-paste code detector). CPD works with Java, C++, and C and is well worth looking into because it is surprisingly good at finding code that has been simply copied and pasted. Copying and pasting code is a lazy shortcut that developers often resort to when they are, for example, creating a new class with a method almost identical to an already existing class. Copying code is never a good thing because if there is a bug in the copied code, invariably the bug will get fixed in only one place. CPD will help ensure code that should obviously have been refactored in the first place never makes it into the product code.

Test Coverage Analysis

A tool you should consider using is one that measures how many source code statements are "covered" by your tests. This measure of coverage can be used to help add tests to get more coverage and even to remove tests, which can be necessary when your tests take too long to run.

Test Coverage: Handle with Care!

Test coverage is an aid to your testing efforts and should never be a focal point. It is a metric that is most useful to ensure all areas of the code receive at least some desired level of testing coverage. Beyond that, the value is dubious in my experience.

A development team at a large well-known software company decided a few years ago to focus on achieving 100 percent code coverage in its tests. They actually did it, but found that their product was unreliable and unstable. The reason? In order to achieve the 100 percent coverage, the developers removed error checking for boundary conditions!

Execution Profiler

An execution profiler is a tool that helps pinpoint performance problems in code. Typically, these tools give you a report of the methods/procedures/functions in your code that are taking the most time in a given run. You can use these reports to examine the methods in question and determine if there are any obvious performance issues.

Execution profilers are often available in an IDE or as commercial tools or open source software. Examples of excellent commercial tools are VTune from Intel and Shark from Apple.

Event Logging

Event logging is a diagnostic tool that lets you put "print" statements in your code to record a log of execution. Instead of writing your own, pick up an open source logger, if you can, such as log4j (Java) or log4cplus (C++).

Source Code Documentation

Source code documentation tools typically extract documentation from your source code and generate a set of web pages. Some tools also produce class hierarchy diagrams and searchable indexes that let you find method and variable names quickly. Probably the best-known source documentation tool is JavaDocs, which is available as part of the Java language. If you aren't using Java, or need to produce documentation for many languages, there are many open source projects (I recommend doxygen), which you should evaluate before attempting to build your own.

Memory and Resource Leak Detection

Memory leaks result when a developer allocates memory for an object but doesn't free it, and she is using a language and/or run-time environment that has no memory management (such as C and C++). I use the term resource leak to refer to the related case when the developer uses a system resource (such as a graphics context) and stops using it without releasing it back to the system. Both of these types of leaks result in serious defects and unpredictable application behavior that customers definitely notice.

Tip: A Simple Leak Detection Method

One very simple leak detection method you can use is to redefine the memory and resource allocation and free calls with your own in your debug build. All your redefined versions need to do is increment or decrement a counter. Then, in your unit tests you could simply run all the tests and then print the value of the counter. If it isn't zero, you've probably got a leak somewhere that you could then track down with a real detection tool.

Luckily, there are good tools available to help detect memory and resource leaks. Some IDEs have basic memory leak detection built in, so all you need to do is turn on a compiler flag, and there are also commercially available tools such as Purify and Boundschecker.

Configuration Management

All assets developed for and used in a project should be stored in a configuration management (CM) tool. Having a versioned copy of all your changes will help prevent defects, because the good tools help the arduous and error-prone task of merging new code in with the existing code, especially when there are multiple developers working on the same piece of code. A good CM tool should allow a team to do basic version control (check-in, check-out, labeling or "stamping" the current version of all files in a product, branching of source trees, etc.) over a network and also provide some form of merging changes together.

The Value of CM

This practice seems completely obvious, but it is surprising how many software teams manage versions of their software simply by taking copies and not using a CM tool.

This example is admittedly extreme, but one of my friends worked at a company where the developers kept copies of their product on their local disk with only occasional attempts to keep their copies in sync with each other. One of the most distressing stories my friend related was when a customer received a new version of the product from one of the developers. The customer found a problem in the product and reported it, but this caused panic because the developer's hard drive had crashed and the backups were incomplete. The development team had to try and piece together source code that was as close as possible to what the customer had installed, largely working from memory of the changes made. They eventually fixed the customer's problem, but what took them weeks should have taken only minutes. The distressing part of the story is that after this crisis, the team continued developing as they had before!

Cost is not an acceptable excuse for neglecting to use a CM tool. There are many good CM tools available free or via open source such as CVS. If you have the money, it is also a good idea to buy hardware for your CM database that provides some form of robustness, especially in case of hard drive failures.

CM tools are extremely easy to set up and use, and a good CM tool will save you a ton of time. There is absolutely no reason to not extensively use a CM tool!

The Promise of Open Source

What if all the software teams in the world contributed to the creation of tools that are meaningful to software developers, especially for defect prevention? Commercial software vendors have by and large done a poor job of creating the tools that software developers require to be more efficient and prevent defects. Further, too many software companies invest effort in the creation of their own internal tools, usually because they have a "tailored" environment (or think they do), but also due to a lack of knowledge of what's available or even an inability to contribute because of legal reasons (the work most developers do during work hours is the property of the company).

Open Source tools like junit (and all its derivatives), pmd, log4j, cvs, bugzilla, and Eclipse all show the potential of open source for creating a more powerful software infrastructure for the industry. There is an awful lot of duplicated effort going into proprietary tools, and if even half of that effort were put into building a set of easy-to-use tools, we would all be better off.