Bug-Tracking Tools

---

At last I’ve arrived at the portion of the chapter that you might have been expecting right from the start: using a tool to track bugs. Tracking bugs is one of those best practices that every developer should use on every application of any size or complexity. Keeping track of bugs— and their resolutions—is essential for two reasons. First, it’s the best way to make sure that a bug actually gets resolved, rather than merely being forgotten. Second, it gives you a way to see what common mistakes have plagued your work in the past, and helps you figure out where to concentrate your code-improvement efforts. In the final part of this chapter, I’ll show you how to choose and use a bug-tracking tool in your own development.

Choosing a Bug-Tracking Tool

There are dozens of bug-tracking tools available; indeed, if you include tools that are used within a single company, I’m sure there are thousands. These tools range from freeware applications to enterprise-level applications that will cost you hundreds of dollars per user. Some are cross-platform, some run as Windows applications, and some run as web applications. One list of bug-tracking applications is at http://testingfaqs.org/t-track.html; a search on Google will turn up even more alternatives.

Given this overwhelming variety of tools, how do you choose the one that’s right for you? To some extent, it’s going to be a matter of random chance. Few of us can afford the time to test out dozens of alternatives for a single tool. By asking developers you trust and reading reviews, you can probably come up with a shorter list to take a look at. When you do, here are some issues you might consider:

• Can you afford this particular tool? Consider not just the price of licensing, but also the cost of possibly needing a dedicated server or a database to go with it. If it’s out of your price range, regardless of how good the tool is, it’s not worth evaluating.

• Do you need to pay a license fee for each user, or is there a onetime price? Will there be additional costs as your company grows and you hire more developers and testers?

• Do you need to be able to track bugs on more than one platform (such as Linux and Windows)? If so, you’ll want to limit your search to cross-platform or web-based solutions.

• Do you need to share bug-tracking information with a distributed team? If so, make sure your solution for bug-tracking includes a web or other remote component so that team members in remote locations can share in the process.

• Does the tool send e-mail notifications to users when they are assigned a bug? If not, you’re going to have to find some other way to force developers to actually keep an eye on the bug list.

• Does the tool have a web-based or other interface that allows people outside your organization to enter bugs? If not, you’ll have to retype any bugs you get from beta testers in order to get them into the system.

• Does the tool allow adding custom pieces of information to each bug? If not, you’re at the mercy of the tool developers as far as what information you can track. This is fine, as long as you can’t think of anything that you critically need to know that’s not on the list.

• Does the tool distinguish visually between bugs and feature requests (or sort them)? If not, you’ll probably end up using two different tools to track bugs and feature requests, which means developers need to look yet another place to do their jobs.

• Does the tool integrate with your other management tools? In particular, integration with requirements management tools and source code control tools can be useful as you attempt to trace exactly what happened with a particular feature.

• Does the tool allow a flexible workflow where, for example, a bug might be closed and then reopened when the tester discovers that it wasn’t fixed after all? If not, you’re going to be trying to adopt your process to the tool rather than the other way around, which is usually a bad idea.

• Where does the tool store information? If it’s in an XML file or a SQL Server database, for example, you have some assurance that you can get the data back out if you ever switch tools (perhaps because the original vendor went out of business). If it’s all in some proprietary format, you’re staking your ability to remember history on the vendor’s continued existence.

TECHNOLOGY TRAP: The Burden of Process

For some reason, bug tracking seems to attract people who believe that life is best structured with a very detailed process. I’ve seen some bug-tracking systems that made entering a bug a 30-minute process, by the time you collected all of the required information (library versions, operating system version, amount of RAM, other applications running at the same time…) and looked up the responsible developer so you could properly assign the bug.

Loading too much process on top of your bug-tracking system is dangerous. If you make the process of entering a bug difficult, people will not react by being more diligent about collecting the “necessary” information. Rather, they simply won’t enter bugs at all. Instead, bug reports will come to developers via e-mail, telephone calls, and hallway conversations. The bug system will deteriorate into a sham, used only by managers and those testers too new to know any better.

If you’re working with a team and you discover that the bug-tracking system isn’t getting used, find out why. Most likely you’ll discover that it’s too cumbersome to enter a new bug. Get rid of the excess process, or find a tool with less process, to make your team happier and better track your bugs.

Beyond considering this list and looking at feature comparisons, it’s important to spend a bit of time trying out any bug-tracking tool you’re going to use. That’s the only way to make sure that it’s not an impediment to your workflow. Ideally, you want entering and tracking bugs to be so easy that testers and developers are never tempted to work around the tool. Fortunately, most vendors in this area offer some sort of trial version so that you can get a feel for functionality.

Using a Bug-Tracking Tool

For my own projects, I’m currently using FogBUGZ (www.fogcreek.com/FogBUGZ/index.html) as a bug-tracking tool. FogBUGZ suits my requirements well (being web-based, it’s useful with the far-flung network of partners I have, and it integrates directly with my SourceGear Vault source code control system), though it’s hardly the only package I could be using.

Figure 9.1 shows a typical FogBUGZ bug report. In this case, it’s a closed bug from a vertical-market application that I helped develop (I’m not using Download Tracker for this example because I haven’t yet started building up a database of Download Tracker bugs).

Figure 9.1: A sample bug report

The first thing to notice here is the workflow. The bug was opened by someone testing the application and then assigned to me, since I was the developer of the feature in question. I fixed the bug, resolved it as fixed, and FogBUGZ automatically assigned it back to the person who originally entered it. He then tested the software again, agreed that the bug was fixed, and closed the report. It’s important that developers not close their own bugs (indeed, good bug-tracking software won’t even allow this). Otherwise, the temptation is simply too great to implement a quick-and-dirty “fix” that doesn’t address the entire problem, and to sweep any problems back under the rug.

The bug report itself, while short, includes the three essential pieces of information that should be identified for every bug:

• What happened

• What the tester thinks should happen instead

• Steps to reproduce the problem

Without these three pieces of information, it’s difficult or impossible to fix bugs (or even to know if they are bugs). Sometimes you don’t need to be explicit about what should happen; if a particular series of steps causes your software to erase the user’s hard drive, it’s pretty obvious that “not erasing the hard drive” is the desired outcome (unless you’re writing disk-partitioning software). But when in doubt, a good tester will be explicit when entering a bug report. Reproduction steps are especially critical. The easier it is for a developer to see the code actually failing, the more likely they are to be able to fix the problem.

You can also see the effect of FogBUGZ-Vault integration in this example. Because I entered the bug number when I checked the fixed files into Vault, those files are listed on the bug report together with the appropriate version number. The hyperlinks in the bug report let me see the history of the affected file or the actual affected code.

This particular bug went through its entire life cycle in a matter of days, with most of the work being concentrated in a few hours. That’s not unusual for bugs in applications that I work on. With tiny bugs, the temptation will always be there to just e-mail or phone the developer to get it fixed. Resist the temptation! By putting all bugs and feature requests in the bug-tracking system, no matter how trivial, you’ll gain two big advantages. First, managers can be sure that there’s a single place to go that contains every potential work item, and use it to monitor the workload. Second, developers can use the bug-tracking system to prioritize their work, because everything they need to do is stored in that single place.

RULE

Integrate bug tracking with your daily planning routine.

---