Using the Information in the Bug Tracking Database

Consider the following questions:

• What areas of the software you're testing have the most bugs? The fewest bugs?

• How many resolved bugs are currently assigned to Martha?

• Bob is leaving for vacation soon. Will he likely have all his bugs fixed by then?

• Which software tester has entered the most bugs?

• Can you please bring a list of all the open Priority 1 bugs to the project review meeting?

• Does the software look like it's on track to meet the scheduled release date?

These fundamental questions are routinely asked over the course of a software project. They aren't rocket science, they're simple, straightforward questions to which you and the rest of your test team and the project team will eventually need to know the answers.

It may be surprising that a bug-tracking database can become such a fundamental means for measuring a project's status and answering such important questions. If you didn't know better, you'd think it would be the master schedule or the project plan or something that the project manager handled. In reality, though, those documents reflect the project's original intentionsthe bug database reflects the project's reality. If you want to choose a high-quality restaurant, you could select one based on the chef's résumé or the owner's history. But, if you want to be sure to pick a good one, you'd read the latest food critic review or the history of health inspection reports. The project's bug database works the same way. It tells you what has happened in the past, what's happening now, and allows you to look at the data to make an educated guess of the future.

NOTE

The term used to describe a measurement of a particular attribute of a software project is a software metric. The average number of bugs per tester per day is a metric. The number of bugs found per area of the software is a metric. The ratio of Severity 1 bugs to Severity 4 bugs is a metric.

Because the bug database is continually updated with new bugs, bug entry and fix dates, project member names, bug assignments, and so on, it's the natural means to pull all sorts of metrics that describe the project's statusas well as an individual tester's or programmer's status.

Therein lies one of the potential problems with using the bug database for metrics. The same database that can tell everyone how many Priority 1 bugs are still left to fix can also tell management how many bugs were created by a specific programmer. It can also tell your boss how many bugs you entered compared to the other testers on your team. Is publicizing that information a good thing? Maybe, if the programmer is very good and you're a great tester. But, what if you're testing a good programmer's code? There might be fewer bugs to find and your bug-find metrics suddenly wouldn't look so hot compared to other testers testing some really bug-ridden code.

It's not the intent of this chapter to get into the moral and interpersonal issues that can arise from how the data in the bug database is used. In general, though, it should primarily be used to track project-level metrics, not an individual person's performanceunless the metrics are private, understood, and unambiguous. If you're working on a project that uses a bug-tracking database, discuss with your manager and the project manager what information will be collected and how it will be used so that there won't be any surprise expectations.

Politics aside, using the bug database as a source for metrics is a super-efficient means to gauge a project's status and your own progress. All the information is there, it's just a matter of pulling it out of the database and arranging it into a useful format. The remainder of this chapter will discuss some of the common metrics that you'll see used in software projects and explain how they're generated and interpreted. Of course, projects vary greatly, so don't assume that these are the only metrics possible. Just when you think you've seen the weirdest possible pie chart, someone will think up another that demonstrates a new and useful view into the project's data.