Getting Your Bugs Fixed


Way back in Chapter 3, "The Realities of Software Testing," you learned that despite your best efforts at planning and executing your tests, not all the bugs you find will be fixed. Some may be dismissed completely, and others may be deferred or postponed for fixing in a subsequent release of the software. At the time, it may have been a bit discouraging or even frightening to think that such a concept was a possibility. Hopefully, now that you know a great deal more about software testing, you can see why not fixing all the bugs is a reality.

The reasons listed in Chapter 3 for not fixing a bug were

  • There's not enough time. Every project always has too many software features, too few people to code and test them, and not enough room left in the schedule to finish. If you're working on a tax-preparation program, April 15 isn't going to moveyou must have your software ready in time.

  • It's really not a bug. Maybe you've heard the phrase, "It's not a bug, it's a feature!" It's not uncommon for misunderstandings, test errors, or spec changes to result in would-be bugs being dismissed as features.

  • It's too risky to fix. Unfortunately, this is all too often true. Software is fragile, intertwined, and sometimes like spaghetti. You might make a bug fix that causes other bugs to appear. Under the pressure to release a product under a tight schedule, it might be too risky to change the software. It may be better to leave in the known bug to avoid the risk of creating new, unknown ones.

  • It's just not worth it. This may sound harsh, but it's reality. Bugs that would occur infrequently or appear in little-used features may be dismissed. Bugs that have workarounds, ways that a user can prevent or avoid the bug, often aren't fixed. It all comes down to a business decision based on risk.

One more item should be added to this list that can often be the contributing reason for all of them:

  • Ineffective bug reporting. The tester didn't make a strong enough case that a particular bug should be fixed. As a result, the bug was misunderstood as not being a bug, was deemed not important enough to delay the product, was thought to be too risky to fix, or was just plain considered to be not worth fixing.

As in the case with Chicken Little, running around screaming that the sky is falling is usually not an effective approach for communicating a problem (unless, of course, the sky really is falling and it's obvious that it is). Most bugs that you find won't be as dramatic as this. They will require you to clearly and succinctly communicate your findings to the team making the fix/no-fix judgment so that they have all the information they need to decide what to do.

NOTE

Because of all the different software development models and possible team dynamics, it's impossible to tell you exactly how the fix/no-fix decision-making process will work for your team or project. In many cases, the decision lies solely with the project manager, in others it's with the programmer, and in others, it's left to a committee.

What is universal, though, is that some person or group of people will review the bugs you report and determine whether they will be fixed. The information you provide that describes the bug is used to make that decision.


You don't need to be a lawyer or an ex-debate team captain to know how to persuade everyone that your bugs need to be fixed. Common sense and basic communication skills will take you a long way. Later in this chapter you'll learn about the different systems for bug logging and tracking, but for now, consider these fundamental principles for reporting a bug:

  • Report bugs as soon as possible. This has been discussed many times before, but it can't be emphasized enough. The earlier you find a bug, the more time that remains in the schedule to get it fixed. Suppose that you find an embarrassing misspelling in a Help file a few months before the software is released. That bug has a very high likelihood of being fixed. If you find the same bug a few hours before the release, odds are it won't be fixed. Figure 19.1 shows this relationship between time and bug fixing on a graph.

    Figure 19.1. The later a bug is found, the less likely it is to be fixed, especially if it's a very minor bug.


    This may seem strangethe bug is still the same bug whether you find it today or three months from now. Ideally, it shouldn't matter when it's found, just what the bug is. In reality, however, the risks of fixing that bug go up over time and increasingly weigh on the decision-making process.

  • Effectively describe the bugs. Suppose that you were a programmer and received the following bug report from a tester: "Whenever I type a bunch of random characters in the login box, the software starts to do weird stuff." How would you even begin to fix this bug without knowing what the random characters were, how big a bunch is, and what kind of weird stuff was happening?

    AN EFFECTIVE BUG DESCRIPTION

    Effective bug descriptions would be as follows:

    • Minimal. It explains just the facts and the details necessary to demonstrate and describe the bug. Stating "a bunch of random characters" isn't minimal. Give an exact sequence of steps that shows the problem. If more than one set of inputs or actions causes the bug, cite a couple of examples, especially if they show a pattern or a clue that might help the programmer find the cause. Be short and to the point.

    • Singular. There should be only one bug per report. This sounds obvious, but it's sometimes difficult to differentiate between similar looking bugs and in the rush to get a product out it may seem easier to just lump them together. The problem with reporting more than one bug in a single report is that it's usually only the first bug that gets attention and is fixedthe others become forgotten or ignored. It's also impossible to individually track multiple bugs that are listed on the same report (more on this later).

      It's easy to say that bugs should be reported individually and not grouped together, but it's not always straightforward to do. Consider this bug report: "The following words are misspelled on 15 different pages in the online help file: …." That, obviously should be reported as 15 separate bugs. But what about, "The login dialog won't accept passwords or login IDs with uppercase characters"? Is that one bug or two? From the user perspective, it looks like two, one against the passwords and another against the login IDs. But, at the code level, it may be just one where the programmer didn't handle uppercase characters correctly.

      A quick tip: When in doubt, enter individual bugs. You're looking for symptoms, not causes. Several bugs may turn out to have the same cause, but you can't know that until the bug is fixed. It's better to err on the side of individual reports than delay or, worse, forget about fixing a bug because it was lumped in with others.

    • Obvious and general. A bug described with numerous complex, convoluted steps that shows a very specific instance of a bug is less likely to get fixed than one described with easily performed steps that shows the bug to be very general and readily seen by a user.

      Reporting bugs that your test tools or automation finds is a good example of this. Your automation may have run for six hours before finding a bug. A project manager deciding on a bug would be hesitant to fix one that takes six hours of constant keyboard pounding to occur. If you spend some time analyzing the results of your tool, however, you could find that it doesn't take six hoursit just takes 10 common and likely keystrokes. This process is known as isolating a bug. Your automation just happened to stumble upon those keystrokes while it ran. If you want this bug to get serious attention, your bug report should list those 10 magic keystrokes, not the thousands that the automation ran to get there.

    • Reproducible. To be taken seriously, a bug report must show the bug to be reproduciblefollowing a predefined set of steps will cause the software to achieve the same state and the bug to occur again. One of the more difficult, but fun, areas of software testing is trying to isolate and reproduce what seems like random software behavioroccasional crashes, chance data corruption, and so on. Later in this chapter you'll learn a few techniques for doing this. Once you reproduce the bug to its obvious and general steps, you can report it.


  • Be nonjudgmental in reporting bugs. It's easy for testers and programmers to form an adversarial relationship. Reread Chapter 3 if you forget why. Bug reports can be viewed by the programmers and others on the development team as the tester's "report card" on their work so they need to be nonjudgmental, nonpersonal, and noninflammatory. A bug report that says, "Your code for controlling the printer is terrible, it just plain doesn't work. I can't believe that you even checked it in for testing," would be out of line. Bug reports should be written against the product, not the person, and state only the facts. No gloating, no grandstanding, no personalizing, no egos, no accusations. Tact and diplomacy are what matters.

  • Follow up on your bug reports. One thing worse than not finding an important bug is finding a bug, reporting it, and then forgetting about it or losing track of it. You've learned that testing software is hard work, so don't let the results of your labor, the bugs you find, become neglected. From the moment you find a bug, it's your responsibility to make sure that it's reported properly and given the attention that it needs to be addressed. A good tester finds and logs lots of bugs. A great tester finds and logs lots of bugs but also continues to monitor them through the process of getting them fixed. You'll learn more about this later in the chapter.

These principlesreport bugs as soon as possible, effectively describe them, be nonjudgmental in reporting them, and follow up on themshould be common sense. You could apply these rules to almost any communications task. It's sometimes difficult, though, in the rush to create a product to remember to apply them to your testing. However, if you want to be effective at reporting your bugs and getting them fixed, these are fundamental rules to follow.



    Software Testing
    Lessons Learned in Software Testing
    ISBN: 0471081124
    EAN: 2147483647
    Year: 2005
    Pages: 233

    flylib.com © 2008-2017.
    If you may any questions please contact us: flylib@qtcs.net