Evaluating Quality

[ LiB ]

On the surface, saying you want a bug-free app is a no-brainer. Because many of the approaches proposed in this chapter have an associated cost, however, it's best if everyone understands the cost. That is, to judge whether an investment is worthwhile, you want to be sure you see the entire value.

Naturally, a known "show-stopper" bug that renders your app useless must be fixed or your app isn't really an app. If you look beyond such critical bugs, however, you'll see bugs of varying degrees. The question is what sorts of bugs can you avoid at what cost. You'll ultimately reach a point of diminishing returns, where the last bug might take months to crush. The key is to always assess bugs before attempting to address them.

Benefits to Client and Developer

Although both programmer and client should be on the same side, they tend to value quality for different reasons. A client might be able to associate a dollar figure with the downtime while an application (bugs) gets repaired. From the developer's perspective, time going back to fix bugs might delay another project or cost the developer money if the developer bid the first project at a fixed price. Finally, although the "pain and suffering" cost (due to stress) may be difficult to calculate, it's real nevertheless.

Ultimately, the idea that time is money is something both developer and client can agree on. Keep in mind also that with projects involving a large team, redoing work at one stage might ripple through and require others to redo their work, too. Fixing a particular bug might not seem so important if the cost is many people's time plus a delay in the overall schedule. Although, obviously, everyone wants quality, you have to consider the cost/benefit from each party's perspective. Also, because avoiding bugs costs money, everyone in a position to benefit should "pay." Even though the client is the only one who really pays, it's best if everyone understands and places a similar value on quality. As you'll see in the "Ensuring Quality" section later in this chapter, avoiding bugs costs money, too, so you need to make sure everyone is willing to pay.

Finding and Assessing Bugs

My personal opinion is that the person who created a bug in the first place (the programmer) is the person least able to find it later. However, you can develop some skills to become a better bug finder. The first step is self-criticism. For instance, you may need to walk through code you wrotecode that made sense when you wrote itbut you have to question every line, effectively calling yourself a liar. Walking through code is more likely to occur while fixing bugs ( covered later), but the concept is the same when looking for bugs. You want to find bugs, but that means you did something wrong in the first place.

Programmers often miss their own bugs because they almost tiptoe around the app. If nothing else, it's common to repeatedly test an application by following the same steps. If you're testing the third screen in an app, for example, you'll probably find the quickest way to navigate to that screen every time you test. However, a bug may pop up when you take a slightly different route to get to that screen. The reason clients inevitably discover bugs immediately is not their uncanny sensitivity but just because they play your app differently than you do. When it comes to testing, you really need to break out of the cycle you followed during most of the development.

Another issue that may be human naturebut makes no sense when hunting for bugsis how some programmers would rather find no bugs. In your dreams! First, accept the fact you're going to have bugs. I sometimes rationalize it by accepting a few bugs as inevitable, figuring I must have been working so swiftly when I created them. (As discussed in the "Ensuring Quality" section later in this chapter, there are ways to avoid making bugs in the first place, but there is no way to write bug-free code.)

You should try to limit your reaction to finding a bug to two responses: happiness and pride . When someone else finds a bug, be happy because you're that much closer to a quality app. When you find a bug, be proud that you were the one to find it. Suppose you're testing your app and you see something happen (a bug, that is). Don't ignore it and hope it never comes back. Document it! Try to remember the exact steps that caused it. Although it might be a big distraction to stop everything and go into bug documenting mode, at least make a note of it. Don't, however, just try to fix the bug when you find it. (As discussed in the next section, you need to take a deliberate approach to fixing bugs.)

Even if you feel like cringing when a client points out a bug, you really need to accept it graciously. Consider that someone else finding a bug is doing you a favor. It's certainly much better to find it early than later. And, besides, you need all the help you can get. Bug finders should be thanked, if not also rewarded.

Just finding a bug is only part of the identification process. The key to effectively fixing a bug is having a clear understanding of the bug. You need to understand the full scope of a bug, because you don't want to fix just part of the bug. Suppose, for instance, that a user describes the bug as "I can't enter my name in the Name field." If there are other related issuesfor instance, the user can't enter his number in the Number field eitheryou need to know about those other issues (so that you don't waste time fixing just part of the bug).

A full understanding of a bug also means that you will be able to reproduce it. After all, how can a programmer fix something she can't see? Documenting the exact steps to exhibit a bug is critical for several reasons. You want to make sure the programmer is seeing the same thing as the bug finder. Also, you'll often find that the steps that caused the bug may almost serve as instructions for fixing the problem. (Many times, as someone is explaining how he got a bug to occur, a light goes off in my head.) You also can use those same steps that caused a bug to test later whether the bug has been fixed.

You can categorize each bug you find. The following are typical categories:

• Causes a crash

• Incorrectly functioning

• Wrong data appears

• Data lost

• Usability issue

• Cosmetic issue

• Typo(s)

To give you an idea of what's possible, Figure 11.1 shows a fairly sophisticated bug-reporting tool used in a large project I worked on.

Figure 11.1. A web application itself, this bug-reporting tool made reporting bugs easy and consistent.

Of course, you can come up with appropriate categories for your own app. You also may want to include other notations, such as in which portion of the app each bug was found. You may find it helpful to work up a little table into which the bug finder can document these things. Depending on the size of your project, it may be worth creating an online data-entry app to streamline the bug-collecting process. Again, it doesn't have to be super fancy or formalyou just want to make sure bugs are found and that people's efforts in finding bugs are not wasted .

Now that you know a bit about finding bugs, the next step is to decide how to address them (which, perhaps surprisingly, isn't always to fix them).

Taking a Deliberate Approach

One of the biggest mistakes programmers make is fixing bugs as they find them. That is, a client calls up and provides a perfect explanation of the problem; the developer realizes the cause and goes and makes the fix. Although this may sound like the correct process, it's actually a bit dangerous. When a bug is found, you really ought to answer the following questions before attempting to fix it:

• How important is the bug? After answering the following questions you'll be able to associate a cost with the fix, and it may, in the end, not be worth fixing.

• How easy do you expect the fix to be? You can even do a little bit of offline research to see whether you can fix it. Of course, if you succeed, you can use that code when implementing the fix.

• What are the possible side effects? Fixing one thing may cause other bugs to crop up. Incidentally, this is a perfectly legitimate rationalization that you can use when a client suggests a fix. I'm not really suggesting you use it as an insincere scare tactic, but it is an important consideration nevertheless.

• How will you ensure the bug was fixed? It's best to have a way to measure success worked out ahead of time. At a minimum, document the steps to cause the bug so that you have steps to follow when testing the fixed version.

• When should you schedule the fix? Considering that you really ought to retest everything after making a fix (to see whether other bugs surfaced), it may be best to wait until you have a few bugs to fix before going in and fixing them. Consider, too, there's often a deployment step necessary that takes time (and introduces more possible bugs).

You may think from the preceding list that I'm living in a dream world and that your clients won't accept anything other than all bugs being fixed. I've been there, too. The truth is, you're doing your clients a favor if you take a well-structured approach to quality assurance. For really large projects, apply the concepts introduced in this chapter (as well as those from Chapter 10, "Production Techniques") and expand on them. You also can apply knowledge from traditional software development, too (and from any other sort of production process for that matter). Just remember: Haste makes waste.

Client Management

Because this discussion may imply that clients are easy to persuade, I want to share the little I know about client management as applied to debugging. I think it really comes down to education. It's not so much to avoid conflictalthough that's nice, toobut the goal is really just an efficient production. Everyone wants all the bugs removed, but I think most would also agree that it's better to leave in a few minor bugs if it means you can fix all critical bugs (as opposed to fixing every bug as you find it and not getting to them all).

Note

Avoid " Groupthink "

One downside to being too persuasive is the concept of groupthink. You don't want to fall into this cycle where everyone helping each other rationalizes his or her way out of fixing anything. It may sound funny , but groupthink is a real phenomenon . (For more about groupthink, check out the writings of Irving L. Janis and others on the topic of organizational behavior.)

After answering the questions found in the "Taking a Deliberate Approach" section earlier in this chapter, you have a number of ways to approach problems. There's often more than one way to fix a bug. If a button is taking the user to the wrong screen under certain conditions, for example, you could just disable or hide the button when that condition occurs. You might immediately think the "fix" is to make the button take you to the right screenbut turning off the button might be a fix. One time when I rented a car, I pointed out a bug in the computer the rental company had set up to print door-to-door directions for any destination. The service person walked up and unplugged the computer, saying "that fixed it." I couldn't dispute that the problem was gone.

I think the best way to manage bug fixes is to think about schedule. Obviously, I don't suggest waiting to fix a critical bug in "version 2" of your application. However, addressing bugs in a logical order is important. Also, you can consider bugs that affect the overall function of an app as critical (as compared to bugs related to features). For enhancement requests , however, it really makes sense to say, "Add that to the list of features for the next revision." Not only does this help you stay focused on the important bugs, it also helps the client to scope out future variations of your app. The concept of "feature creep" is where you keep adding every feature as requested . Feature creep often causes missed deadlines and may result in unusable applications.

[ LiB ]