Risk Management

---

Pop quiz: What’s the biggest threat to the chance of your application shipping on time and within budget? If your answer is that you don’t know, then how in the world do you know whether it’s worth spending time and effort fixing that bug you just found? If the entire application is doomed because Widgets Inc. isn’t going to ship the “dripping wax”—style user interface components that your users demand on time, what’s the point of worrying about the calculation error in the fifty-sixth decimal place? In the memorable words of Rogers Morton, you might as well be rearranging the furniture on the deck of the Titanic. That’s why I’m opening a chapter on bug tracking with a discussion of risk management.

You’re probably at least vaguely aware that software development is a risky business, but you might not have thought about those risks in a structured fashion. To give you a framework for thinking about risks, I’ll discuss risk assessment and risk control, as well as tools for controlling risk in the overall project, and then specifically in your bug-tracking and -squashing efforts.

Risk Assessment

The first step in risk management is simply to understand the risks to your project. The easiest way to do this is simply to brainstorm a list of things that might go badly wrong. These need not be risks that will derail the project entirely. You should also include ones that could have a substantial effect on cost, schedule, or quality (because those factors are interrelated, most risks will have an impact on all three). To come up with such a list, you first need to define the desired outcome of your development project. For Download Tracker, I’d like to have a finished version of the application available on the book’s website by the time that you buy a copy of the book. Here are a few of the things that could torpedo that plan:

• I discover a requirement that I can’t figure out how to implement.

• I have so much consulting work that I don’t have time to finish the code.

• I am unable to register the website at the URL I want to use.

• I can’t arrange a source code control system license.

• I can’t finish all the planned features in the time allowed.

• I lose the program’s source code due to a hardware failure.

• The book schedule slips far enough to conflict with other planned projects.

• My code quality is too low to share without embarrassment.

Because this list is the product of brainstorming, it isn’t in any particular order, and not all of the risks are serious. Note also that the risks will vary from project to project. For example, while I’m working on this book, I’m also helping commercialize some vertical market applications. For those applications, one very real risk is that someone else might put a similar application on the market before my team can finish its work. For Download Tracker, though, that risk doesn’t bother me; because I’m not particularly trying to commercialize this application, it doesn’t matter to me whether someone else beats me to market.

Note

Keep in mind that there are some risks you can’t foresee. A bus could hit your lead developer, or an asteroid could land on your office. Risk management can reduce risk and make you better able to cope with risks both expected and unexpected, but it can’t eliminate the risks entirely.

Having a list of risks is better than having no list at all, but it still doesn’t answer the question as to which is the most important risk. Ordering the list of risks is the next step in risk assessment. To do so, you need to come up with a numeric measure of each risk. Normally this is done by assigning a probability and a cost to each risk; the expected impact of the risk is then the probability of the risk occurring multiplied by the cost if it does occur. Of course, the probability must lie somewhere between zero (the risk will never come to pass and can safely be ignored) and one (the risk will certainly happen). Often it’s convenient to express costs in terms of the project’s schedule, but you can just as well use monetary costs if you can come up with them.

A common objection at this point is that it’s impossible to quantify either the probability or the cost. Actually, it’s quite possible to assign numbers to both; the question is whether the numbers will be accurate. The best answer I can give to that objection is that the numbers will be accurate enough to work from. Once you’ve got some experience developing, you can also consult your gut feelings as to the relative importance of risks; if your gut disagrees with the costs you’ve assigned, it’s time to review your work. Table 9.1 shows my assessment of the risks for this project.

Table 9.1: Risk Assessment for Download Tracker

Risk	Probability	Cost in Weeks	Impact
Too much other work	0.25	12	3.0
Low quality	0.2	12	2.4
Too many features	0.50	4	2.0
Schedule slip	0.1	12	1.2
Can’t get SCC license	0.3	4	1.2
Hardware disaster	0.05	20	1.0
Impossible requirement	0.1	8	0.8
Can’t get URL	0.05	1	0.05

In this case, I’ve chosen as my measure of cost the number of weeks that it would take me to work around the problem if it crops up. Having gone through this exercise, I can now order the risks by priority, from the highest-impact risk to the lowest-impact one.

RULE

Focus your attention by developing a risk assessment for every project you work on.

Risk Control

Unless you want to imitate a cartoon ostrich by sticking your head in the sand, assessing risks is only half of risk management. The other half is controlling the risks. Although you may feel that the risks to your application are completely out of your control, that’s very seldom the case (unless you’ve chosen to include low-probability risks like “asteroid strike” on your list). Normally, even if you can’t completely control a risk, you can take actions to lower its probability, or its cost, or both.

After you’ve identified and prioritized your list of risks, you can formalize the risk control process by creating a written risk management plan. Table 9.2 shows my risk management plan for Download Tracker.

Table 9.2: Risk Management for Download Tracker

Risk	Management Steps
So much consulting work that I don’t have time to finish the code	Avoid work with deadlines until safely after the book is ready to print. Reserve time in schedule to work on book code.
Code quality too low to share without embarrassment	Implement unit tests and track bugs.
Too many features to finish in time allowed	Rewrite Chapter 1 in galleys to remove features if necessary.
Book schedule slips far enough to conflict with other planned projects	Get chapters in by originally scheduled dates. Use vacation time if necessary to get back on track.
Can’t arrange source code control system license	Contact vendor well before the end of the project, so as to have time to switch vendors if necessary.
Hardware failure destroys source code	Keep source code on mirrored drives and in source code control repository on a separate computer.
A requirement that I can’t figure out how to implement	Keep in touch with C# experts; don’t be shy about asking for help.
Unable to register website at the URL I want to use	Register URL well in advance of project end; brainstorm alternatives.

With a risk management plan in hand, you can avoid finding yourself reacting in emergency mode if a risk does materialize. Having decided in advance how you’ll react to a risk means you can simply execute your plan. Also note that many of the management steps I’ve listed are aimed at making sure that the risk never materializes, instead of reacting to it if it happens.

Maintaining the Top Five Risks List

On a project involving several developers, or one that’s going to occupy you for an extended period of time, you may find it useful to maintain a formal Top Five Risks List. As the name implies, this is a list of the most important risks to the project. It’s usually helpful to include some other information, as shown in Table 9.3.

Table 9.3: Top Five Risks List for Download Tracker

Rank This Week	Rank Last Week	Weeks on List	Risk	Management Steps
1	2	11	So much consulting work that I don’t have time to finish the code	Avoid work with deadlines until safely after the book is in galleys. Reserve time in schedule to work on book code.
2	3	9	Code quality too low to share without embarrassment	Implement unit tests and track bugs.
3	1	11	Too many features to finish in time allowed	Rewrite Chapter 1 in galleys to remove features if necessary.
4	-	1	Book schedule slips far enough to conflict with other planned projects	Get chapters in by originally scheduled dates. Use vacation time if necessary to get back on track.
5	4	3	Can’t arrange source code control system license	Contact vendor well before the end of the project, so as to have time to switch vendors if necessary.

The goal of the Top Five Risks List is to give you a single place where you can immediately and easily review the most serious threats to your project, and you can tell which ones are growing or shrinking in importance. If you’re working with a project team, this is information that you should review at weekly status meetings. For larger projects, you may want to expand the list to the top 10 or 15 risks, and review it at the management level on a daily basis.

Bug Triage

But what’s the relationship between risk management and bugs in your software? There are two connections. First, if you’re in imminent danger of being seriously affected by one of the risks on your list, you probably have more important things to do than fixing bugs. Second, the mindset of risk management is the same mindset that you need to have when you’re performing bug triage. Just as not all risks are created equal, not all bugs are created equal.

Triage is a term borrowed from emergency medicine, where it refers to the prioritization of patients for treatment based on the extent of their injuries and chances of survival. In the context of software, it refers to prioritizing bugs based on their seriousness and deciding what to do about each one.

If you learn computer programming in a classroom environment, you may not be exposed to the concept of triage at all. Bugs in your homework assignments are considered mistakes, and you are penalized for each one. Your goal is to create perfectly bug-free programs.

In the real world, things are very different. Every once in a while there’s a news story reporting breathlessly that, for example, Microsoft Windows 2000 contains 63,000 unfixed bugs. That’s not because the developers at Microsoft are incompetent fools. Rather, it’s the reverse; it’s because the developers at Microsoft are very good at what they do, and they know that almost no substantial software project is going to go out the door without bugs. Rather than focus on a zero-bug goal, they focus on having zero unfixed bugs that require fixing—quite a different matter. A bug might be placed on the “don’t fix” list because it’s so minor that no user will ever notice it, or because it doesn’t prevent the application from functioning, or because you know the feature will be completely replaced in the next version. It’s up to you to decide where the quality bar is for your own applications.

Note

In some areas, such as developing software for the manned space program or for use in medical devices, the goal is still zero bugs. The rough-and-ready techniques I’ve been describing in this book don’t apply to projects with such a stringent goal.

When you’re triaging bugs, you need to focus on resolving the bugs rather than fixing them. You might resolve a particular bug in one of several ways. Bug-tracking products vary somewhat in terminology, but this is a typical set of alternative resolutions:

• By design At times, testers will report as a bug something that works exactly the way that you designed it. For example, you might design an inventory application to not allow withdrawals of more material than is available, and to change the requested amount to the maximum available amount in such cases. A tester could report the change of the number he or she entered to another number as a bug in your application. Be careful when using this resolution, though; if something works as you designed but not as end users expect, it might be time to change your design. Alternatively, you might make a note to explicitly spell out this bit of design in the application’s documentation.

• Duplicate Particularly if you have many beta testers, you might find the same bug being reported two, three, or more times. When you spot this situation, resolve the extras as duplicates. Good bug-tracking software will let you link the duplicate bugs to the one that you choose to leave open, so that the ultimate resolution of the issue can be tracked.

• Postponed Sometimes you’ll agree that something is a bug but will not want to fix it right away. For example, you might be planning to add support for Unicode in version 2.0 of the application, in which case you should simply postpone all Unicode-related bugs until you’re ready to work on that version. Resource constraints can also force you to postpone bugs; in this case, you need to find the bugs with the least impact on end users and postpone those. A bug that shows up only when your application is run under the Turkish version of Windows 98 with a particular obsolete CD-ROM drive installed, for example, is a good candidate to be postponed, because such a bug may never affect a real user of the software.

• Not reproducible Testers are not infallible, and sometimes you’ll get bugs that you simply can’t reproduce on your system. Resolve such bugs as not reproducible. Good testers will treat this resolution as a request for more information rather than as an insult, and you may well get the same bug back with additional explanation that makes it possible for you to reproduce the bug. When you’re thinking of resolving a bug as not reproducible, try to figure out how the tester could have seen such a thing happen. Is it something that you fixed in a more recent build? Is it an artifact of a particular data file that you don’t have? “It works on my machine” is a very unsatisfying explanation for resolving a bug as not reproducible, because, as every tester knows, you’re not going to ship your machine to the customer.

• Won’t fix Resolving a bug this way is the same as postponing it forever. You are acknowledging that the bug exists but at the same time claiming that it’s not worth the resources to fix it. Perhaps it’s a problem with a feature that you’ve decided to cut, or a case that you never expect an end user to see. Be cautious about resolving bugs this way; if you refuse to fix too many, testers may wonder why they’re even bothering to test your application.

• Reassigned Depending on your bug-tracking system, this may or may not be seen as a resolution, but it has the same effect: it gets the bug off your desk. You might reassign a bug to another developer if you determine the bug is actually in his or her code rather than yours. You might even reassign it to your subcontractors or suppliers by opening a bug against their code instead of yours.

• Fixed Sometimes, the best way to resolve a bug is just to fix it. In fact, for many projects this will be the most common resolution for bugs.

This is a good place to put in a few words about the “politics” of bugs. In many organizations, the bug-tracking workflow goes something like this:

1. A tester finds a bug and reports it.

2. The bug is assigned to a manager for initial triage.

3. The manager resolves the bug if possible or assigns it to a developer.

4. The developer resolves the bug (by fixing it or otherwise resolving it).

5. Resolved bugs are returned to a tester. Usually this is the same tester who reported the bug, but this isn’t always the case. Bugs reported by external beta testers, for example, may be reassigned to internal testers.

6. The tester either closes the bug or reopens it with additional information or comments, in which case the process starts over at step 2.

With several people involved, bug resolution can become a very political activity. I’ve seen bugs in large organizations ultimately involve dozens of people, as multiple bug reports get duplicated to a single underlying bug, which then wanders from developer to developer in search of someone to claim responsibility. I’ve seen bugs bounce back and forth from tester to developer a dozen times as they argue over whether “postponed” or “won’t fix” is a reasonable resolution (“But we have dozens of customers using that CD-ROM drive in Turkey!”). I’ve seen developers mark a dozen bugs as “won’t fix” without even reading them, as a passive-aggressive protest against having too much work. I’ve seen testers spend their time on the most stable part of a product, because at least the developer of that portion doesn’t ignore their bug reports.

At the end of the day, a bug-tracking system will work only if all concerned keep in mind that their ultimate goal is to ship a high-quality project. If you’re a manager, you need to keep an eye on the bug-tracking system for these political games, and figure out how to rearrange resources or smooth relations between strong egos to keep them from getting out of hand.

---