Risk Targeting

"Yes, it's called risk targeting," replied Roscoe. "Risk targeting is about understanding where the threats are on our project. For example, suppose we are using some new technology that has never been used in production before. That has the potential to take us on a vector in a very wrong direction, because if the technology is inappropriate, we might have to spend a long time working with it, only to abandon it and replace it with another. So let's look at how this might play out in an early iteration."

Roscoe began. "Having identified 'using this new technology' as a risk, we need to rank order it with all the other perceived risks on the project. Suppose we identify it as the number one risk on the list. What do we do then?"

"As a software guy, I think I can answer that," I offered. "The first step is to remove or reduce the risk of this new technology on the next iteration. But, according to your short-vector principle, we want to limit the scope of each iteration. So we need to design a part of the system that will 'wring out' this new technology as quickly and thoroughly as possible. We want to make a decision, and we want to be clear and quick about it. Designing the iteration is a little like designing a scientific experiment: What do we want to determine? How will we determine it? And how can we do it with a minimum of interfering effects?"

"Couldn't have said it better myself," Roscoe chimed in. "So how would you answer the folks who say, 'Arggh, that's not iterative development; that's just prototyping!"?

"Sure, it sounds a bit like prototyping," I continued, "but there is an important distinction: We need to test the technology in the context of how it will eventually work in the full product, so we need to consider scalability and performance, too. 'Toy' experiments at this stage are dangerous, because they can lull us into a false sense of security and make us complacent. A well-designed iteration leads us to build a piece of the system that either validates or invalidates the technology. At the end of the iteration, we need to be able to assess our working software and answer the simple question, 'Do we continue down this path, or not?' If 'yes,' then what we built in that iteration remains a part of the product, unlike a prototype, which gets thrown away.

"And," I summed up, "the bias has to be 'What experiment will show that the new technology is inappropriate?' rather than 'What experiment will make us feel better about the new technology?' Sometimes in our desire to go down a certain pathin this case, using the new technologywe design iterations or experiments to make us feel more comfortable. This might have the unfortunate consequence of making us feel much more uncomfortable later on, when we discover that we have been engaging in wishful thinking."

"Yeah," said Roscoe. "A good recipe is to be 'tough' early to avoid big trouble later.^[5] Using risk targeting gives us a focus for each iteration. We might target more than one risk per iteration, but we should always be addressing the risks in priority order. Remember, the earlier we start working on the hard problems, the more time we have left to solve them," he continued. "This leads me to a huge pitfall I've seen over and over again on software development projects."

^[5] John Walker reminds us of the importance of killing off really bad ideas early. Figure out a way to "falsify" an approach, and then see if you can. Sometimes a literature search can be revealing. As F.H. Westheimer so cogently put it, "A month in the laboratory can often save an hour in the library."

I could hardly wait.