7.2 Waterfall Lifecycle Management

Walker Royce (Royce 1998) states that the "waterfall" lifecycle is less the twenty-first century's "conventional" software lifecycle and more the "benchmark" for newer lifecycle models. By benchmark, we mean that waterfall is something for us to compare new lifecycle models back to. You might think Walker wants to maintain the importance of the waterfall lifecycle because his father, Winston Royce, contributed significantly to it in the 1970s. This might be partially the case, but we feel Walker is right on target with his observation. But first, let's examine what the waterfall lifecycle represents.

Waterfalls ”Beautiful in Nature, Lousy in Software Development

Table 7.1. Advantages of Waterfall

The detailed scope of the project is nailed down early in the project allowing the remaining tasks to be based on a stable set of requirements.

The project team can create "economies of scale" throughout the lifecycle, producing analysis models all at once, test executions all at once, etc.

Project managers can create clear deadlines for each activity; they know when each needs to be done in order to meet the end deadline.

There is no expensive rework ; going back to previous phases is not allowed.

Staffing is predictable; more business analysts are required at the beginning; more programmers and testers in the middle; and more deployment specialists, technical writers, and trainers near the end.

Industry baselines can be created, which give standard breakdowns of how long each activity should take. For instance, requirements and analysis should take no longer than, say, 14% of the total lifecycle. If they take any longer, then it is unlikely the team will be able to meet the deadline at the end.

When the term waterfall is used in software development, it refers to a way of developing software that proceeds through one activity at a time, completes it fully, then moves on to the next . This has a strong set of advantages, which caused a mass adoption in the 1970s, 1980s, and 1990s.

After using waterfall for many years , we began to run up against a strong set of disadvantages, which tends to outweigh the positives. These disadvantages stemmed from the fact that the waterfall model did not satisfy what businesspeople expected from projects.

7.2.1 Nell and the Coffee Shop

Put yourself in the position of a businessperson for a minute. Let's say you are the owner of a small coffee shop. You are an especially sharp coffee shop owner. You know that investments in technology can pay off in increased business if you pick the right ones and use them intelligently and long enough to recover your initial investment.

One day, a young lady named Nell walks into your coffee shop and tells you she can provide a software system that will increase your business during slow times by 50%. She certainly has your attention. She explains the concept of the system to you and you realize that it could possibly work. Now the question is, how much? The price she quotes is reasonable, but you push her off. First, you need to crunch some numbers to determine if the investment in this system will pay back in a reasonable time, let's say within one year. The numbers come out looking good, full payback in nine months. Next, as a businessperson, you try to negotiate a lower price with Nell for the system. After all, payback in seven months is better than payback in eight, right? Surprisingly to you, Nell agrees, but does not negotiate a compensatory reduction in the system to accommodate the reduction in price. She just gets an annoyed, worried look on her face and accepts the deal. Oh well, it is always nice to get something for nothing, you think.

The Coffee Shop

Nell says the system will be ready in one month. Perfect, you say, because in one month there will be a lot of tourists coming in for the annual festival. Increasing business during slow times with all those tourists around will mean a lot more money in a short time.

First, she interviews you quite extensively to understand how your business runs and how the system will need to work to give you the gains you need. You're impressed. She is skillful at getting to the core business issues that drive profitability and that cause the slow times. Then she explains that she will be gone for the rest of the month, designing, developing, and testing the software system to be ready by the deadline. So far, so good.

Nell is gone for only about three days before you realize you forgot to mention an important fact about the coffee shop that may help her. You call her and explain this fact, but you're surprised at her response. She says it is too late to introduce changes; only the information discussed in those interviews was relevant. Everything else will have to wait until "version two." You are not sure about the impact of this new fact, but in order to keep a good relationship with Nell, you back off. She talks about " changecontrolmanagement " or some such thing. Okay, okay.

After a week and a half, you are talking to a customer and she tells you about the coffee shop two streets away that has begun to offer ice cream as well as coffee, in anticipation of the tourists. The combination of coffee and ice cream is the newest wrinkle in coffee service; everyone's doing it. You think about it, and you realize that you could get the freezer in, set up the supplier relationships, and hire new staff in time to quickly add ice cream to your offerings. Oh, almost forgot about Nell! You give Nell a call to tell her about the changed situation, quite proud of yourself for thinking to include her. This time her response is even more severe. She says it is impossible to incorporate the ice cream part of your business into the system and still meet the one-month deadline. She gives you an estimate of two months to do the complete system for coffee and ice cream. Two months? The tourists will be long gone by then. Isn't there some other way?

Nell, ever the vigilant project manager, says that she can't wring blood from a stone, something has to give. You can't have all these features, by this deadline, for this price.

At first you're upset, but then you go over her words one more time. All these features, by this deadline, for this price. What a relief! You realize there are many features you could easily do without; you could use them as trade-offs to get the ice cream features incorporated.

Sorry, Nell informs you. She's already done the analysis and design and has begun coding. The feature set was locked in when we had those interviews; we can't go back and change them without adding to time and cost. Darn! You now realize you're either not going to have the software in time for the festival or you're going to have to live without the ice cream features, which are critical if you add that to your business. You begin to wish you had never started doing business with Nell.

7.2.2 Disadvantages of Waterfall

Waterfall lifecycles have not worked well in the creation of business software. They have contributed to our pitiful success ratio and much frustration and overtime for technology professionals. The reality of today's business is that things change constantly. In the 1970s business changed, too, but at a slower pace. Corporations could take years to create new products, months to make decisions, and weeks to provide customer service. In the twenty-first century, those time windows have shrunk dramatically. Forget about e-Business as the reason, this is an issue of global competitiveness . Every industry has so many more new players from around the world that customers (businesses and consumers) can expect and demand changes to happen quickly or instantly. Those businesses that can offer the best quality, the lowest price, and the greatest ability to adapt to customer needs consistently win. The business software must accommodate those needs as well. If internal software departments or companies cannot keep up with the rate of change in business, the business world will find some way around them. Nell's insistence on an unchanging set of features for one month might not have seemed unreasonable to Nell, but to the coffee shop owner that was an unworkable assumption. Yet, even he could not tell Nell what changes he expected to make during that month in the interviews with her at the beginning. He couldn't even remember to tell her everything he did know; no one's perfect.

In process terms, we call the waterfall lifecycle "brittle." If nothing changes and no one forgets anything during the lifecycle, waterfall works well. Unfortunately, this rarely, if ever happens in the creation of business software. Add the fact that the average business software project takes much longer than one month, probably closer to 6 “12 months, and you can see the scope of the problem.

This brittleness to changes is interesting. Whenever you speak to someone in the software field, the topic of change comes up often. The change we are mainly concerned about is the rate of change of technology. But the rate of change in business causes us many more problems, especially when we utilize the waterfall lifecycle. Change in business requirements we see mainly as a people problem. If businesspeople were more reasonable, less demanding, knew what they wanted, and could make up their minds, that problem would go away. But masking it as a people problem is not the answer.

There are other issues with waterfall that Nell and the coffee shop owner did not even encounter. Waterfall projects often have a problem with risk. By executing activity-by-activity, we push the risk of the system actually working to specification far into the future, devilishly close to the end of the project. These questions will not be answered until late in the lifecycle:

• Will the businesspeople accept the system?

• Will they enjoy using it?

• Have we represented it accurately with our artifacts (analysis models, user interface storyboards, prototypes , etc.)?

• Will the pieces of technology work together as we've planned?

• Will the team perform the tasks quickly enough to meet the deadline?

• Will the quality be high enough to avoid excessive problems in testing or production?

• Will there be business changes that occur in this lifecycle that make our delivery on-time and on-schedule impossible?

The longer the lifecycle (3/6/9/12/18/24/36 months) the more risk there is.

Table 7.2. Disadvantages of Waterfall

Changes that occur after an activity is complete disrupt the lifecycle. For instance, changes to requirements that are known only after the requirements activity is complete will either (a) increase the cost/effort, (b) lengthen the schedule, or (c) cause user dissatisfaction if not implemented.

High rates of change are the norm, whether in Internet software companies, automotive manufacturers, energy producers , state government, or banks. No area of business is untouched by the "change bug."

The idea of gathering "all the requirements" at an early stage of the lifecycle is made impractical by human nature ( forgetting , missing the point) and business changes.

Businesspeople see trade-offs in features as much fair game as trade-offs in schedule, cost, and quality. Waterfall lifecycle managers, however, can only handle schedule and cost, with quality usually being an underdiscussed topic, something that suffers when everything else is held constant. With waterfall, changes in features after the requirements activity cannot happen without cascading changes in schedule and/or cost.

What About Change Management?

You may be thinking that this desecration of waterfall lifecycles sounds like heresy. (But if we were writing a book that just confirmed all your existing beliefs, why read it?)

You may say to yourself, "I have mechanisms for dealing with change throughout the lifecycle; I even have a discipline called 'change management' that covers this need." True. However, we feel that the term change management is not representative of the true aim of this activity. Change management sounds like accepting that things will change constantly throughout the lifecycle and you have created a system to handle incorporation of those changes into the end product when they make sense. In every instance we've seen, the actual activity is more aptly called change discouragement . Keep changes from disrupting the lifecycle as much as possible. Lock them down. When they are absolutely necessary, use trade-offs in cost, schedule, or quality to offset the required rework for already completed activities as well as additional tasks in upcoming activities. Changes that are possible are those which cause the least disruption of the lifecycle, the least rework of completed activities. If we are right in using this new name , does it give the impression of a software organization committed to changing with the business?

In our software project management, we have the goal of "reducing disruptive change to a minimum." But the businesspeople are looking to us to "handle as much change as possible." Our views are seriously at odds.