Section 6.6. Diagnosing Software Requirements Problems

6.6. Diagnosing Software Requirements Problems

Project teams that have problems specifying the software requirements often find that their projects suffer from a few typical problems. Strangely, many programmers and project managers do not realize that they are problems at all. There is a common and mistaken belief that they are just characteristics of how software is typically developed. But when software teams adopt good requirements engineering practices, they find that these problems are greatly reduced, and that their projects go much more smoothly.

6.6.1. Iteration Abuse

Iteration, or the repetition of a particular task or activity with the goal of improving its main work product, is a practice employed by many programmers. There are several popular development methodologies (Extreme Programming and Rational Unified Process, for examplesee Chapter 12) that rely on iterative development. These methodologies use iteration effectively, by planning and executing a series of small steps that build on previous iterations to produce a series of intermediate builds; each successive build adds new functionality to a previous build. Each build is intended to be production quality, and is fully tested. Changes are controlled by having each new iteration concentrate on adding new functionality, rather than making many changes to existing behavior.

However, many software teams abuse iteration. They do not use successive iterations to add new functionality to a build that is already production quality. Instead, the iterations are meant to be a sort of guessing game. The first iteration is presented to users and stakeholders, who are expected to tell the programmers what's wrong with it. The programmers then go back, fix those things, and deliver a second build, which the users and stakeholders again correct. This is expected to continue until the software does everything that the users and stakeholders need it to do.

This sort of iteration makes intuitive sense to everyone involved. The programmers like it because they can dive right into the project and do what they do best: program. And it feels right to the users and stakeholders as well, because it's much less work for them than sitting down and really thinking about what they need out of the software. All they have to do is play with the demo versions and give their opinions.

Unfortunately, this is all too good to be true. The first iteration is easy to deliverexpectations are low, and people are excited to get their hands on a build so quickly and to give their input. The next iterations seem to be doing their job, and the software does look like it's starting to get better. But as the software gets more complex, each iteration takes longer and longer to do. There seem to be more bugs in each new build. The programmers find that they have to take more time to rip out things that they put in earlier. The code starts to degenerate into an unmaintainable mess because the programmers keep patching and repatching code that they should have built differently in the first place. The iterations finally end, but not with fully working software. Rather, the users and stakeholders get increasingly fed up with long delays and with buggy software that doesn't do what they need. They eventually settle on a build that is close enough to something that fulfills their needs; they can concoct workarounds for any remaining problems that keep them from getting their work done.

This sort of "iterative" development falls apart because software is difficult to change. After an object or function is written, it is generally used elsewhere in the code. The objects or functions that reference it, in turn, are called from other objects or functions, and so on. Any changes made to one function can ripple through the rest of the software, causing many unforeseen problems. Even a minor change in the basic behavior of the software usually requires that the programmers make extensive alterations to its design. As the changes pile up, they become increasingly awkward because the programmer needs to patch existing code without breaking it. Each additional design change requires another layer of patching, and most of those patches require some compromise between implementing new behavior and not breaking existing behavior. This is a messy and difficult way to code, and tends to introduce many defects and delays after a few iterations.

Had someone written down a description of the behavior of the software before the programmers started building it, many of these iterations could have been avoided. It's much easier to change the behavior of the software on paper than it is to change the code. By adopting a highly iterative software requirements engineering process, the project team can identify most of the changes before a single line of code is written. They may still have to make changes to the software, but those changes will be much smaller and therefore much less risky.

6.6.2. Scope Creep

Many software projects have been driven to failure by poorly controlled changes. Typically, one of the project team members, users, stakeholders, or decision-makers sees a change that he thinks ought to be implemented. The change seems "small," so programmers begin altering the software to include it. But the change turns out to be difficult to implement, and the entire project is thrown off track. This problem is usually referred to as scope creep , because the scope of the project slowly drifts over the course of many incremental changes.

Scope creep is so common that many project managers don't realize that it's a problem at all. They feel that refusing any change request would be considered "inflexible" by others in the organization, and that any project manager or team member who refuses to make a change is not being a team player. But in a team that does not control changes, progress on the project starts to slow as the changes pile up. At every status meeting, the team reports that they are 90% donebut it seems that they have been 90% done for the last 3 months, and they don't seem to be making any headway on that last 10%. Terms like "quagmire" and "death march" get tossed around. The project plan is a joke, a product of wishful thinking that was abandoned months ago. When projects finally do deliver, it feels less like the team successfully met their goal than they finally crawled out from under a suffocating mess.

In a project like that, the programmers usually understand the problem. They can point to portions of the code that had to be rewritten several times in order to accommodate changes. The team members know that had they been asked to do the correct project the first time, they could have built it much more quickly and easily, but that it took many changes to finally arrive at the product they have now. And nobody even wants to think about maintenancethe code is a mess, because it had to be patched and hacked so many times.

To many people, this is all just a fact of life: it's the reality of how software is developed. They feel that changes happen, and that there's really nothing that can be done about them. But it doesn't have to be that way. As with iteration abuse, if more of these changes can be identified on paper during requirements elicitation and reviews, by the time the programming team starts building the software they are much closer to the goal and will have to deal with far fewer changes.

However, there will still be requests for changes, and those changes should be controlled using a change control process. One reason changes are easy to sneak through is because it's often easy to describe a difficult change using simple language. It's human nature to be accommodating, so the programmers are willing to incorporate that change because it seems like it would be easy to implement. The change control process requires that a description of the change be written down, and that engineering team members take the time to estimate how much effort the change would require. This forces the project's decision-makers to decide if the change is worth the effort. If it is, the schedule is updated. But many changes that seem "simple" turn out to be much more involved than the person asking for them realizes, and many changes turn out not to be worth the extra effort. If the changes are starting to make it harder to develop the software, this will cause each additional change to have a greater impact, which in turn will make it less likely to be approved. Even if the changes are all approved, the team does not have to fight against an unrealistic deadline to get them doneinstead, the project plan is updated, and it remains realistic.