Section 1.1. Agile and Plan-driven Methodologies

1.1. Agile and Plan-driven Methodologies

The most widely used software development methodology is the waterfall model that was originally espoused by Winston Royce[10] in 1970. His paper starts by describing a single-pass development cycle that evolved^[1] into the waterfall model. Although software engineering practices have long been based on the single-pass development life cycle he specified, Royce's paper said that it was "risky and invites failure" for projects with any innovation. He pointed out that you aren't sure that anything will work until very late in the project. At that point, changes in the design could cause up to 100% rework.^[2]

^[1] The terminology for the analysis stage has changed, but the philosophy is the same as Royce's original diagram.

^[2] Royce went on to propose a two-pass cycle with a number of other interesting innovations to reduce the risk. His insights are still relevant and worth reading.

Royce's solution to this fail-late methodology was to propose that we make two passes through the waterfall: the first pass focuses on the riskiest parts of the system, which results in essentially a (possibly throwaway) working prototype, while the second pass through the cycle actually results in the product. Royce's strategy was extended by Boehm's Spiral Model[4] that advocates prototypes, replanning, and risk analysis at every phase of the project. However, these methods still advocate specifying the entire system during the requirements analysis and design phases of development. The philosophy behind these methods is that if we plan carefully enough, we will be able to build the entire product without any rework after we enter the coding phase of development. Therefore, these methods are known as plan-driven methods.

In reality, in organizations that use plan-driven methods, most projects use repeated passes through the waterfall as they produce subsequent releases of the project. In doing so, they have acknowledged two things. First, today's systems are too complex to build in a single pass. They have complex interactions with the world and encompass ever-changing requirements, and these characteristics prohibit thorough up-front specifications. Second, market pressures require a company to ship product in order to survive. Developing the complete product defers income for too long. Instead, a critical subset of the required functionality is selected and put into Release 1.0. The team develops that release before time is allocated to plan the functionality of subsequent releases. By doing this, we have opened the door to rework because portions of the system may have to change significantly in subsequent releases, but this choice seems inevitable.

Agile methodologies have taken this iterative approach to development to its extreme. Instead of working on an entire release, they develop the system in small, fixed-length intervals called time-boxed iterations that are focused on delivering customer-visible functionality. In fact, it has been stated that these iterations are the "universal starting point for all agile software development approaches."[9] Although not everyone agrees on the optimal length of an iteration (recommendations range from one week to three months), the consistent factor is that within an organization, these iterations are of the same length every time.

It is very important that we notice the difference between an iteration and a release. Iterations are focused on delivering customer-visible functionality, and the goal is to show that functionality to the customer at the end of every iteration. Agile teams will produce workable functionality in every iteration.^[3] However, an iteration does not have to be a "release." Although some teams have customers that really want access to every iteration, most customers do not want that rate of change to be delivered to them. In these cases, the results of a number of iterations become a release.

^[3] Many teams take this to the extreme of ensuring that the product is always ready to release (called continuous integration). This ensures that they are continuing to make progress. Because nothing is "complete" until they see it work, they are sure that their progress is visible.