Section 1.2. How Time-boxed Iterations Help Us Handle Change

1.2. How Time-boxed Iterations Help Us Handle Change

In plan-driven methods, the scope of a release defines the length of time needed to produce the required functionality. In other words, the releases vary in length and in the scope of the functionality to be delivered. With time-boxed iterations, the iteration length is always the same, which defines the scope of functionality that can be developed in each iteration.^[4] Essentially, agile methodologies reduce the project's development risks by allowing the time available in the iteration to define the scope instead of letting the scope define the length of the release. (Read that sentence again; it's important!) Using time-boxed iterations is the fundamental difference between agile and plan-driven methodologies.

^[4] Again, releases may vary in length, but planning, tracking, and feature production are being done at the iteration level.

The next few sections detail the risks and benefits of time-boxed iterations. Agile methodologies require a very significant change in how we plan; however, as you'll see, there are many positive repercussions of this single change.

1.2.1. Risk Management via Estimation Accuracy

One of the biggest benefits of time-boxed iterations is that it improves estimation accuracy, which helps lower our risk of underestimating the task at hand. First, short iterations reduce the magnitude of our estimation errors. For example, a 10% error in a 12-month plan causes a 1.2-month slip, whereas the same error in a two-week plan causes only a one-day slip. Errors of that magnitude are much easier to manage.

However, the benefits of shorter, regular plans do not stop there. The basic fact that we are making smaller estimates reduces the error rate of our estimates. In general, the error in estimates increases with the size of the estimate. For example, it's a lot easier to recognize the difference between estimates of two and three hours than it is to distinguish 20 hours from 21 hours. Therefore, the absolute magnitude of our errors in estimation is likely to be much smaller if the length of our iterations is smaller.

In plan-driven development, estimating smaller tasks mitigates this non-linearity in estimation accuracy. Instead of estimating an entire release, the tasks necessary for that release are estimated individually, and the sum of those estimates is used as an estimate for the entire project. However, the assumption that those estimates are cumulative is questionable. Often, the actual length of the project does not correspond with the sum of the individual estimates of individual tasks. In most large releases, the tasks are related to each other, and therefore the errors in their estimates may interact. For example, if I forgot that one task would require a change to the database, I may have made the same type of error in the estimates of other tasks. This means that even if we make the tasks small enough to improve our accuracy, accumulating those estimates for a large release will induce more error. The bottom line is that estimating small tasks within small iterations will greatly improve our estimation accuracy.

With fixed-length iterations, estimation accuracy improves because we are estimating smaller things. However, that is not the only source of estimation accuracy improvement that results from fixed-length iterations. In addition, the rhythm of the iterations further improves estimating accuracy. After a few iterations, the team members learn how much they can complete during each iteration, and their intuition increases for estimations of that magnitude. The result is a reduction in the error of the estimates.

1.2.2. Reducing the Problem of Sequential Tasks

In The Goal[7], Goldratt gives an excellent example to show the problems with sequentially dependent tasks. Picture a Boy Scout troop walking single-file along a wooded trail. If the boys toward the front of the line are faster than a boy in the middle (who is named Herbie), the front part of the troop will pull ahead and will eventually be sitting at the end of the path waiting for the rest of the troop to emerge. Furthermore, the faster boys behind Herbie will be unable to achieve their maximum speed and will back up behind him. Regardless of the potential speed of these trapped boys, they will not be able to exit the path until Herbie is out of their way.

Essentially, the boys are a metaphor for sequentially dependent tasks in your development cycle, and the order of the boys matches the completion order of each task (i.e., the first boy represents the task that will complete first). The project is complete when the last boy reaches the destination. If there are sequential tasks and one is a Herbie (is delayed), the entire project will be delayed. Our plans may rarely be linear, but often tasks are dependent. Having a Herbie anywhere in your plan can greatly reduce your efficiency. The result may be idle people and altered plans.

It's also important to note that completing the early tasks (those that are ahead of Herbie) ahead of schedule might not alleviate the problem caused by Herbie. For example, if the Herbie task cannot start earlier, then it doesn't matter how early the tasks before Herbie complete. Underestimating Herbie still delays the completion of the project.

If you are planning a full release, then you are usually estimating tasks at the granularity of weeks, and it is safe to assume that the probability that any individual task will be underestimated is roughly the same for every task. As the number of sequential tasks in your plan increases, the probability that at least one of them will be underestimated (or delayed) approaches one, as shown in the chart below. Therefore, the longer the sequence of tasks is, the higher your odds are of delivering late.

Figure 1.1. Sequential tasks increase the probability of late delivery

Delivering the release as a sequence of iterations has the same effect as regular clearings along the path would have on the Boy Scout troop. If the boys had the opportunity, they could redistribute the weight in their backpacks at each clearing. Similarly, short iterations give us regular opportunities to adjust how we are approaching the problem. In addition, because each iteration is short, it is unlikely that Herbie can have much impact on the project, and the team can adjust before the next iteration begins.

The need for the traditional study of critical path analysis as a way of measuring and mitigating risk is a direct result of the length of the release. When we plan shorter releases, the number of dependencies between the tasks is reduced. That reduces the length of the critical path and the risk that we will deliver late. In fact, when iterations are small enough (less than a month), there are very few dependencies between the tasks, and we are not required to plan the order in which they will be completed. In effect, there will not be a single critical path. The team can see what dependencies exist and handle them without managerial input.

1.2.3. Feedback

One of the main goals driving the move to time-boxed iterations is to get feedback at regular intervals. By producing customer-visible functionality at every iteration, we allow the customer to evaluate the direction in which the product is developing and to change that direction if necessary. The team also has the opportunity to gather metrics about its behavior at fixed intervals, which greatly simplifies the detection of trends or anomalies. In fact, the regular rhythm of time-boxed development supports statistical process control much better than the long, variable-length releases of plan-driven development. The bottom line is that feedback is a cornerstone of all agile methods. We want feedback from the customer, from each other, and from the process itself. We want feedback about how effective our methodology is, how well our code works, the quality of our design, our individual performances, our performance as a team, and so on.

In fact, agilists are quite greedy about feedback: we want it as soon as possible. The short iterations guarantee reasonably quick feedback from our customers, but we also look for feedback in a number of the other techniques we deploy: feedback from each other at daily meetings, instantaneous feedback on our code from automated testing, regular feedback from the process as part of continuous integration, instantaneous design review feedback through pair programming, and so on. The value of feedback cannot be overstated: rapid feedback ensures that we notice when our performance is less than optimal and gives us the opportunity to fix the problem with a minimum of lost effort.

1.2.4. Is It Rework?

Some plan-driven organizations consider rewriting a piece of code as "rework," which is considered to be waste. Therefore, they go to great efforts to try to ensure that this never occurs. However, because agile teams plan and develop in short iterations, we clearly don't plan the design of the system for the long term. Therefore, it is likely that we will develop software during one iteration that will have to be significantly redeveloped in later iterations.

In fact, some agilists are decidedly shortsighted. Extreme Programming (XP) advocates do "the simplest thing that could possibly work" at each iteration without considering the functionality required by subsequent iterations. The justification is that we often are only guessing at the functionality that will be required, and requirements often change, so designing for those future requirements may turn out to be wasted effort. As an example, if a customer says the system only needs to remember five items, an XP team would store them in an array (or five variables) if that seemed the simplest solution. Later, if the customer required an arbitrary number of items to be remembered, the team would implement a more complex data structure like a list or hash map. If she never required more, the complexity of the hash map would be wasted, but if she did, the team would have some rework to perform.

Clearly, with short iterations, it is highly likely that code we develop for one iteration will need to be changed in later iterations. Plan-driven methods consider this to be rework and try to eliminate it. With those methods, changes late in the development cycle are risky and costly because they may require modification of the documentation and significant regression testing. In addition, changes made late in the development process are more likely to introduce defects and are therefore more risky. Agile methods have a number of strategies for reducing that risk, including automated unit testing, automated acceptance testing, and source code refactoring.

Essentially, agile methods weigh the savings resulting from the simple implementations against the costs and risks of needing to change the code later. Because these methods control those costs and risks, the benefits of reduced initial development outweigh the effort of "rework." In fact, agile teams do not consider making the necessary changes to be "rework" at all; those changes are just what the customer wants.