Per Kroll Iterative development cycles give rapid and timely feedback so that the software can be continually improved along the development lifecycle. ProblemToday's software applications are too complex to allow you to sequentially define the requirements, come up with an architecture and design, do an implementation, carry out testing, and get it all right. Whether you use waterfall or iterative development approaches, your initial requirements, architecture, design, and code will be suboptimal. With waterfall development, you typically do not get meaningful feedback on what improvements can be made until it is so late in the project that it is too costly to make them. By contrast, dividing the project into a series of time-boxed iterations allows you to deliver incrementally capabilities that can be assessed by stakeholders at the end of each iteration. This approach provides rapid and timely feedback loops enabling issues to be addressed and improvements made at a lower cost while budget and time still allow, and before the project has gone so far ahead that major rework is required. BackgroundAssume that you want to write a book. How would you go about it? You could start by writing page 1 and then continue page by page until you have completed the entire book. You could then give it to people to read and review. But what might happen if you followed such a process? Your publisher might say that there is no market for your book, at least not without a major rewrite; your reviewers might point out that the book needs major restructuring to be readable; and your editor might ask you to write in a different style altogether. In the end you might need to scrap half of the manuscript or more and do it all over again. This sounds like a crazy approach, right? But many teams develop software this way. Let's look at an alternative approach to writing a bookthe one we used when writing this one. The main idea was to get rapid feedback on key elements of the book so that we could rethink and improve it. We wanted to get this feedback before we invested the time required to write the complete text. We therefore divided the writing into a number of succinct iterations, each of which resulted in a well-defined deliverable that could readily be assessed, to ensure that we all understood the status of our book project and to get meaningful feedback on how to improve the book. The iterations included the following:
As you can see, each iteration has a well-defined deliverable that can be readily assessed. Based on that assessment, we may scrap some of the work done and then determine what changes we should make to the overall project, including modifying the timeline or scope of the book. You can also see that we first flesh out a broad solution and then keep detailing and refining it. Finally, we keep spearheading the effort by picking critical, but narrow, areas where we provide detailed solutions to drive out risk and to understand what the overall solution will look like. The above iterations map well to the Unified Process lifecycle, where Inception is iteration 1; Elaboration is iterations 2 and 3; Construction is iterations 4 and 5; and Transition is iterations 6 and 7. Many software teams, however, are in effect still writing the book straight through from the first page to the end by using a waterfall process for development projects. They complete each phase in strict sequence: requirements analysis, then design, then implementation/integration, and then testing. Or, more commonly, they use a modified waterfall approach, with feedback loops added to the basic overall flow described above. Such approaches defer integration and testing until the end of the project lifecycle, when problems tend to be tough and expensive to resolve, as well as posing serious threats to release deadlines and project budgets.
By contrast, an iterative approach divides a project into a sequence of iterations, each of which evolves through the requirements, analysis, design, implementation, and testing assets, as you can see in Figure 2.2. Early iterations place a greater emphasis on requirements, analysis, and design; later iterations focus more on implementation and testing. Each iteration has a well-defined set of objectives and produces a partial working implementation of the final system. Further, each successive iteration builds on the work of previous iterations to develop and refine the system until the final product is complete. Figure 2.2. Iterative Development.Each iteration involves some requirements, analysis, design, implementation, and testing. Each iteration builds on the work of the previous one to produce an executable that is one step closer to the final product.
Advantages of the Iterative Approach[8]
Some project managers resist the iterative approach, seeing it as a kind of endless and uncontrolled hacking. The risk-driven iterative approach is, however, very disciplined: the iteration length is fixed; the objectives of iterations are carefully planned; and the tasks and responsibilities of participants are well defined. Additionally, objective measures of progress are captured. Some reworking takes place from one iteration to the next, but this too is done in a structured fashion.
The iterative approach has proved itself superior to the waterfall approach for a number of reasons:[9]
Applying the PracticeTo build your project effectively in iterations, you need to (1) decide how many iterations you need and the appropriate length of the iterations; (2) determine what risks you are facing and how they could impact your iterations; (3) figure out what executables and deliverables should be produced in each iteration; and (4) assess the iterations and leverage the experiences gained during the execution to fine-tune your plan for the next iteration. Let's explore each of these activities. Decide the Number of Iterations and Their LengthTo decide on the number of iterations and the iteration length suitable for your project, you need to have an idea of the staffing level and duration of the project. First, size the project using the estimation approach of your choice, such as function points, COCOMO, [14] or expert opinion, where past experiences from similar projects are used to estimate future projects. Then determine a rough staffing profile, and use this information to assess the approximate project duration.
Next, you need to determine what is an appropriate iteration length for your project. We recommend using four weeks (this is the default recommendation by RUP and Scrum, while XP recommends one or two weeks) as a starting point for your discussion around iteration length. Then analyze factors that will impact the iteration length. Table 2.3 provides an overview of many of these factors.
You also want to keep the total number of iterations to a reasonable number. Project teams new to iterative development may have problems handling a project with more than three to four iterations, whereas project teams experienced in iterative development may have projects with ten or more iterations. Based on the above factors, you could find that the appropriate iteration length is two weeks for a four-month long project with four collocated, experienced team members with a good tool environment. Meanwhile, a two-year, technically complex, distributed project with sixty team members, inexperienced in iterative development and with a poor tool environment, may require an iteration length of six weeks. Large projects should, however, consider breaking up the project into several smaller projects, each delivering a part of the end solution. See also the section Iteration Profiles for Very Long Projects on page 53. Determine What Risks Need to Be Addressed and in What OrderEach iteration should produce the working software and, potentially, other deliverables required to address a set of risks, but what risks should be addressed at what time? You want to address the most serious risks firstthose that will most severely impact the project outcome. But how do you handle risk that is not known early in the project? This is where the four phases in the Unified Process lifecycle come in handy.
The Unified Process lifecycle provides a consistent approach to managing projects when the phases are coupled to risk mitigation. These phases are also powerful tools in iteration planning. They are, in order:
Using the iteration length and number of iterations, you can now determine how many iterations you need in each phase to address the type of risks and produce the type of results expected from each phase. If your project is facing a lot of risk associated with the business case, scope, and stakeholder buy-in, plan for more iterations in the Inception phase. If your project is facing a lot of architectural risk, plan for more iterations in the Elaboration phase, and so on. It could be that the four-month project with two-week iterations will have an iteration profile of 1, 1, 4, 2, that is, one iteration in Inception, one in Elaboration, four in Construction, and two in Transition (see Table 2.4). This profile would indicate that the project planners expect to produce stable requirements, a business case, and a baselined architecture very quickly. On the other hand, the two-year project with six-week iterations may have an iteration profile of 2, 4, 8, 2, meaning that the planners expect to need a lot of time to get the architecture right and expect to make steady progress once that is done.
Determine a Series of Executables and Deliverables That Will Take You to the End Point
Once you have established the initial iteration profile (which you will have opportunities to rethink later in the project based on progress made), outline at a high level the expected end result of each iteration, primarily expressed as what capabilities should be implemented and tested with favorable feedback from customer representatives. These are similar to Sprint Goals in Scrum (Schwaber 2002). It is often enough just to document a few bullets. The Background section at the beginning of this chapter provided a good example of such a brief list in relation to our book project, showing what should be accomplished in each iteration. This is a part of what the Unified Process calls the Phase Plan,[15] and there is one such plan for the entire project. The Phase Plan is purposely kept at a high level, typically only one or two pages long, for small projects.
Assess Iteration and Plan the Next IterationProject managers who produce detailed iteration plans for entire projects at the start of a project are usually wasting everybody's time conveying false precision where there can be none. Instead, detailed planning should be carried out incrementally, when sufficient information is available.[16] Toward the end of each iteration, the team produces a detailed plan of the next iteration and outlines what capabilities should be implemented and tested, including how to assess the iteration through, among other means, test cases, and demonstrations to key stakeholders.
At the end of each iteration, assess the iteration by analyzing test results, reviewing what was produced, and collecting feedback from customer representatives. The iteration assessment gives you an objective and precise reading of where you are. You can now compare where you are with where you expected to be and where you need to go; and you can then use this information to update the Phase Plan as needed. The Phase Plan is hence used as a high-level and evolving roadmap, showing where you expect to be at the end of each iteration and where you need to end up at the completion of the whole project. Using a Phase Plan in combination with iteration assessments is a very powerful tool that allows you to understand overall project performance and helps you decide whether you need to make changes to the project, or even to kill it altogether.
Iteration Profiles for Very Long ProjectsSome large or long projects may be difficult to break up into smaller projects. For these projects, we use the notion of super-iterations and mini-iterations. A super-iteration is typically several months long and provides a well-defined integration point for the entire project that is formally assessed, resembling what XP calls quarterly cycles. In contrast, mini-iterations are typically a few weeks long and provide sub-teams with the benefits of short iterations with quick feedback loops. Let's look at an example: A 100-person project lasting two years may have super-iterations that are three months long: every three months there is a very well-defined and well-tested deliverable integrating everything that has been done and delivering something of tangible value to the end users. These super-iterations are tracked in the Phase Plan and used as a major decision point for the project. The project may be divided into ten or so subteams. Each subteam will divide its work into, for example, three-week mini-iterations, with four mini-iterations in each super-iteration. Each mini-iteration is tested and validated with the stakeholders specifically interested in that set of capabilities. The subteam thus receives quick and timely feedback on what it is doing, as well as gaining increased maneuverability by becoming less dependent on the work of other teams. Projects should aim to integrate and test all code frequently, ideally creating a build and automatically testing it after each check-in, as recommended by XP. In our experience, this method is not feasible for many larger or legacy projects, due to lack of skills, lack of test automation, poor build processes,[17] or the complexity of larger projects in which meaningful automated test suites may take too long to run. Other examples include distributed development, or development done across organizational borders, leveraging different infrastructures, or cases where some subprojects develop components, such as hardware components, that cannot be evolved continuously as software can. The mini- and super-iterations provide a framework for managing these situations by having subteams do frequent integration and testing of their set of components, while doing integration and testing of the larger systems less frequently. Comprehensive testing, such as load and performance testing, may be carried out primarily toward the end of super-iterations, with the last few weeks focusing solely on stabilizing the build and making quality improvements.
Other MethodsA strict waterfall project runs a project as one iteration, with a strict sequence of requirements, design, implementation, and testing. Often, such a project is forced to add one or several bug-fixing iterations at the end to deal with unexpected integration and quality issues. In practice, few projects do strict waterfall development. Instead, many apply mini-iterations as part of a larger waterfall project. As issues are identified, teams are assembled to do a mini-iteration to drive out a specific risk. At any given time zero to several teams may be doing mini-iterations, often without synchronization in terms of common iteration ends. In 1988 Professor Barry Boehm[18] introduced the spiral model, which is an incremental and risk-driven development approach. In this model a project is divided into a number of iterations, each of which goes through planning, requirements, design, implementation, integration and testing, and so on sequentially. Each iteration is hence a mini-waterfall project, delivering a product increment.
Iterative and agile methods such as RUP, Scrum, XP, and OpenUP/Basic apply an iterative approach in which each iteration continuously develops the requirements, design, implementation, and testing without a sequential order. Sometimes you do a little coding to understand how to improve the design, only to do more coding the next day, followed by some testing, and then some requirements work. RUP, Scrum, XP, and OpenUP/Basic vary, however, in their recommendations for iteration lengths and how individual iterations are executed.
XP recommends short one- to two-week iterations with continuous integration and testing to provide quick feedback loops. The work of each iteration is prioritized by the customer or customer representative, and should deliver what is deemed most important to the customer. Approaches using the Unified Process lifecycle, on the other hand, prioritize based on a combination of risk and customer importance. Each iteration delivers an executable that may be put into production. Scrum recommends four-week iterations. Scrum projects focus on continuous communication, with a short daily stand-up meeting with the entire project team at which each person briefly reviews status and discusses any roadblocks that need to be resolved. Each iteration has a well-defined objective, and no changes to the objectives are allowed during the iteration. Scrum manages large projects through a "scrum of scrums," in which representatives of each subteam get together daily to resolve cross-team issues. The Unified Process has many similarities to Scrum. It recommends four-week iterations but advises you when to consider longer or shorter ones. Iterations in both Scrum and Unified Process should be executed without changes to the objectives, except in extraordinary circumstances. There are also some notable differences between the two methods, though. The Unified Process lifecycle defines four distinct phases, each having a well-defined business objective and containing one or several iterations. Another key difference is that the Unified Process integrates the management lifecycle with a process for how to carry out the technical work, to facilitate practical implementation of the management process, and ensure that the management process works well with the technical process. Scrum, on the other hand, addresses only the management aspect, allowing integration with a variety of processes. Levels of AdoptionThe iterative development practice can be adopted at three levels:
Related Practices
Additional InformationInformation in the Unified ProcessOpenUP/Basic guides you in how to divide your project into iterations, and how to plan iterations by balancing the continuous need to deliver end-user capabilities with the need to mitigate risk. RUP provides more in-depth guidance on iteration planning, issue, and exception management and how to deal with problem resolution. RUP also provides guidance on formal reviews, Project Review Authorities, and monitoring and control processes appropriate for larger or more complex projects or projects concerned with compliance. Future versions of RUP will provide explicit guidance on mini- and super-iterations. Additional ReadingThe following books provide guidance on iterative development leveraging the Unified Process lifecycle:
The following books provide guidance on other iterative approaches:
|