Section 6.1. Planning Innovations

6.1. Planning Innovations

6.1.1. Agile Planning Background

Agile planning techniques are based on the assumption that we need a plan but that it is likely that the plan will change. In fact, some organizations don't "plan," they "speculate."[2] This underscores the fact that as the project continues, our knowledge of the system and its requirements evolves, and the "plan" will have to adjust to that new knowledge. In this way, plans "serve as a vehicle for embracing change, not blocking it."[1] In other words, a plan is a commitment that holds only while our knowledge stays the same. It is also a good way to measure how much our knowledge has changed and an indication of when we need to replan.

Acknowledging that the plan is likely to change as we learn more about the project implies that time spent building long-term plans is not time well spent. In one plan-driven organization where I worked, management maintained a planning device they called the Every Person Chart (EPC). It reported which project every engineer was to be working on during each week for the next eighteen months. Because projects were regularly "missing commitments" (i.e., project members were learning and therefore revising their plans), maintenance of the EPC took on a life of its own. A new version came out about once a week to which various constituents would react causing further revisions. This was an incredible waste of effort.

The flaw in the EPC was that it was planning at a level at which we didn't have sufficient knowledgeit's very hard to know the details (to the week) of a development activity that is eighteen months away. Agile planning adjusts the granularity (the expected accuracy of the estimates) of the plan depending on how close we are to achieving the plan. The plan for the current iteration will be built from estimates whose granularity is in hours, but we will make estimates at a much higher level (weeks or months) for high-level plans like release plans.

When we make iteration plans, the estimates on which they are based will have some error in them. As the engineers start to work on the tasks, there is some chance that they will learn something that changes the estimates. This does not mean they have made a terrible mistake; this kind of learning is expected in an agile team. Because iterations are short, we need to be able to recognize these situations quickly so that we can replan as necessary. Tools like Burn-down Charts help us recognize these challenges without requiring a lot of overhead.

The error in longer-term plans is likely to be significantly greater than the error in an iteration plan for two reasons: the cumulative effect of estimate errors and the granularity of the estimates. Longer-term plans generally are based on a greater number of estimates than shorter-term plans. Each estimate has some error potential, and those errors are cumulative in the sum of the estimates. In general, those errors tend to be underestimates (where the estimate is less than the actual) because engineers are optimistic and often don't anticipate the need for some portion of the development. This means that errors in our estimates don't in general cancel each other out but instead tend to be cumulative. The more estimates that are combined in a plan, the higher the likelihood is that the plan will need to be revised. In addition, the error in an estimate is roughly proportional to the granularity of the estimate. When we are estimating that a task will take four hours, the error is likely to be measured in hours, whereas an estimate of two weeks is likely to have an error that is in days or weeks.

In general, there are three levels of agile planning. At each level, we must specify the required functionality and other activities (like development of load tests, user training, etc.) that are required to produce that functionality. Product plans cover the entire product from the current state of the project through completion. Release plans specify what will be included in each published release of the product. Iteration plans specify the functionality to be developed in a single iteration. Not all agile projects use all three of these levels of plans.

To support agility without sacrificing plan quality, all agile planning techniques share a number of common philosophies:

• Planning shouldn't take a long time.

• The more stakeholders involved in making the plan, the better it will be.

• The level of detail in the plan depends on the type of plan; short-term plans need more detail than long-term plans, and near-term plans need more detail than future plans.

As a result, most planning is done in meetings where the plans are structured so that they are visible to everyone and very easy to rearrange. In fact, many of the planning techniques involve index cards being arranged on a table or wall. The level of detail on the cards and the rules for how they can be manipulated depend on the chosen planning technique. Scrum, XP, Lean Software Development, and the Crystal methods all involve long-term and short-term planning on index cards.

6.1.2. Product/Release Planning

In Scrum, the product owner maintains the activities required by the project on a set of cards called the Backlog. These activities always have current estimates and priorities. This makes it easy to select the activities for the next iteration or for a release plan. For the iteration, the highest-priority activities whose estimates fit within the iteration velocity are the activities for the next iteration. Similarly, partitioning the list into sub-lists based on the velocity available for each release results in a release plan. In the backlog, higher-priority activities are more near-term activities, so they require more accurate estimates. Activities that have low priority in the backlog do not have to be specified in detail because it is likely that those details will change before the activity becomes high priority. Scrum does not specify a technique for finding and specifying the items in the backlog but allows the backlog to evolve over time. A single product owner who is responsible for the long-term vision of the project controls the backlog.

Although Scrum allows the backlog to be discovered over time, Crystal Clear offers Blitz Planning as a way for the stakeholders to find much of the product plan in an afternoon meeting. In addition to prioritizing functionality, blitz planning also specifies interdependencies between activities and includes this information in sorting the activities. Blitz planning also allows the stakeholders to group functionality into a release plan. In this way, releases can have logical goals. For example, marketing constraints may require that you put all of the advanced reporting features into a single release, even if not all of those features had the same customer priority. Because all stakeholders see all of the activities laid out during the meeting, these types of rearrangements are not difficult.

6.1.3. Iteration Scope Planning

Iteration planning requires two steps: determining the functionality to be delivered and determining how that functionality will be developed.

Selection of the functionality for an iteration requires a process that includes the customer, management, and the development team. The development team has two inputs into that process. First, they provide the estimates of the time required by each piece of functionality under consideration. This helps ensure that the estimates are realistic and that everyone has considered all of the activities that will be required.^[1] Second, they specify the amount of development time they will have available for the iteration. This is called the team's velocity and may be measured in hours or other effort-related units. When the team sets its velocity, it can account for planned absences (vacations, doctor's appointments, etc.) and for internal activities (tool development, upgrades, or significant source code refactorings). Because these planning activities happen with every iteration, any functionality that has to be delayed doesn't get pushed back very far because the customer can choose to put it in the subsequent iteration when it is planned.

^[1] However, as in longer-term plans, the estimates are often an accumulation of a number of rough estimates and will have some error component. They are only as accurate as the knowledge of the engineers allows and often include some measure of how confident the engineers are in the estimates in order to help the customer to manage the risk associated with the tasks they select.

In Scrum, the functionality to be delivered in the next iteration is just the highest-priority activities from the backlog that fit within the velocity of the sprint. In addition to that functionality, a goal for the iteration is specified that ties together the functionality being developed. This can be used later if the team runs into trouble. Although they may need to scale back the details of the functionality being developed, they continue to try to meet the general goal of the sprint.

In XP, the goal of the Planning Game is to maximize the value that the functionality has to the customer while making the risk explicit. The planning game allows the customer to specify all of the functionality and then select what they want the team to work on in the current iteration. Sometimes the planning game is used to select the functionality for a few iterations at once to give a slightly longer (but not usually release- or product-level) view of the project.

6.1.4. Iteration Planning

After the requirements of the iteration have been specified, the development team has to make a plan for how those requirements will be met. This is called the Iteration Plan. In other words, the development team needs a more detailed plan for the iteration. To promote ownership and accuracy in the plan, iteration plans are generally developed by the engineering team and not by its leadership. The iteration planning process is generally a structure that allows the team to generate the list of tasks required to develop the required functionality and determine who will be responsible for each task, but the formality (or lack thereof) of that process can vary significantly. Some teams follow a formal structure, while others use a more ad hoc planning philosophy. The decision for how to structure this process is generally left to the team.

The simplest iteration planning technique is the visible pull system known as Kanban Scheduling in which the team determines the set of activities that will be required to complete the iteration and posts them in a display. There is no scheduling of tasks. Instead, everyone selects a task whenever he is idle.

Generally, the tasks in the iteration-level plan should be able to be completed in between a half-day and four days, but the maximum length you can tolerate may depend on the length of your iterations. Scrum and XP develop their iteration plans using the Sprint Planning Meeting and Iteration Planning Game, respectively.

Although iteration plans are detailed in their specification and estimation of tasks, it is common to ignore the dependencies between tasks. In general, because the iterations are short and the team is working closely, there are few dependencies, and the team can handle those that exist without management intervention. This means that there is no need for dependency and critical path planning with PERT charts. Iteration plans generally only need to specify the tasks and who is responsible for each task in order for the iteration to go well.

The goal of iteration planning is for the team to develop a plan that it believes it can complete. Plans without commitment are not useful plans. If the team completes the planning only to discover that the tasks do not fit into the time given to the iteration, then we have an early indication that too much functionality has been committed for this iteration, and we can work with the customer to adjust appropriately.

The sprint planning meeting allows the team members to use whatever method they prefer to compile the list of tasks that will be required to meet the sprint goal. In addition, they are allowed to "self-organize" in that they determine how tasks will be assigned to engineers. In fact, the team members can choose to plan only the first portion of the sprint if they think that those early activities could change how they plan the later activities.

The iteration planning game is more structured than the sprint planning meeting; it includes specific strategies for developing the list of tasks, mapping tasks to engineers, and load-balancing the activities among the engineers. In particular, it is very clear about the development of estimates for activities: only after an engineer has chosen to work on a task can he give an estimate for the effort that task requires. This is a critical part of promoting the ownership of tasks and estimates that is core to XP's values.

6.1.5. Iteration Length Planning

To improve your feedback or maximize your efficiency, sometimes you need to Change Iteration Length, but sometimes Pipelining Iterations is a good alternative. In pipelining, portions of your development process are completed in sequential iterations. For example, if your product requires significant requirements analysis that cannot be completed during iteration planning, you could make the requirements analysis for the next iteration overlap the development of the current iteration as shown here.