Misconstruing the Message


Over the last five years, people have been quoting agile rhetoric without adopting the elements that make the agile approach work. This section is my little attempt to correct some of the misquotations of the agile model that overzealous would-be agile developers inflict upon their managers.

Before getting into ways to misconstrue the message, let's first review what the message is supposed to be (or at least how I think about it).

  • First, create a safety net for the project, one that increases the chances of a successful outcome. This safety net consists of the following:

    • Everyone on the team sits within a small physical distance of each other (Osmotic Communication).

    • The team uses an automated tool suite that integrates the code base every short while (continuously, each half hour or each hour), runs the test suite, and posts the result on an information radiator.

    • The team, working in small increments so that they check in code and tests several times a day, creates unit and system tests for the automated integration system to run.

    • The sponsor and key users visit the project team often (daily or once a week) to offer insights, feedback, and direction.

    • The team delivers running, tested features (RTF)[1] to real users every week, month, or quarter.

      [1] A neat term coined by Ron Jeffries in http://www.xprogramming.com/xpmag/jatRtsMetric.htm.

    • Following each delivery, the sponsors, key users, and development team get together and review their mission, the quality of their product, and their working conventions.

  • After the safety net is in place, see what can be removed from the development process. Typically, requirements documents can be lightened, planning documents put into a simpler form, and design documentation done in a different way. Typically, more training also needs to be brought in, more test automation done, designs simplified, and community discussions increased.

The point I wish to make is that simplifying the process is a reward the team gets for putting the safety net into place. Many would-be agile developers take the reward without ever putting the safety net in place.

Here are some of the harmful misquotations of the agile manifesto I have heard.

"Iterations must be short"

A macho culture has developed around making iterations shorter and shorter.[2] Teams that fall into this trap end up with iterations that are too short to complete features within. The iteration devolves to a mere planning period.

[2] This issue is discussed at length in the article "Are Iterations Hazardous to Your Project?" (Cockburn 2005d)

A telltale sign that the iterations are too short is that the team gets preoccupied with the word iteration. We see teams charting one iteration's worth of work without seeing how the various specialists' work fits together or how the output of the iterations grows toward a useful delivery.

The mistake is assuming that short iterations are uniformly good. An iteration should, in theory, permit the following:

  • The business experts work out the details of the features to be built in this iteration.

  • The programmers program them.

  • The users, business experts, and testers check that what got built is adequately correct and comment on what needs to be changed.

  • The programmers change the software to take those comments into account.

  • They repeat the last two steps until the business experts, users, and sponsors feel the features are suitable for deployment.

Now imagine a fairly normal team of business experts, programmers, and testers (UI design experts will notice the usual absence of UI designers!) who have chosen an iteration length of two weeks. The team finds it cannot do all five steps in just two weeks. So, they decide to pipeline their work:

  • In the first two-week cycle, the business experts work out the details of feature set 1.

  • In the next two-week cycle, the business experts work out feature set 2, and the programmers program feature set 1.

  • In the next two-week cycle, the business experts work out feature set 3, the programmers program feature set 2, and the testers test feature set 1.

They run into a problem immediately: The groups are all working on different feature sets. Discussion that felt like collaboration when they were working on the same feature set now feels like interruption. When the testers in the third iteration ask the business experts a question about feature set 1, they force the business experts to remember issues examined two cycles ago, a month in their past.

To protect themselves so that they don't get bothered while they work out their new decisions, the business experts write documents to give to the programmers and testers. Similarly, the programmers create documents to hand to the testers.

In pipelining their work, they have created a time barrier that can be crossed only using archival forms of documentation, which as we saw in Chapter 2 are expensive, tedious, and relatively ineffective forms of communication.

Some agile experts are really opposed to lengthening the iteration, and for good reason. As the iteration lengthens, people have a tendency to delay their work, pushing it all toward the end of the iteration. As Dan Gackle once wrote, describing his team's use of one-month iterations:

"Meander, meander, meander, realize there isn't much time left, freak out, get intense, OK we're done. Repeat monthly."

Agile coaches are torn between the desire to lengthen the iteration to get good feedback, and the desire to shorten the iteration to get results quickly. Going overboard in either direction creates a trap for the project.

"So what should we do?"

Become preoccupied with the word "delivery."

Understand that each iteration builds toward the next delivery, and know exactly what each iteration contributes to the next delivery. With this in mind, determine the length of the iteration based on knowledge of the domain, the technology, the user base, and the communication patterns. Consider at what pace the team can deliver running, tested features to real users while also collecting and incorporating feedback from those users. Balance brevity of iteration with the quality and completeness of team interactions.

Retain in all cases the idea that a feature is not "done" until the key users have had a chance to see it and change it at least twice. Although this is not in the agile manifesto, it is a critical policy for the software to be considered successful in use.

"Agile teams must be colocated"

Distributed teams lose the high levels of communication, trust, and project safety afforded by colocation. Project managers who once led colocated teams but then got assigned distributed teams repeatedly comment to me about the loss of trust and the difficulty in rebuilding it. This is independent of whether or not they were using an agile approach.

While it is clear that working with a distributed team makes agile development harder, it doesn't mean you can't use the agile model. The would-be-agile distributed team must put more energy into its communication and team-building strategies and find times to get the team together face to face.

Automation tools and high-speed communication technologies help. Web cameras and microphones attached to workstations let people see and talk to each other in real time. Instant messaging lets them trade code in real time. Net meeting software lets them pair program. Automated build-and-test tools let people in different time zones work from the same code base.

Distributed teams must be careful to work in small increments, integrating and testing multiple times a day. Because communication is not as fast, synchronizing the code base becomes more important as the team gets separated.

A distributed team must put more energy into simplifying the system's architecture (Cockburn 2004a). The reason has to do with Naur's "Programming as Theory Building." Because the communication channels between people are constrained, it is more difficult for them to build a shared theory of what they are developing. The more complicated the architecture is, the more complicated the theory is, and the less likely it is that they will work to the same decision principles. Conversely, with a simpler architecture, they have a greater chance to keep the theory intact across distance and intermittent communication.

Even with all that, there is still no substitute for personal, face-to-face encounters to build personal safety and trust. Good distributed teams get together initially and periodically for both technical work and social interaction. Through these get-togethers, they build shared mental models of technical topics and personal trust.

Part of the trouble with distributed development is that executives and managers have never learned how to develop software here. They come to think that perhaps they know how to develop software there, or that if there is going to be a mess, it is better out of sight, and if there is going to be a failure, the failure will be less expensive if it happens there. What I am interested in is learning and teaching how to develop software here, for any here. Once we learn this, we can develop software both here and there.

I don't expect to reverse the trend of distributed development. I do want executives to consider more of the issues involved, particularly the higher communication costs.

A success in this regard was when one executive, who was running projects split between Pennsylvania and North Carolina, turned to her colleague and said (roughly): "I think that we could have run that last project just in Philadelphia. Even though Jim (the best programmer) is in North Carolina, we could have used Martin (in Philadelphia) instead and saved a lot of trouble." She was balancing the increased expertise she could get in multiple sites against the lower communications costs and higher team productivity she could get with a colocated team.

"So what should we do?"

Pay even closer attention to the agile values than colocated teams do. Simplify the architecture. Add new face-to-face communication opportunities. Use an automated build machine and check your code in every few hours.

Keep your ear to the grapevine and pick up new ideas being tried in other teams.

"Agile teams don't need plans"

Early critics of the agile movement worried that we were dropping project planning, which would be dangerous to the project. Sadly, I have seen exactly this (dropping planning) happen all too often, and with the expected damaging consequences.

It seems that quite a number of overzealous would-be agilists say to their managers, "If we are sufficiently agile, we don't need planswe can always react quickly enough. If we need plans, we are just not agile enough." I have lost count of the number of executives who tell me that this is how their programmers present agile development to them.

The final value comparison in the agile manifesto states that agilists value "Responding to change over following a plan."[3] That is followed by the tempering statement, "While we value the items on the right, we value the items on the left more." Planning is useful in both cases.

[3] See http://agilemanifesto.org.

The value comparison in the agile manifesto is intended to compare

  • Making a plan and periodically reconsidering whether the reality currently facing the team is better served by keeping to that plan or by creating a new one, against

  • Making a plan and continually trying to get back onto it.

When some aspect of the project doesn't work out as expected, a sudden staff change hits the team, or some external event changes the sponsors' priorities, the team must decide whether to get back to the written plan or to change both the goal and the plan. It is not part of the agile manifesto to assert which is betterto get back onto the plan or to create a new one. It is a part of the agile manifesto to assert that one should make that choice deliberately.

Except on certain exploratory projects, there is almost always enough information to make a coarse-grained plan that is aligned with the project and the organization's overall vision. This coarse-grained plan serves to make coarse-grained cost and time estimates. Those estimates are badly needed by the sponsors to weigh the project against other possible initiatives.

We worked to repair the situation as we could in the Declaration of Interdependence. The third principle in the DOI states:

"We expect uncertainty and manage for it through iterations, anticipation, and adaptation."[4]

[4] See http://pmdoi.org.

This phrase highlights that uncertainty is a factor that cannot be completely eliminated but can be mitigated with the feedback mechanisms in standard use by agile teams (iterations, reflection, adaptation), plus making use of information that is already availableanticipation.

It remains to be seen whether upper management will accept that uncertainty cannot be eliminated. It also remains to be seen whether agile zealots will accept that anticipation has value.

In practice, agile teams tend to produce plans at two levels: long-term coarse-grained plans and short-term fine-grained plans.[5]

[5] Examples of each are shown in Cockburn 2005a, pp 154-165.

On one 18-month, $15 million fixed-price project, we worked from 240 casual use cases,[6] a list of external systems that would have to be touched, an estimate of the number of domain class and screens that would have to be built, and a release plan that consisted of a dozen bubbles with dependency arrows between them, with lines marking the five deliveries that would be made. Such coarse-grained long-term plans are typically used for projects greater than three months in length.

[6] "Casual" use cases are written in only two paragraphs. They are described in (Cockburn 2001c).

The short-term fine-grained plan is used for timeframes between one week and three months. Inside the one-month timeframe, work items are typically broken down into pieces between half a day and three days long.

These two-level plans have several advantages:

  • They paint an overall picture of the project and still allow room for movement.

  • The long-term coarse-grained plan shows the overall direction, the timing, the approximate cost, and how the iterations fit together to deliver the best value over the long term.

  • The short-term fine-grained plan is used to coordinate teams in detail.

  • Each level can be updated quickly.

  • The team can balance the needs of setting overall direction with tracking detailed work.

Any change in a long-term plan invalidates a large amount of work in a detailed plan, and we can be certain that there will be changes in the long-term plan. By expanding the detail of the plan just prior to starting an iteration or a delivery cycle, we reduce rework in planning.

"So what should we do?"

Create a coarse-grained long-term plan to know where the target is and a fine-grained short-term plan for the next week, month, or quarter.

Helmuth von Moltke

Agilists often defend their aversion to detailed long-term plans by quoting the German military strategist Helmuth von Moltke: "No battle plan survives contact with the enemy."

I found this a compelling quote until I learned that there were two Helmuth von Moltkes who might have said it.

Moltke the Younger (Helmuth Johann Ludwig von Moltke, 1848 to 1916) has been (controversially) blamed for Germany's loss in the Marne campaign and consequently the First World War.

"As Chief of the General Staff Moltke was responsible for the development and execution of the strategic plans of the German Army. There is considerable debate over the nature of his plans. Critics from the so-called 'Schlieffen School' argue that Moltke took his predecessor's plan . . . modified it without understanding it, and failed to execute it properly during the First World War, thus dooming German efforts."[7]

If Moltke the Younger said that no battle plan survives contact with the enemy, then those opposed to the agile viewpoint would respond that this was spoken by someone who clearly hadn't planned well enough.

On the other hand, Moltke the Younger's uncle, Helmuth Karl Bernhard von Moltke (1800 to 1891, known as Moltke the Elder), didn't lose any wars for Germany, was very successful in battle, and was eventually promoted to Field Marshall. If it was Moltke the Elder who said the quote, then the previous retort can't be given, and "No battle plan survives contact with the enemy" meaningfully supports the view that even the best plans need frequent updating.

Fortunately, it was Moltke the Elder who said it.


[7] See http://en.wikipedia.org/wiki/Helmuth_von_Moltke_the_Younger and http://en.wikipedia.org/wiki/Helmuth_Karl_Bernhard_von_Moltke.

"Architecture is dead; refactoring is all you need"

Similar to not needing plans is the overzealous view that architecture is not needed: "If you are sufficiently agile, you don't need an architectureyou can always refactor it on the fly," say some people.

Let's look at both sides of the argument, with the idea that the extreme of either probably is not good for most projects (but possibly is good for some project somewhere).

Good agile developers start with simple architectures and then evolve them. I have seen good developers start with simplistic inter-computer messaging mechanisms and later substitute commercially provided messaging systems without difficulty. This pattern of development is called Walking Skeleton with Incremental Rearchitecture (Cockburn 2004a, Cockburn 2005a).

The advantage gained is that the team, sponsors, and users have a system up and running early. From there, they can steer the feature set, the architecture, the staff composition, the use of technology, and the development process. The cost to the strategy is that the team must make changes to the system while development continues.

The countering argument is that a more complex original architecture would probably also be wrong in unexpected ways and would also need to be changed.

I have seen and heard of projects saved by deploying a simple architecture early and changing it later after they saw where its shortcomings were, and I have seen and heard of projects that failed due to an overly ambitious initial architecture that could never be fully developed and deployed at all. I therefore consider the cost of upgrading a simple initial architecture as nothing more than the insurance premium for increased project safety and for reduced risk afforded by having an early, running system.[8]

[8] A longer discussion of paying extra for flexibility is part of the articles series "Learning from Agile Software Development" (Cockburn 2002a).

Changing the architecture is possibly a good thing in itself. I ran across the following argument somewhere along the way and have examined it closely with some friends. We think it is sound. See if it makes sense to you.

Figure 5.1-1 shows the cost of refactoring versus not refactoring code over time. The core idea is that when people make changes to the code, they add small, extra bits of complexity, perhaps an extra branch condition or some code duplication. These little bits of complexity are not independent but start to intertwine. A new branch condition is based on a previous branch condition; code duplication gets copied, modified, and gets its own new branches. When these little bits of complexity start to intertwine, the total system complexity goes up, not linearly as it would if they were completely independent, but exponentially.

Figure 5.1-1. The graphical argument that constant refactoring is less expensive in the long run and extends a system's life span (see text).


The cost for updating the software looks like interest growth on a financial loan: (1+c+u)n. Here, c is the necessary complexity added to the code through the new business rules, u is the cost for understanding the unnecessary intertwining across the changes, and n is the number of changes put into the code. Even if u is a small number, over thousands of changes across a period of years the cost to understand and modify the code grows to a point where the system simply can't be updated in the time-frame needed by the business.

The graph shows how modifications to a software system start out easy but become harder and harder over time until finally the programmers tell the business owners, "We can't make any more upgrades to this system. We have to start over."

When you refactor constantly to clean up the code, the exponential curve is not as steep. It does cost time to clean up each new piece of complexity, but these cleanup activities don't intermix with each other, so the cleanup activity costs less. The cost is (1+c)n+ (r*n), where r is the cost of doing the refactoring. The code still grows more complex, but the growth is slower, so the system lasts longer before slowing down the business (rather like paying back a loan with 5% interest instead of 11%).

Notice that initially, the refactoring cost is greater than the cost of understanding a few extra bits of complexity, but over time, the two trade places, as shown in Figure 5.1-1.

I note that the increased cost of understanding the system as it grows in complexity is part of what Peter Naur talks about as "theory building" in Appendix B. It is an intrinsic cost, meaning that it can't be removed.

So far in the argument, we have seen that starting with a simple architecture is good and refactoring is good. Where's the problem?

The problem is that sometimes the development team has information available that can reduce the refactoring cost. Ron Crocker's example illustrates (Cockburn 2004a):

Ron Crocker was lead architect at Motorola on a new mobile phone platform that was used from the transmission towers to handheld and PC-based devices. They were faced with one core architectural question: whether to design the mobile phones to handle both circuit-switching and packet-switching traffic right from the start, or just circuit-switching. They suspected they would need both. The refactoring approach would have had them design the system to handle circuit switching, to add the capability for packet switching only when that became strictly necessary, and to refactor their architecture at that time.

Ron's development team consisted of 250 people in six countries, and I knew that this was a factor in his decision to design for both right from the beginning. So I asked Ron to estimate the cost of communicating the new redesign under the simplest circumstancespretending that their project consisted of only 30 people, all located in the same building on the same floor. After some pause, he estimated that they might have been able to migrate the developers to understanding the new architecture in "merely" a few weeks.

"But why would I want to do that?" he asked, meaning, why would they want to cost the entire team two weeks of their time, when the requirement was known from the beginning?

In fact, with 250 people in six countries, the cost of communicating the revised architecture would have been enormous. Since the requirements were known from the start, it made sense for his team to design for both right from the start. It is relevant to mention that his team had designed these types of systems before, so they were embarking on a familiar, rather than unfamiliar, path.

On another project, the lead architect, who was contracting out the work to an XP team, watched in frustration as the subcontractors ignored the known requirements to support a variety of peripheral devices and implemented a series of designs one after the other, adding each peripheral device as a surprise addition each time. He was frustrated because he knew they could have cut quite a few revisions if only they had considered the future requirements for the peripherals much earlier.

This is where some balancing of the issues is called for.

We have learned in the last ten years that simple designs with clean interfaces really can be altered and extended with quite low cost. The cost is low enough that very often, a simple, clean design followed by a targeted change comes out at a better overall cost than an overly complicated initial design that takes longer to implement and is difficult to both learn and modify.

What we need to take into account is that good agile developers are quite experienced and have good design reflexes. Their designs are simple and clean and refactor easily. The refactored design is again tidy and clean and allows itself to be refactored again.

Inexperienced designers are likely not to have such good design reflexes. They can benefit from being forced to think through the issues of scalability, performance under growing loads, and probable changes in requirements. The thinking time is well spent, even if they decide to implement a simpler design.

This has been a long section. Architecture is not dead, and refactoring is not a cure-all. Thinking about design is still important, as is developing good reflexes about simple, tidy designs.

"So what should we do?"

This question has no simple answer, as far as I can tell. Different people think ahead to different time horizons.

I like Ron Crocker's advice: Create a simple architecture that handles all the known "big rocks" (requirements that are particularly hard to incorporate late in the project).[9] Handle the "small rocks" as they appear during the project.

[9] (Cockburn 2004a).

A good architecture is simple and understood by all on the team. This takes thinking. At the same time, good agile teams have demonstrated that with clean designs, even significant changes can be made remarkably quickly.

"We don't need no *%@! managers!"

Along with plans and architecture, overzealous agilists dismiss the role of project manager. Here again, we need to look at the issues involved.

From interviewing project teams and managing projects myself, I have made the following list of ten factors critical to project success, in five areas (see Figure 5.1-2):

Figure 5.1-2. Ten critical factors in project success.


  • Sponsorship. Without adequate sponsorship, the project will starve for resources. Sponsorship consists of

    • Money. For staff, training, and equipment.

    • Decisions. About direction and priorities.

  • People. The people have to be both capable and willing to do the work. This includes

    • Ability. Meaning talent, the latent part of ability, and skills, the trainable part.

    • Motivation. The willingness of the people to participate in both social and technical sides of the endeavor.

  • Incremental development with reflection. Without incremental development, the team doesn't have a chance to detect and correct problems in its direction and process; without reflection, the team won't actually redirect itself. These are

    • Incremental growth and integration of the product, preferably with deliveries.

    • Reflection after each increment about their direction (including target market and feature sets), their rate of movement, and their process.

  • Community. The quality of the community affects the speed of movement of ideas; it is one place where dynamic shifts in team productivity can occur. Community includes

    • Communication paths. The speed of the project is limited by the speed at which information moves between people (erg-seconds/meme, see p.109).

    • Trust issues (personal safety and amicability). If people won't speak or won't listen, the information won't get from where it is to where it needs to be.

  • Focus. Meaning both direction and time.

    • Priorities. Knowing the two or three key items to work on at any time.

    • Focus time. Having quiet time to get the work done.

Figure 5.1-3 shows role of the project manager.

Figure 5.1-3. Role of the project manager in modern agile projects.


If the team does not make its results visible to the sponsors, the sponsors will cut the supporting flow of money and decisions. Indeed, when I look through stories of failed agile projects, this is a dominant pattern that emerges: The project team dismissed the project manager and did not create an adequate line of visibility to the sponsors, and the support got cut off a short time later. This visibility-providing role need not be in the hands of the project managerit just usually is because that is part of the normal job description for a project manager (PM). If an agile team wishes to work without a PM, the team must find another way to get the visibility and support they need.

The PM also serves the team by acting as a wall against interruptions. Indeed, from the time I was a junior hardware designer in the 1970s through today, I have heard developers say, "The only thing a project manager does for me is to keep interruptions away!"

The largest part of a PM's job is "getting people to work together who would otherwise be sitting in the cafeteria arguing."[10] This means working on communication, amicability, personal safety, and motivation.

[10] Thanks to Carl Ellison, who spoke these mysterious words to me around 1978. They have been rattling around in my head for decades, waiting for me to make sense of them!

Finally, it is the project manager who helps people identify and get the skills development they need; who helps them clear time for focusing on work; who makes sure everyone knows their priorities and ensures that the priorities are aligned within the team and with the organization's; who organizes and may lead the reflection sessions.

Even after we drop the bureaucratic load of the PM to nil, there is still plenty for a good project manager to do. If you choose to eliminate the role of "project manager," make sure that all these critical factors are taken care of.

"Agile development is low in discipline"

Notwithstanding the discussion in the first edition of this book ("Countering with Discipline and Tolerance," p. 77, and "Dissecting XP," p. 201), many people have formed the impression that agile development is low on discipline.

The best agile teams are extremely disciplined, writing test cases before they write new code, checking in and integrating running-tested code every few hours, refactoring their code to simplify it, showing their results to users every week, and reflecting on their work habits every week or every month. I find these practices difficult to introduce in most organizations. I am always surprised at how many agile groups manage to do them all, enjoy doing them, and insist on doing them.

Agility and Discipline

The idea that agility is not disciplined comes from people working with a one-dimensional comparison line (see Figure 5.1-4). This view causes trouble because people are forced to place agile development somewhere between cowboy coding and plan-driven development.

Figure 5.1-4. The old view of a one-dimensional spectrum of discipline between cowboy coding and plan-driven.


But it doesn't sit between those two choices at all. Another dimension is in playnamely, how much flux or change is happening with the requirements and the code (see Figure 5.1-5).

Figure 5.1-5. A more accurate view shows discipline and flux as separate dimensions. Agile development occupies the high-flux zone and permits a varying amount of discipline.


Cowboy coding (what I sometimes call "lost in the woods" development) involves changing direction a lot, as well as lacking discipline. The changes in direction may or may not be useful to the company. Plan-driven development intends to slow down the changes by adding a certain level of discipline.

Agile development permits greater changes in requirements and design by adding a certain level of discipline. There are numerous agile methodologies, some of which call for more discipline than others. XP probably calls for the greatest amount of discipline among the named agile methodologies (see "Dissecting XP," p. 201). Crystal and Scrum are fairly deliberate about lowering the requirements for discipline as low as possiblebut both still require more discipline than cowboy coding. Judicious use of agile principles and techniques lets the team respond faster to changing requirements and design than cowboy coding does.

Thus, agility does not go against discipline but is an independent selection. One management book captures the importance of having both: In Big Winners, Big Losers, Alfred Marcus (2005) reports on his study of companies that stayed successful across leadership changes. He found that they all had three characteristics:

  • Agility

  • Discipline

  • Focus

Focus, which provides knowledge of direction and the ability to not work on things not related to that direction, simplifies people's choices. Agility, which provides the ability to change directions quickly, drives effective team work habits. Discipline, which provides the capacity to do the unpleasant, keeps the team doing what works.

Agility, discipline, and focus are as essential to an effective software team as to the company as a whole.

Tolerance for Uncertainty

This is the appropriate time to offer a graph that Jim Highsmith and I drew up in one of our early meetings but that we haven't published up to now (Figure 5.1-6). It relates to the organizational tolerance of flux.

Figure 5.1-6. Some groups cannot use highly productive strategies because those strategies require more tolerance for ambiguity and uncertainty than the groups can manage.


The most productive strategies operate with a high level of flux, ambiguity, and uncertainty. Incremental development, iterative development, and concurrent development are three strategies in this line. Anyone who has lived through a concurrent-engineering project understands the higher levels of flux, ambiguity, and uncertainty it generates.

The horizontal axis shows the level of ambiguity and uncertainty the people in the organization can tolerate. If they can tolerate more, they can apply more productive concurrent strategies. If they can tolerate less, they are restricted to simpler, less productive strategies (waterfall, for example).

Each project requires a certain level of productivity to complete in time. If the project requires a lower level of productivity, then a team with a lower tolerance for ambiguity and uncertainty can still complete the project with their simpler strategies. However, if the project requires the highest levels of productivity, the same team will fail because the people are unable or unwilling to use the more productive strategies (that call for the extra ambiguity and uncertainty).

This is what the Boehm-Turner diagram indicates with the Culture axis (see Figure 5.1-7, in the next section). Some organizations need order, or the fiction of it. Those organizations are less suited to trying out agile strategies because the agile strategies will bring ambiguity and uncertainty with them. Organizations with higher tolerance will have an easier time with the agile approach.

Figure 5.1-7. Starfish diagram showing the shifting home territories of agile and plan-driven development (from Boehm 2003).


Figure 5.1-6 makes one more very important point. Some projects require the more advanced, more productive, uncertainty-generating strategies in order to succeed. Others don't. If your organization's projects don't require that extra level of productivity, then your organization doesn't need the added stress caused by agile methods. If, on the other hand, your project needs that level of productivity, but your organization can't tolerate the needed level of ambiguity and uncertainty, then you're in trouble.

I think that this graph hints at the limits of adoption of the agile approach.

"So what should we do?"

Do not choose between agility and discipline. You can have either, or neither, or both.

"Agile only works with the best developers"

Many writers have offered a peculiar objection to the agile approach. They observe that very bright designers articulated the principles and then jump from that observation to the assertion that only an all-expert team can succeed with the agile approach.

This objection is peculiar because the argument is so riddled with flaws that I am surprised to find it being offered at all, let alone by people I respect. I include here a discussion of this peculiar objection only because so many people cite it.

The manifesto's authors were mostly consultants at the time of its writing. As consultants, weand here I certainly speak for myselfwere never given the opportunity to work with "all expert" teams. Those are not the teams that call us for help. It is the non-expert teams who call in the consultants. Therefore, we were writing from experience with teams with a fairly normal distribution of expertise: maybe a good person or two, some young novices, some mediocre people. The project teams I interviewed in the early 1990s, from whom I learned the agile ideas, had those standard skill distributions. That skill mix did not change when I started consulting. The agile approaches were learned from, debugged on, and practiced with teams having the usual team skill mix.

Besides the previous practical objection, there is an interesting logical flaw in the argument. It uses a rhetorical cognitive illusion[11] to fool an audience into believing a false result. The cognitive illusion is that we are quickly drawn to believe the inverse of a proposition when only the contrapositive logically follows.

[11] See Hevitable Illusions (Piattelli-Palmarinia 1996), which talks about other cognitive (as opposed to optical)

Here is a quick refresher. Suppose that when Jim eats fish, he gets sick.

  • To form the inverse (which won't be true), negate both parts and leave them in the same order: "If Jim didn't eat fish last week, then Jim didn't get sick last week." On the surface, this looks reasonable, but with a bit of thinking, we realize that Jim could have gotten sick from some other cause.

  • To form the contrapositive (which will be true), negate both pieces and reverse their order: "If Jim was not sick last week, then Jim didn't eat fish last week." This is true, as the following table summarizes.

Proposition

AB

"If A, then B"

Inverse

~A~B

Doesn't follow

Contrapositive

~B~A

Follows


The cognitive illusion is that we find ourselves drawn to the unsound, simpler inverse rather than to the more complicated but sound contrapositive (the length of the word contrapositive doesn't help).

With that refresher, let's take a look at an assertion that I have seen at the top of more than one speaker's slides: "A problem with agile processes is that they require competent and experienced people."

  • Proposition: "If I'm using an agile process, then I need some competent and experienced people."

The simple, unsound inverse is:

  • Inverse: "If I am not using an agile process, I don't need any competent and experienced people on my project."

The implication is that if we use a non-agile process, we can avoid using any competent and experienced people.

This would be an amazing result if it were true! However, if we look at project histories, one of the simplest observations is that projects of all shapes and sizes rely on at least a few competent and experienced people to set it up and pull it through.

In reality, no matter what process we use, we are going to need at least a few competent and experienced people. So, the inverse proposition doesn't make sense.

Now let's look at the contrapositive:

  • Contrapositive: "If I don't have a few competent and experienced people, I can't use agile processes."

Or anything else, for that matter! Having a certain number of competent and experienced people on a project is simply a critical success factor, no matter what the process type. This contrapositive is true but almost meaningless! And therefore so is the sentence, "To use an agile process, I need some competent and experienced people." True, but basically without meaning.

If I had a project consisting only of novices, I would put my money on their doing better with an agile rather than a plan-driven process. I can't imagine novices coming up with a meaningful plan from the beginning, or delivering a successful project by sticking to that plan, or updating the plan every time they found a mistake in it. An agile process would call for them to work together, integrate frequently, test their code, show their results to their sponsors and users, and update their plans and working practices accordingly. With this approach they would have a better chance to learn and improve within the project.

Barry Boehm and Rich Turner came up with an interesting variation of the objection. They wrote, "You need a higher ratio of competent and experienced people on an agile project than on a plan-driven or paper-centric projects."

This statement is true, as far as I can tell, through an interesting and sort of backhanded line of reasoning that also appears to be true. It goes like this:

On an agile project, there is very little extra paperwork to produce. Most of the time is spent writing code or writing tests. Poor performers stand out in this climate.[12] The project can get away with fewer people because there is so little extra paperwork to produce.

[12] One of the objections to the agile approach in some organizations is exactly that the low performers get revealed so quickly. If those people have political clout, this can be enough for the organization to reject an agile process. As humorist Dave Barry writes, "I am not making this up."

On the other hand, there is a lot of paperwork to produce in the average plan-driven process. It is a waste to have top programming talent doing paperwork, so it makes sense to hire less-qualified people to do it.

Referring to them as level-1 (for beginners) and level-3 (for competent and experienced) people, we see the following.

  • On an agile project, it is sensible to have a lower ratio of beginners to experienced people. I have seen ratios from 3:1 to 5:1 work well.

  • On a plan-driven project, it is advantageous to have more beginners around, so perhaps between 5:1 and 8:1 could make sense.

Barry Boehm and Rich Turner capture this line of thinking in their book, Balancing Agility with Discipline[13] (Boehm 2003), recaptured in Figure 5.1-7, in which they show a starfish diagram with five different characteristics of a project situation:

[13] It is an odd title. Agility and discipline are not opposites to be traded against each othersee "Agile development is low in discipline" on page 254. Within the book, they shift to the term plan-driven, which is better but still not my preferencesee "Predictable, Plan-driven, and Other Centerings" on page 266 (later in this chapter). Those terms aside, they give an outstanding presentation of the tension between planning and moving, and the need for use of the large middle ground.

  • The number of people being coordinated

  • The criticality of the project

  • The ratio of level-1 to level-3 people

  • The rate of requirements change

  • The tolerance of the organization to change (also refer to Figure 5.1-6)

This starfish diagram looks correct to me, and I commend it to others to study.

On the question of the ratio between level-1 and level-3 people, we might note an implicit assumption in that axis of the diagram: that the organization is obliged to use all those level-1 people. That is, someone obliges the project managers to find ways to keep all those people busy. If that were not the case, then I would suggest either of two strategies:

  • Get rid of the extra level-1 people and use an agile process with the reduced team size.

  • Use an agile process with the smallest number of people possible to get the job done; let the rest of the people do anything they want just so long as they don't interfere with the progress of the development team.

Odd though this latter strategy sounds, I have met with two project teams that did just that, and both recommended the strategy highly. They said that payroll costs were fixed with respect to the number of people being paid, but their speed was higher without the level-1 people slowing them down. This unusual strategy illustrates good sense in playing the cooperative game, though perhaps a perverse attitude toward the organization's personnel department.

"So what should we do?"

Get some competent, experienced people. Work with agility and focus in all cases. Improve the team's speed by adding disciplined agile practices as the team can or is willing.

"Agile is new, old, failed, and untried"

It is common to object to an idea by saying that it is "not new"meaning that it has been tried and failedand simultaneously that it is newmeaning that we don't know if it works. When I hear these objections, I think that Tom Waits has the only correct answer:

"It's new, it's improved, it's old-fashioned."

Tom Waits, "Step Right Up"

Every idea in the agile canon has been traced as far back as the 1960s and even the 1950s.[14] We find incremental development, collocation, pair programming, frequent integration, test-first development, customer interaction, and so on.

[14] See, for example, Craig Larman's Agile and Iterative Development (Larman 2003), and Fred Brooks' description of "growing software" in his "No Silver Bullet" article (Brooks 1995).

As far as I can tell, good developers have long been using these ideas but weren't articulate about what they were doing. I once read (and sadly, have lost the reference for) a personal account in which the writer said that he was so struck by the silliness of the waterfall style-methodologies being published in the 1970s that he didn't see the need to respond. He assumed they would fall into oblivion from their own weight and was shocked when people continued to reference them seriously.

My theory is that agile approaches have always been used but did not show a market advantage until the 1990s, when business and technology were changing so fast that only the agile practitioners were delivering systems in time to the market. Certainly that was the case in the early 1990s, when I was just starting to debrief projects for the IBM Consulting Group in order to construct a methodology for object-technology projects. In those initial interviews, there was a very clear dividing line between those project teams who were delivering projects and those who weren't. Those who were delivering were following the values and principles expressed almost ten years later in the agile manifesto.

A better way to think of the agile manifesto is that the manifesto authors "cherry picked" a very small set of values to pay close attention to out of hundreds of possibly important things. We were asserting, in essence, that these few items outweighed the rest.

It is natural that the items we chose had been around for a long time. What was new was that we bothered to name a particular and critical combination, and there were certain new wrinkles on the old practices to offer.

Agile is new, it's improved, it's old-fashioned.



Agile Software Development. The Cooperative Game
Agile Software Development: The Cooperative Game (2nd Edition)
ISBN: 0321482751
EAN: 2147483647
Year: 2004
Pages: 126

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net