Agile

Agile implies being effective and maneuverable. An agile process is both light and sufficient. The lightness is a means of staying maneuverable. The sufficiency is a matter of staying in the game.

The question for using agile methodologies is not, "Can an agile methodology be used in this situation?" but "How can we remain agile in this situation?"

A 40-person team won't be as agile as a six-person colocated team. However, each team can maximize its use of the agile methodology principles and run as light and fast as circumstances allow. The 40-person team will use a heavier-agile methodology; the six-person team will use a lighter-agile one. Each team will focus on communications, community, frequent wins, and feedback.

If they are paying attention, they will reflect periodically about the fit of their methodology to their ecosystem and keep finding where the point "barely sufficient" has moved.

Sweet Spots

Part of getting to agile is identifying the sweet spots of effective software development and moving the project as close as possible to those sweet spots.

A team that can arrange to land on any of those sweet spots gets to take advantage of some extra-efficient mechanism. To the extent that the team can't arrange to land in a sweet spot, it must use less-efficient mechanisms. At that point, the team should think creatively to see how to get to the sweet spot and to deal with not being there.

Here are five sweet spots:

Two to Eight People in One Room

Information moves the fastest in this sweet spot.

The people ask each other questions without overly raising their voices. They are aware of when the other people are available to answer questions. They overhear relevant conversations without pausing in their work. They keep the design ideas and project plan on the board, in easy view.

People repeatedly tell me that although this environment can get noisy, their most effective projects have been those in which their small team sat in the same room.

When you leave this sweet spot, the cost of moving information increases very fast. Every doorway, corner, and elevator multiplies that cost.

The story "E-Presence and E-Awareness" on page 113 tells of one team not being able to land in this sweet spot. They used Web cams on their workstations to get some of the presence and awareness of sitting in the same room. They used chat boxes to get answers to the very many small questions that constantly arose. They were creative in mimicking the sweet spot in an otherwise "unsweet" situation.

Onsite Usage Experts

Having a usage expert available at all times means that the feedback time from imagined to evaluated solution is as short as possible, often just minutes to a few hours.

Such rapid feedback means that the development team gains a deeper understanding of the needs and habits of the users and makes fewer mistakes when coming up with new ideas. They try more ideas, which makes for a better final product. With a good sense of collaboration, the programmers will test the usage experts' ideas and offer counterproposals. This will sharpen the customers' own ideas for how the new system should look.

The cost of missing this sweet spot is a lowered probability of making a really usable product and a much higher cost for running all the experiments.

There are many alternative, if less effective, mechanisms you can use when you can't land on this sweet spot. They have been well documented over the years: weekly interview sessions with the users; ethnographic studies of the user community; surveys; friendly alpha-test groups. There are certainly others.

Missing this sweet spot does not excuse you from getting good user feedback. It just means you have to work harder for it.

One-Month Increments

There is no substitute for rapid feedback, both on the product and on the development process itself. Incremental development is perfect for providing feedback points. Short increments help ensure that both the requirements and the process itself are repaired quickly. The question is, "How long should the delivery increments be?"

The correct answer varies, but project teams I have interviewed vote for one to three months, with a possible reduction to two weeks and a possible extension to four months.

It seems that people are able to focus their efforts for about three months, but not much longer. People tell me that with a longer increment period, they tend to get distracted and lose intensity and drive. In addition, increments provide the team with built-in opportunities to repair its process. The longer the increment, the longer the time between such repair points.

If this were the only consideration, then the ideal increment period might be one week. However, there is a cost to deploying the product at the end of an increment.

I place the sweet spot at around one month but have seen successful use of two- or three-month increments.

If the team cannot deliver to an end user every few months, for some reason, it should prepare a fully built increment in that period and get it ready for delivery (pretending, if necessary, that the sponsor will suddenly demand its delivery). The point of working this way is to exercise every part of the development process and to improve all parts of the process every few months.

Fully Automated Regression Tests

Fully automated regression tests (unit or functional tests, or both) offer two advantages:

• The developers can revise the code and retest it at the push of a button. People who have such tests report that they freely replace and improve awkward modules, knowing that the tests will help keep them from introducing subtle bugs.

• People report that they relax better on the weekends when they have automated regression tests. They run the tests every Monday morning and discover if someone has changed their system without their knowledge.

In other words, automated regression tests improve both the system design quality and the programmers' quality of life.

There are some parts of the system (and some systems) that are difficult to create automated tests for.

One of those is the graphical user interface. Experienced developers know this and allocate special effort to minimize the portions of the system that are not amenable to automated regression tests.

When the system itself does not have automated regression tests, experienced programmers find ways to create automated tests for their own portion of the system.

Experienced Developers

In an ideal situationthe sweet spotthe team consists of only experienced developers. Teams like this that I know report much different, and better, results compared with the average, mixed team.

Because good, experienced developers may be two to ten times as effective as their colleagues, it is possible to shrink the number of developers drastically if the team consists entirely of experienced developers.

On project Winifred, we estimated before and after the project that six good Smalltalk programmers could develop the system in the needed time frame. Not being able to get six good Smalltalk programmers at that time, we used 24 programmers. The four experienced programmers built most of the difficult parts of the system and spent much of their time helping the inexperienced programmers.

If you can't land in this sweet spot, consider bringing in a half-time or full-time trainer or mentor to increase the abilities of the inexperienced people.

The Trouble with Virtual Teams

Virtual is a euphemism meaning not sitting together. With the current popularity of this word, project sponsors excuse themselves for imposing enormous communication barriers on their teams.

We have seen the damage caused to a project by having people sit apart. The speed of development is related to the time and energy cost per idea transfer, with large increases in transfer cost as the distance between people increases and large lost opportunity costs when some key question does not get asked. Splitting up the team is just asking for added project costs.

I categorize geographically distributed teams into three sorts, some of them more damaging than others. My terms for them are multisite, offshore, and distributed development.

Multisite Development

Multisite development occurs when a larger team works in relatively few locations, each location contains a complete development group developing a subsystem, and the subsystems are relatively well decoupled.

Multisite development has been performed successfully for decades.

The key in multisite development is to have full and competent teams in each location and to make sure that the leaders in each location meet often enough to share their vision and understanding.

Although many things can go wrong in multisite development, it has worked many times, and there are fairly standard rules about getting it to work (unlike the other two virtual team models).

Offshore Development

Offshore development is when "designers" in one location send specifications and tests to "programmers" in another location, usually in another country.

Because the offshore location lacks architects, designers, and testers, this is quite different from multisite development.

Here is how offshore development looks, using the words of cooperative games and convection currents:

The designers at the one site have to communicate their ideas over a thin communications channel to people who have a different vocabulary and sit several time zones away. The programmers need a thousand questions answered. When they find mistakes in the design, they have to do three expensive things: first, wait until the next phone or video meeting; second, convey their observations; and third, convince the designers of the possible mistake in the design. The cost in erg-seconds per meme is staggering, the delays enormous.

Testing Offshore Coding

One designer told me that his team had to specify the program to the level of writing the code itself and then had to write tests to make sure that the programmers had correctly implemented every line they had written. The designers did all the paperwork they considered unpleasant, without the reward of being able to do the programming.

In the time they spent specifying and testing, they could have written the code themselves, and they would have been able to discover their design mistakes much faster.

I have not been able to find methodologically successful offsite development projects. They fail the third test: The people I have interviewed have vowed not to do it again.

Fortunately, some offshore software houses are converting their projects into something more like multisite development, with architects, designers, programmers, and testers at the programming location. Although the communications line is still long and thin, the team members can at least gain some of the feedback and communication advantages of multi-site development.

Distributed Development

Distributed development is when a team is spread across relatively many locations with relatively few, and often only one or two, people per location.

Distributed development is becoming more commonplace, but it is not becoming more effective. The cost of transferring ideas is great, and the lost opportunity costs of undetected questions is even greater. The distributed development model works when it mimics multisite development, with meaningful subteams of one or two people. In this scenario, each person's assignment is clear and contained.

However, the following is more common:

Criss-Crossed Distribution

A company was developing four related products in four locations, each product having multiple subsystems.

A sweet spot would be to have all systems of one product developed at the same location, or one subsystem for all products. With either of these, the people would be physically proximate to the people they needed to exchange information with.

Instead, the dozens of people involved were arranged so that people working in the same city worked on different subsystems of different products. They were surrounded by people whose work had little to do with theirs and were separated from those with whom they needed to communicate!

Occasionally, people tell of developing software effectively with someone at a different location. This indicates that there is something new to discover: What permits these people to communicate so well over such a thin communications line? Is it just a lucky alignment of their personalities or thinking styles? Have they constructed a small multisite model? Or, are they drawing on something that most of us haven't learned to name yet?

Successful Distributed Development

I spent an evening talking with a couple of people who were successfully using four or five people who never met as a group.

They said that besides partitioning the problem carefully, they spent a lot of time on the phone, calling each person multiple times each day.

In addition to those obvious tactics, the team coordinator worked particularly hard to keep trust and amicability levels very high. She visited each developer every few weeks and made sure that they found her visits helpful (not blame sessions).

This coordinator was interested in replicating the successful development model.

We concluded, by the end of the evening, that she would need to find another development coordinator like herselfsomeone with a similar personal talent for developing trust and amicability.

Two aspects of their development struck me:

• Their attention to building trust among themselves

• The vast amount of energy they invested into communication on a daily basis, to achieve opportunistic learning, trust, and feedback

Open-Source Development

Open-source development, although similar in appearance to distributed development, differs in its philosophical, economic, and team structure models.

In contrast to the resource-constrained cooperative game most software development projects play, an open-source project is playing a non-resource-constrained cooperative game.

An industrial project aims to reach its goal in a given time frame with a given amount of money. The constraints of money and time limit how many people can work on it, and for how long. In these games we hear three phrases:

"Finish it before the market window closes!"

"Your job is to make the trade-off between quality and development time!"

"Ship it!"

An open-source development project, on the other hand, runs with the idea that with enough eyes, minds, fingers, and time, really good designs and really good-quality code will show up. There is, in principle, an unlimited number of people interested in contributing and no particular market window to hit. The project has its own life and existence. Each person improves the system where it is weak, at whatever rate that time and energy indicate.

The reward structure is also different, being based on intrinsicas opposed to externalrewards. (See "Individuals" on page 65.) People develop open-source code for pleasure, as a service to a community they care about, and for peer recognition. The motivational model is discussed at length in "Homesteading the Noosphere" (Raymond URL).

A goal for an industrial developer would be to become the next Bill Gates. The corresponding goal for an open-source developer would be to become the next Linus Torvalds.

Finally, the open-source team structure of open-source development is different. Anyone can contribute code, but a designated gatekeeper protects the center, the code base. That gatekeeper does not need to be the best programmer but needs to be a good programmer with good people skills and a very good eye for quality. Over time, the best few contributors come to occupy the center, becoming intellectual owners of the design. Around these few people is an unlimited number of people who contribute patches and code suggestions, detect and report bugs, and write documentation.

It has been suggested, and is plausible, that one of the key aspects of open-source development is that all communication is visible to anyone. Consider the following comparison with industrial projects:

On an industrial project with a colocated team, trouble occurs if the team evolves into a society with an upper and a lower class. If analysts sit on one side of the building and programmers sit on the opposite side, an "us-them" separation easily builds that causes hostility between the groups (you could also say "factions"). In a well-balanced team, however, there is only "us"; there is not an "us-them" separation. A key role in the presence or absence of this split is the nature of the background chitchat within the group. When the seating forms enclaves of common specialists, that background chitchat almost inevitably contains comments about "them."

In open-source development, the equivalent situation would be that one subgroup, the colocated one, is thought to be having a set of discussions that the others cannot participate in. The people in the distributed group might be inclined to develop a sense of being second-class citizens, cut away from the heart of the community and cut off from relevant and interesting conversations.

When all communication is online, visible to everyone, there is no natural place for rumors to grow in hiding; once again, there is only "us."

I would like one day to see or do a decent investigation of this aspect of open-source development.