Results from the Experience

Ok, now we get to the good stuff. The team of developers and analysts consists of approximately 35 developers, eight business analysts, and seven QA testers. The developers rely on the analysts to be the customers of the project. There is a real customer, however, and the analysts work with them to collectively make customer decisions. Tables 33.1 and 33.2 show the elements of XP as discussed in [Beck2000] and briefly describe how each aspect was being used at different stages in the project. We use these tables to analyze our team's natural selection of practices and explain how textbook XP evolved on a large project to support a 50-person team that has taken a project to completion.

Table 33.1. Use of XP Practices during the Project (First Six Practices)

	Planning	Release Cycle	Metaphor	Design Simplicity	Testing	Refactoring
1/2000	Large iteration planning meetings. Full team of developers and analysts met for an entire day to discuss new story cards and estimate. Most developers sign up for new functionality.	One month	None	Transitioning with an existing code base. Existing code still complex, but new code as simple as possible. This phase included throwing out and rewriting existing functionality that "someday might be used."	Unit tests started with some of the new code. Push being made to have a large test base. QA does all functional tests and has the authority to pass/fail a story card.	Refactoring done as needed when touching old code.
7/2000	Same as 1/2000, but definitely feels very inefficient. Most attendees not concentrating or participating. Long, drawn-out discussions. Fifty-person meeting is overwhelming.	One month	None	Designs continuing to be as simple as possible. Refactorings of existing designs being done. Code reviews being done to collectively discuss new designs so that the whole team is familiar with coding/design trends.	Better unit test coverage but still not full coverage. Coded Functional Tests (CFTs) to help test coverage.	Most developers are heads-down in delivering functionality, very little refactoring done. Code base getting worse at end-of-iteration crunch to deliver cards to QA.
1/2001	Try making iteration planning meetings faster by doing more prep work with small groups of developers before the actual meetings.	Two weeks	None	Most of the designs built on existing designs standards kept, code reviews tapered off because new types of designs/refactorings not being done.	Attempted to get rid of CFTs and replace with a screen scraper and failed went back to CFTs. New unit tests being added, but coverage still lacking. QA starts to automate functional tests with screen scraper.	Refactorings being done much more often as code starts to spaghetti in some parts of the app. Major clean-ups being done because implemented simply, and then iteration deadlines caused code to grow without the important refactorings being done.
6/2001	Several meetings for cards/sets of related cards between developers interested in or knowledgeable about the functionality and analysts who are responsible for the story card.	Two weeks	None	Large part of the team and all of QA working on delivering version 1.0 to the customer. Code base split and new functionality being added without QA. More reliance on unit tests.	Test coverage seems to have stabilized at a level that is lacking. QA has been out of the loop with new functionality because of focusing entirely on version 1.0 delivery.	Refactorings on the release are minimal, while on the continuing product branch developers are refactoring more conscientiously after being forced to do larger, more painful refactorings earlier in the year.
	Pair Programming	Collective Ownership	Continuous Integration	40-Hour Week	On-site Customer	Coding Standards
1/2000	Because we decided to use XP, the entire team read [Beck2000] and were encouraged to pair program. Everyone tried it, and it caught on with most of the team.	In the initial phase, we were moving from function-oriented groups, so we did not feel collective ownership, but also did not feel protective of our code.	Online from iteration 1. See [Fowler+2000].	This was a proof-of-concept work, and we wanted the client on-board. So a lot of extra hours were put in to meet the deadline.	Business analysts are the on-site customer. Fifteen analysts on the team. The actual client was off-site, and the analysts communicated with them.	None, other than normal Java syntax.
7/2000	Pair programming still strong; developers are pair programming for new functionality, but bugs and maintenance have developers working solo. Some programmers stop pair programming altogether.	Ownership of code is completely diluted as more developers touch different pieces of the app. Very good communication takes place via informal chats, code reviews, and short stand-up meetings.	CFTs also added to the build process.	Cyclic hours hitting 50- and 60-hour weeks at the end of each iteration to try to pass the story card.	Same as above.	Biweekly code reviews give developers a chance to discuss ways that different subsystems are being implemented where we agree on informal ways of coding similar parts of the system.
1/2001	Pair programming definitely less because more of the coding is straightforward or because a developer is refactoring a piece of code.	Stand-up meetings are dropped for being inefficient, but code ownership remains diluted. Developers start specializing in parts of the system again.	Stable same as above.	With two-week iterations, developer estimates become more accurate and closer to a 40-hour week.	Same as above.	Code reviews tapered off; saturation of design/coding standards.
6/2001	Definite trend of almost all new functionality being pair programmed and almost all debugging/maintenance being done by a single developer.	With specialization, there is a trend of a small set of developers knowing more about different parts of the code so we tend to have them be more active in the continuing designs, but still have communal ownership of the code.	Stable same as above.	Same as above.	Same as above.	Same as above.

After examining these tables, we notice that almost all the practices have evolved over 18 months. Some have stabilized, because we see many "same as above" entries in columns such as Continuous Integration and On-site Customer. So, after 18 months on a 50-person team, what are our recommendations and lessons learned?

1. Have an iteration planning meeting at the beginning of each iteration, where the customer and developers split into groups all day to discuss the latest story cards and estimate them. At the end of the day, regroup to present your estimates and findings, and then have developer sign-up. This will keep the whole team in the know about what is happening without burdening everyone with an extremely tedious and long meeting.

2. Keep releases as small as possible two weeks works for us but at the same time be flexible when larger pieces need to be done over several iterations. Allow sign-up for a multiple iteration card, but always review progress at every iteration.

3. Write as many unit tests as you can that is self-evident. You should also have an automated suite of functional tests to keep the test coverage at an acceptable rate. A QA team cannot be replaced, no matter how many tests developers write; we are flawed in our biased understanding of how the system works or should work.

4. Simple designs have helped us consistently release a working product to the customer. Frequent design meetings (lunch is the best way to gather the development team) are very helpful during stages of adding intense new functionality. This helps prevent implementing parallel and maybe incompatible solutions in different parts of the application.

5. Refactoring is the only way to be able to have simple designs as stated in item 4. Refactoring of designs is just as important as refactoring of the code. It will always be tempting to not refactor and to just patch a solution, but if it is patched too much, the team will be forced to make major refactorings later on.

6. Pair programming should be religiously followed when new functionality is added and should be skipped when fixing bugs or doing repetitive tasks that have already been "solved" before by a pair of developers.

7. Collective ownership goes hand in hand with communication. The team must figure out a way to communicate effectively. It may be no more than just informal discussions, which worked best for our team; otherwise, regular stand-up meetings that last for ten to 15 minutes are a good way to disseminate information.

8. With a large project, a group of individuals are needed to be the customer, to generate enough work for the large number of developers. This strictly depends on the where the business knowledge is.

9. Coding standards have been very informal, and this has not been detrimental to our progress. What is more important is communication of ongoing work through presentations. Code is not documentation enough developers need to see the big picture also, and that cannot be relayed through code.

We also found that the following things didn't work for us.

1. Biweekly stand-up meetings were not efficient. We opted for informal communication with once-a-month iteration team meetings.

2. Full team meetings during the iteration planning meeting did not work. A day of small group meetings, with a 30- to 45-minute review of the cards at the end of the day, worked better.

3. One-month iterations were too long and were detrimental to code quality. We moved to two-week iterations, which are easier to track and make estimations more accurate.

4. The approach in item 3 does not work all the time for larger chunks of code, especially when refactoring a large part of the system. So exceptions are made when one card spans several iterations.

5. Metaphors are unrealistic in large projects. They are just too complex. Period.

6. A 40-hour week has never been an issue for us. Forty hours is the minimum, and we have not been adversely affected by working more than 40 hours, but then again, we are not pair programming 100% of the time.

What about the hypotheses? Where do we stand after completing the project that we set out to recover two years ago by going to XP? How does reality compare with our initial expectations? We initially listed five major points that we hoped to gain by using XP, and here is where we stand today looking back.

1. We received tremendous buy-in from our customer, which resulted from our involving the client very heavily in decision making throughout the development cycle.

2. "Managing expectations" was much easier because of the incremental releases. When things fell behind, we didn't just say, "Oh, we are six months behind." But as functionality slipped at the end of each iteration, the client knew and was part of the planning to adjust our plan to reality.

3. To be honest, the quality of the product is better but definitely not as good as we hoped. The main fault here is that we were not rigorous about adding tests.

4. Integration problems are not an issue anymore. Collective ownership has given developers the knowledge and ability to work on all the code, and small iterations have always kept the product in a condition to be delivered to the client.

5. We have successfully delivered two separate releases of the application as of February 2002. The application is live and in production, being used for the daily business operations of our client. This, above all, is what we gained from XP. Without it, we would have never been able to sustain the momentum for such a large and complex application while at the same time remaining flexible enough to give the client what they really needed.

That's it. XP, or our evolved version of it, has done wonders for us as a team. We have already delivered our product to our client, where it is being used to manage their day-to-day business successfully. The users find real value in the product because, in the end, we have delivered something that they really want and need, not just what they thought they needed a little over two years ago.