Case Study: The Server Tools Project (Using a Defanged, Much Less Extreme but Still Very Agile Process)

In this section, Matt describes his latest project, which has many similar characteristics to an XP project, albeit with some key differences (detailed up-front design being one of them). As it turns out, it s very close to the refactored process that we re aiming for.

Brief Overview

The details of the project itself aren t important for this discussion, except for one or two items that help to explain why the process has been tailored the way it has. So a brief overview is probably useful.

The company is a technology start-up based in central London, with a server product that controls business processes via aWeb-based front-end. The front-end is constructed dynamically from various back-end Web servers. So the server product is like a cross between an Enterprise Application Integration (EAI) server and a Web screen-scraper (although that does understate its role somewhat ”the server contains a lot of very neat technology, written by some very talented people).

Initially, the server was being configured in-house by the engineers , via a combination of XML files and command-line tools. The next stage (and with a pressing need to get something released as soon as possible) was to produce some GUI tools for configuring the server and defining its business processes. The project I m heading up is tasked with the job of creating these GUI tools.

The first tool was released very quickly (first a prototype, followed quickly by the real thing). It was basically just a GUI front-end on the most-used command-line tool being used by the engineers, with added functionality taking advantage of the graphical environment. This tool was essentially a stopgap that allowed the engineers to get their job done more quickly while they waited for the real high-level GUI tools to arrive .

The temptation was, of course, to rush out these high-level tools and then tweak them gradually based on the users feedback. Luckily, we didn t need to rush to do a quick release because we had provided the stopgap tool straightaway. This gave us a breathing space to concentrate on getting the main GUI tools right, rather than launch the project straight into one long maintenance phase.

Given the intense focus of the project on user interfaces, much of our analysis-work has involved use cases, user experience walk-throughs , UI workshops with some of the prospective users, and so on. We also have on-site HCI experts who play an important role in shaping the product s interaction design.

The team size is currently five programmers plus a tester, a dedicated interaction designer, and various analysts, product managers, and consultants who together form the on-site customer role ”so about 12 to 15 people in total. This is quite a small size, but it is roughly the number that might be considered the top end for an XP project. Even at this size of project, we found it necessary to tailor many of the XP practices (as described in the next section).

As the project ramps up, the plan is to bring in more programmers and produce different tools in parallel. The insular nature of each tool means that each one can be treated as a miniproject (although they re all kept integral and consistent via a documented architectural framework and user experience spec).

Would XP Have Sufficed?

Many of the practices we follow as a team are pretty much the same as XP: unit testing; test-first design; pair programming; short, fixed iterations; small releases; and on-site customer team. The planning process is also quite similar: a detailed plan and task list for the next iteration and a broader plan for the long term .

There are, however, vital (and quite deliberate ) differences:

• The role of XP coach is filled by the team leader. The main difference being that the team leader has greater authority (the team leader also performs some project management duties ).

Also see the section Technical Team Leader earlier in this chapter and the XP Coach vs. Team Leader sidebar in Chapter 14.

 

• Interaction design is a first-class citizen. That is, we involve usability experts who don t themselves program (because the goals of interaction design and program design frequently conflict). There s also a full-time interaction designer on the team.

Also see the section Interaction Designer earlier in this chapter.

 

• Pair programming is encouraged for solving difficult problems, but it isn t mandatory. Each programmer gets to have a say in the way he works. (Also, some of the programmers like to work from home on occasion, so this must be taken into account.)

• We do frequent integration (as opposed to continuous integration) once per day. Toward the end of each day there is a build ”all code is checked in, rebuilt, and unit- tested ”and any integration issues are resolved then and there. This allows us to catch code divergence problems early enough to prevent them becoming a problem, without the added burden of continuous integration.

Having said that, continuous integration can work well if it s sufficiently automated. We recently set up a low-spec PC to be a dedicated build machine. Once an hour , it rebuilds the system and runs the unit tests. If a test fails, the person responsible is e-mailed automatically. QA loves this setup because the QA team s functional tests get run on the same PC, including automated GUI click tests. ^[17]

As the project really gets under way and increases overall velocity (as more people are now being added to the team), we re finding this level of automation to be a real bonus. That said, the team still keeps its routine of making sure all code is checked in and working at the end of the day. It provides an amnesty period to get everything integrated before people clock off for the day (or to make a note that someone is midtask and needs to integrate as soon as possible the following day).

• Our iterations each last for exactly 1 month (a la Scrum ^[18] ) rather than 1 or 2 weeks. There is always a release (internal, at least) at the end of each monthly iteration. We are finding that this cycle is really helping to get everyone focused. The creation of these minideadlines creates a reasonable sense of urgency and encourages everyone to create that last push toward the end of the month to get the release ready.

Also see the section Short Iterations and Small Releases earlier in this chapter.

 

• Test-first design complements our design process but doesn t drive it. We have found that this approach works very well for us. The test-first approach (that is, writing the unit tests before you write the code, and driving the design from this) operates very much at a micro level ( designing in the small ) ”designing for one test case at a time.

Also see the section Test-First Design Complements Up-Front Design earlier in this chapter.

 

• At the core of the project there s a designed framework. Into this framework, we plug the tools that we re creating (e.g., to share data). The framework was designed up front but is being developed in parallel with the tools themselves. For more about this approach, see the next section, Framework.

• We use a combination of requirements and use cases instead of user stories. A lot of the requirements actually read like user stories, but we have high-level requirements clearly delineated from behavioral requirements. This helps management to more effectively prioritize what it wants delivered each month.

For a comparison of use cases, user stories, and traditional requirements, see Chapter 10.

 

• The requirements are stored electronically . But they can be conveniently accessed by any of us via the project intranet.

• Project velocity is measured in terms of the formal requirements completed per iteration, rather than user stories per iteration. This is a subtle difference, but it means that when comparing the time taken to complete different requirements, we re more likely to be comparing like with like.

We round off this case study with descriptions of a couple of aspects of the project that differ significantly from XP and that warrant discussing in more detail.

Framework

Although XP isn t specifically against frameworks, it s philosophically against designing and building a framework up front. We were in the lucky position of not having to deliver something immediately, because we had given the engineers a stopgap tool in the meantime. This in turn meant that we could afford to spend some time getting the underlying framework right ”paving the way. . . .

For more about applying the stopgap method, see the section titled (fittingly enough) The Stopgap earlier in this chapter.

 

To design our framework, we began with a quick-and-dirty prototype (including some prototype GUI-based tools) to get a feel for what we needed. Then we defined an architecture suitable for plug-in tools, shared data and services, and so on; documented it; and got as many people as possible to read through it, see the prototype, and give their feedback.

While this was going on, some necessary data analysis work was taking place to help define the server configuration data that the tools would be creating. Through a combination of luck and planning, these two activities finished at the same time. We then began to develop the framework in parallel with the first set of tools.

Ironically, we ve found that taking the time to think through, prototype, and create a robust framework has made the project much more agile overall. One reason for this is that the framework is just plain right for what we need. It s also well designed (having been through a lot of iterative feedback from senior and junior engineers alike), hence it s easy to modify should we need to.

Another reason is that the source code for the tools themselves is generated automatically ^[19] from a very small set of definition files ^[20] and accompanying Java code templates. The generator itself was developed in-house and is really very simple for what it does: It just steps through the definition files and iteratively munges them with the Java templates, creating reams upon reams of lovely, well- formed Java Swing code (plus XML descriptors, manifests , and so on). The size ratio of generated code to definition files is roughly 100:1.

This code generation is probably the single most agile aspect of the project.-It means that if the server product changes spec in any way, we just update the definition files and crank out a new version of the tools ”within seconds!

Another way in which the code generator makes us more agile is that it wouldn t be a very tall order to switch frameworks if we needed to. Currently, our framework is based on the NetBeans platform, but (if market forces dictated) we could switch to, say, Eclipse, just by swapping in a different set of templates (e.g., to generate SWT code instead of Swing). The framework design also helps reduce the cost of change because it keeps the (code-generated) tools themselves nicely decoupled from the framework implementation.

More Than One Master

The people who drive the direction of our project are many and varied. Come to think of it, have you ever encountered a project where this wasn t the case?

The users, of course, have a big say in what goes into the product. Currently, the tools are being used by the team of engineers who produce the accompanying server product. Because they re in the same room, this gives us quick and valuable feedback whenever we release a new version. So essentially, the engineers are currently our on-site customer.

However, our customer is also various other people ”for example, the product management team located elsewhere in the building; the marketing team, who in many ways drive the direction of the product; and the team of consultants who will use the tools externally on customer sites.

So the majority of our customers are not colocated (at least, not in the same room). We do, however, meet regularly with them, send them screen shots, solicit their feedback, and so on. It s a good habit to get into.

There would be potential for chaos here if we followed the XP route of an on-site customer team drip-feeding us with requirement details. Instead, we need to recognize that we have many masters, each one trying to pull the product in a slightly different direction (and with good reason). The only way we could possibly reach a middle line that everyone agrees on would be to elicit requirements from everyone involved, write them down in detail, and get all the on-site customers to sign them off. So that s what we do. ^[21]

It could be viewed that this is the least agile ( certainly the least XP-like) of our practices, but quite frankly we don t care. All the customers involved are happy that they get to have a say in the direction of the project. (They would have their say anyway, but this way our process takes the fact into account.)

^[17] QA also has its own separate test environment, but running the tests regularly in Engineering means that not only are the functional tests testing our software early, but also the software is testing the functional tests early as well. This virtually eliminates the big bang delivery problem, which affects test scripts equally as much as it affects production software.

^[18] See http://www.controlchaos.com.

^[19] Matt Stephens, Automated Code Generation, http://www.softwarereality.com/programming/code_generation.jsp , May 6, 2002.

^[20] We re using Java properties files for this purpose because they re much simpler to work with than XML (not to mention easier to read and edit). If the project increases in complexity and demands more flexibility, we will change as needed.

^[21] XPers tend to view this practice with quaint amusement , and yet this was a primary factor in the early termination of the C3 project (as we discuss in the sidebar Did Oral Documentation Kill C3? in Chapter 7).