Agile Methodologies

To give you a flavor of the variety and amount of activity going on in the agile world, in this section we briefly describe some of the most popular (or certainly the most talked about) of the agile methodologies:

• Extreme Programming (XP)

• Test-Driven Development (TDD)

• Agile Modeling (AM)

• Agile Database Techniques (AD)

• Adaptive Software Development (ASD)

• Crystal methodologies

• Feature-Driven Development (FDD)

• Dynamic Systems Development Method (DSDM)

• Scrum

Some processes are designed to complement other methodologies (such as RUP or XP) with additional practices, for example:

• AM adds high-level, people-oriented practices, plus guidance for the amount of up-front modeling that should take place, which models to maintain, and so on.

• Scrum adds agile project management and planning in an effort to control the chaos of conflicting interests and needs from which many projects suffer.

• ICONIX Process can be applied to agile projects, to help disambiguate the requirements and the design before coding begins. This can add a vital element of up-front analysis and design to processes that might otherwise be lacking in such areas.

Whatever such practices add, their primary goal is to reduce the level of risk involved in driving the project forward while the requirements take shape.

Let’s look at each of these agile processes in turn. You won’t need to know all (or indeed any) of these processes to be agile, but it helps to know what’s out there and the various directions that agile methods are taking.

Note 

To be agile, you also don’t have to follow all of the practices that we describe in this chapter. You should always tailor a process to suit your project and do just the practices that fit. In Chapter 4, after analyzing what it really means to be agile, we’ll present a core subset of agile practices. Of course, you should tailor even this core subset to suit your project.

Extreme Programming

Extreme Programming (XP) is the most popular (certainly the loudest) of the agile methodologies. For more information, see www.extremeprogramming.org or the book Extreme Programming Explained: Embrace Change by Kent Beck (Addison-Wesley, 2000). For a detailed analysis of how XP can be high risk without extensive tailoring, see our book, Extreme Programming Refactored: The Case Against XP.

In this section, we briefly describe the agile practices in XP and some of the thinking behind them.

Test-Driven Development

This practice involves designing the code by writing unit tests. Each test proves that a particular feature of the design either works or doesn’t work. You sometimes hear test-driven programmers shouting “Green bar!” or “Red bar!” when running the tests. This is because the popular xUnit family of test frameworks^[6.] shows either a green or a red bar to indicate whether the tests passed or failed.

In XP, the TDD practice also covers writing customer acceptance tests (often using an acceptance testing framework such as FitNesse) to define the requirements.

We cover TDD in more detail in Chapters 12.

The Planning Game

XP has two planning games: the release planning game and the iteration planning game. Planning in XP involves breaking the project into very short, 1- to 3-week iterations and taking the project one iteration at a time. For each iteration, the customer needs to decide which user stories to include and which to leave until a future iteration.

Any piece of work that takes time is written up as a user story and written on a story card. User stories must be testable and estimable. To aid this goal, stories are often broken down into estimable tasks. Each task is assigned an arbitrary value to indicate how long the programmers think it will take. Over time, these values (and the number of tasks completed) are measured to give an indication of the project velocity.

Whole Team

This practice involves trying to squeeze as many people as possible (including the customer or business analysts) into one room. That may sound like a recipe for a noisy, stinky room on a hot summer’s day, but the thinking behind it is quite pragmatic: the more barriers (e.g., walls) there are between team members, the more expensive it becomes to share information, and the less likely it is that team members will actually communicate. So putting all these people into one room increases not only the amount but also the quality of communication—at least that’s the theory.^[7.]

Small Releases

The aim here is to get feedback from the user as quickly as possible, primarily by releasing working software often. In XP, this may mean releasing as often as once every couple of weeks, though you would, of course, modify this time frame depending how feasible it would be to achieve on your project.

The thinking behind small releases is that valuable functionality is placed into the user’s hands as soon as possible and that the system never gets too far removed from what the customer really wants.

System Metaphor

The system metaphor is a simple shared story of how the system works. All naming and design theories stem from that one story. Lately, XPers have begun to adopt domain-driven design (i.e., driving the design from a domain model) as an important method of keeping the design consistent and sharing a common vocabulary. Domain-driven design is also a fundamental part of ICONIX Process (as we’ll see in Chapter 3).

Simple Design

An important lesson in all aspects of software design (including designing the user interface) is to keep it simple. This doesn’t mean removing all software layers so that your JSP pages are littered with SQL code, for example. In fact, sometimes adding a slightly more complex design can ultimately make the software much simpler to write. It’s a judgment call—but when in doubt, keep it simple.

Refactor Mercilessly

In any project, as time marches on, the design inevitably starts to fall apart. New requirements (which were never thought of when the software was first designed) need to be hacked in, so the design starts to become crufty (i.e., more fragile and increasingly difficult to extend). In XP, this common effect is countered by constant attention to the design: so-called merciless refactoring.

Refactoring is the practice of improving the design of existing code while still passing all of the unit tests. Improving the design generally means simplifying it—making your classes more cohesive and less tightly coupled. It’s a constant process, something that XP author Robert C. Martin describes as “continuous care.”^[8.]

Collective Ownership

With collective ownership, everyone is responsible for the entire code base. If a programmer sees something in the code that she doesn’t like, she (and her pair-programming buddy) can go in and change it, regardless of who originally wrote it. (XP’s safety nets here are comprehensive unit and acceptance test harnesses, and constant mandatory pair programming.)

Pair Programming

In XP you can code on your own for “nonproduction” code such as prototypes. However, for all production code (i.e., code that will be used on a live system by “real” end users), XP recommends that everyone on the team program in pairs. Pairing up is an effective way of catching errors and helping each other identify methods to further improve the design.

Continuous Integration

By integration, we mean putting your latest changes to the source code into a central source control system (e.g., PVCS, Visual SourceSafe, and CVS). Continuous integration means doing this as often as possible (at the very least once per day). This practice is often also supplemented by a dedicated “build” PC that regularly gets all the latest source code, builds it (in the Java world, typically using an Ant build script), and runs all the unit tests. If this is done often (e.g., once per hour), then build or test errors are caught very quickly, almost as soon as they are introduced.

In the nonagile world, many teams either forget to integrate their code for weeks at a time or hold back on doing it because they know that it’s a major headache. And it’s no surprise, really: if several programmers are working on different parts of the same system in parallel, they’re bound to diverge over time and will produce incompatible code. So, though it may seem counterintuitive, the answer to the integration issue is not to do it less, but to integrate your code more often.

Sustainable Pace

The more tired a programmer is, the more mistakes he’ll make. Many of us have experienced the horrors of late-night coding, up against some perilously important deadline or other. XP makes the point that on projects where this is the norm, programmers simply burn out and end up as near-catatonic dropouts propping up street corners and smelling heavily of stale beer. (Perhaps this is why XP talks about “coder smells”?^[9.])

Sustainable pace (which used to be called “40-hour week”) will result in higher productivity levels in the long run than making frequent mad dashes that quickly burn out your programmers.

Coding Standards

Coding standards are important in any programming project, and more so in an XP project because there’s no specialization: any programmer pair can dive in and work on any part of the code base. The thinking is that we spend five to ten times as much time reading code as we do writing code, so if we all write it to the same conventions, it should improve our productivity.

Test-Driven Development

Test-Driven Development (TDD) is an integral part of XP, but it can also be extracted in its entirety and applied to non-XP projects. For example, in Chapter 12 we show how TDD can be used in conjunction with ICONIX Process to good effect.

Using TDD, you design code by writing code. While this sounds circular, it means that you write client test code that serves as both an example and a definition of what you want the object code to do. For example, if you were writing a hotel booking system, you would write tests for placing a booking, adding customer details, retrieving the details, and so on, and then assert that the correct results are returned.

Agile Modeling

The Agile Modeling (AM) methodology^[10.] is a “chaordic”^[11.] collection of practices, guided by principles and values, that should be applied by software professionals on a day-to-day basis. AM is not a prescriptive process—in other words, it does not define detailed procedures for how to create a given type of model. Instead, it provides advice for how to be effective as a modeler. Think of AM as an art, not a science.

Overview of Agile Modeling

An agile modeler is anyone who models following the AM methodology, applying AM’s practices in accordance with its principles and values. An agile developer is someone who follows an agile approach to software development. An agile modeler is an agile developer. Not all agile developers are agile modelers.

AM has three goals:

• To define and show how to put into practice a collection of values, principles, and practices pertaining to effective, lightweight modeling. What makes AM a catalyst for improvement isn’t the modeling techniques themselves—such as use case models, robustness diagrams, sequence diagrams, or class diagrams—but how to apply them productively.

• To address the issue of how to apply modeling techniques on projects following agile software development methodologies. Sometimes it is significantly more productive for a developer to draw a sketch while thinking through an idea, or to compare several different approaches to solving a problem, than it is to simply start writing code. There is a danger in being too code-centric; a quick sketch can often avoid significant churn when you are coding.

• To address how you can improve your modeling activities following a “near-agile” approach to software development. In particular, this applies to project teams that have adopted an instantiation of the Unified Process such as the Rational Unified Process (RUP) or the Enterprise Unified Process (EUP). Sometimes it is more productive for developers to spend less time writing documentation and creating complex models, and more time creating models that are just barely good enough.

In short, AM strives to find the “sweet spot” between too little and too much modeling. Because this sweet spot changes from project to project, you will need to tailor your application of AM each time.

Agile Modeling Practices

AM consists of a variety of core principles, secondary principles, core practices, and secondary practices, not to mention values. You can find full descriptions of these at the AM website (www.agilemodeling.com). However, to give you an idea of what AM is about, here are the core practices:

• Active Stakeholder Participation

• Apply the Right Artifact(s)

• Collective Ownership

• Consider Testability

• Create Several Models in Parallel

• Create Simple Content

• Depict Models Simply

• Display Models Publicly

• Iterate to Another Artifact

• Model in Small Increments

• Model with Others

• Prove It with Code

• Use the Simplest Tools

The top 10 points to remember for AM are as follows:

• 10. You should create several models in parallel, iterating back and forth as required.

• 9. Models aren’t necessarily documents, and documents aren’t necessarily models.

• 8. Modeling is a lot like swimming: it’s very dangerous to do alone.

• 7. Sometimes the simplest modeling tool is a whiteboard; sometimes it’s a complex CASE tool.

• 6. Keep your models as simple as possible.

• 5. Travel light and update documentation only when it hurts.

• 4. Models don’t need to be perfect—they just need to be barely good enough.

• 3. Create the right models for the job, even if they’re not defined by the UML (e.g., data models).

• 2. Your project stakeholders must be actively involved in modeling.

• 1. The three secrets to successful modeling are feedback, feedback, feedback.

Agile Database Techniques

Agile Database Techniques (AD) applies refactoring and emergent design principles to database models. It correctly recognizes that as a project develops, it isn’t just the design of the code that changes—the database model also evolves. AD provides a number of techniques for evolving databases while keeping a close eye on the integrity of your data. It’s a difficult subject because we’re dealing with live customer data, but it’s an important one to keep in mind.

This methodology is described in the book Agile Database Techniques: Effective Strategies for the Agile Software Developer (John Wiley & Sons, 2003) by Scott W. Ambler (creator of AM). Also see www.agiledata.org.

Adaptive Software Development

Adaptive Software Development (ASD) consists of the following:

• Adaptive Conceptual Model: The theoretical foundation

• Adaptive Development Model: A software development life cycle for complex projects

• Adaptive Management Model: Principles for managing complex projects

ASD places an emphasis on adaptive development cycles (basically iterations). Adaptive cycles are mission driven, component based, iterative, time boxed, risk driven, and change tolerant. A project is also divided concentrically into versions, cycles, and builds.

This approach to software development is described in the book Adaptive Software Development: A Collaborative Approach to Managing Complex Systems by James A. Highsmith III (Dorset House Publishing Company, 2000).

Crystal Methodologies

According to Alistair Cockburn’s Crystal Methodologies site at http://alistair.cockburn.us/crystal/crystal.html

Crystal collects together a self-adapting family of “shrink-to-fit,” human-powered software development methodologies based on these understandings:

• Every project needs a slightly different set of policies and conventions, or methodology.

• The workings of the project are sensitive to people issues, and improve as the people issues improve, individuals get better, and their teamwork gets better.

• Better communications and frequent deliveries reduce the need for intermediate work products.

We’ll look at the Crystal methodologies and practices in more detail in the sections that follow.

Overview of the Crystal Methodologies

The Crystal methodologies are delineated according to factors such as project size, proximity of team members, and project risk. For example, Crystal Clear^[12.] is targeted at small projects, whereas Crystal Orange is targeted at projects with up to 40 people sitting in one building, working on a system that might cause loss of discretionary monies.

The Crystal methodologies were created by Alistair Cockburn, and are described in his books Agile Software Development (Addison-Wesley, 2001) and Surviving Object-Oriented Projects (Addison-Wesley, 1997). A key point about the methodologies in the Crystal family is that they are self-adapting—that is, you need to keep reviewing and tailoring the methodologies throughout the project.

You should find that there are many similarities (not least in the philosophy behind software development) between Cockburn’s work and James Highsmith’s ASD work. In fact, Highsmith and Cockburn have begun combining their efforts. See http://alistair.cockburn.us/crystal/crystal.html for the latest developments.

Crystal Practices

Crystal Clear is targeted at similar projects to XP (small projects with up to 12 people located in the same room), so we’ll describe it briefly here.

The key person in a Crystal Clear project is the senior designer-programmer. Other roles (which require separate people) are the sponsor, designer-programmer, and user (part-time at least).

The policy standards are that^[13.]

• Software is delivered incrementally and regularly, every 2 to 3 months.

• Progress is tracked by milestones consisting of software deliveries or major decisions, as opposed to written documents.

• There is some amount of automated regression testing of application functionality.

• There is direct user involvement.

• There are two user viewings per release.

• Downstream activities start as soon as upstream is stable enough to review.

• Product and methodology-tuning workshops are held at the start and middle of each increment.

Crystal Clear includes some mandatory work products, including a release sequence, a schedule of user viewings and deliveries, use cases or feature descriptions, design sketches and notes as needed, screen drafts, a common object model, running code, migration code, test cases, and a user manual. By contrast, Crystal Orange scales up to meet the challenge of larger projects. For example, it includes 14 roles over Crystal Clear’s 4, and more formal work products (such as a requirements document, a user interface (UI) design document, interteam specs, and status reports).

DSDM

Dynamic Systems Development Method (DSDM) has been around for at least a decade. It predated the agile movement and was originally marketed as a Rapid Application Development (RAD) methodology. However, DSDM is continuously evolving, so its definition today is still up to date with current software knowledge. DSDM differs from the other agile processes described here in that it is commercially driven. Companies that use DSDM pay a membership fee. The DSDM Consortium also offers training and certification.

The DSDM “spec” is released as a specific version (much like software). At the time of this writing, the Consortium is on version 4.2. For more information, see www.dsdm.com.

Scrum

Scrum is an agile development process aimed primarily at managers. Its focus is on helping project leaders to manage teams of highly skilled developers, and get the best from them without treading on their toes. As such, Scrum can be “wrapped” around more technical development processes such as XP or ICONIX Process.

Scrum divides projects into monthly “sprints,” with an incremental amount of new functionality delivered at the end of each sprint. Uniquely in an agile process, the customer is not allowed to change the requirements during the monthly sprint. If the requirements are discovered to be off base once the sprint is under way, the customer must wait until the 30-day sprint has ended before instigating a change. While this can help to stabilize development (and to make project stakeholders very focused on getting the requirements right), most other agile processes (including ICONIX Process) make the point that if a requirement is discovered to be wrong, you should take steps to correct it as soon as possible.

The book Agile Software Development with Scrum by Ken Schwaber and Mike Beedle (Prentice-Hall, 2001) contains useful advice on how to introduce agile methodologies into large organizations. Also see www.controlchaos.com.

Feature-Driven Development

Of the agile methodologies described here, Feature-Driven Development (FDD) is in many ways closest at heart to ICONIX Process.

FDD places an emphasis on up-front design techniques and driving the design from “features” described using a domain object model. FDD is closely linked with the Together CASE tool/development environment (originally from Peter Coad’s company TogetherSoft, which was bought by Borland). FDD can, of course, also be used with other software tools if needed.

FDD was originated by Peter Coad, and later the methodology was described in much more detail by Stephen R. Palmer and John M. Felsing. Also see www.featuredrivendevelopment.com.

Feature-Driven Development Practices

As previously mentioned, FDD places a strong emphasis on up-front design modeling, driving the development effort from self-contained functional requirements, or features.

FDD consists of five processes, each one providing the input into the next:

1. Develop an overall model.

2. Build a features list.

3. Plan by feature.

4. Design by feature.

5. Build by feature.

Let’s look at each process in turn.

Develop an Overall Model

The domain and development team members work together to create a high-level walk-through of the system. Then the domain members present more detailed walk-throughs of each area of the problem domain.

Build a Features List

Building on the knowledge gathered during the initial modeling activity, the team next constructs as complete a list of features as it can. A feature is defined as a small, client-valued function expressed in the form: <action> <result> <object> (e.g., “calculate the total of a sale”). Existing requirements documents (use cases or functional specs) are also used as input.

Within each domain area (also called a major feature set), the features are grouped into feature sets, where a feature set reflects a particular business activity. Then the users and customer review the feature list for validity and completeness.

Plan by Feature

The feature sets are sequenced into a high-level plan and assigned to chief programmers (aka team leaders). The classes identified in the modeling activity are assigned to individual developers; the owner of a class is responsible for its development.

In large projects, DSDM-style time boxing may be used (e.g., a time box of 3 months might be set to complete the first few feature sets and demonstrate the working software to the customer).

Design by Feature and Build by Feature

These two processes are highly iterative and are where the serious development work goes on. The team leader selects a small group of features to develop over the next iteration (ranging from a few days to 2 weeks). He identifies the classes likely to be involved and matches these up with the class owners. In this way, he selects his feature team for this iteration.

The feature team creates detailed sequence diagrams for the features, and writes class and method skeletons. The team then conducts a design inspection. After a successful inspection, the class owners add the actual code for their classes, unit test, integrate, and hold a code inspection. Once the team leader is satisfied, the completed features are promoted to the main build, and the designing and building processes repeat for the next group of features.

As you might gather from reading this description (and especially after reading Chapter 3), FDD and the ICONIX object modeling process are a very suitable match for each other. As ICONIX Process places an emphasis on sequence diagrams, it can fit into FDD’s design-by-feature process, and it can also complement FDD’s review stages with its own critical design review techniques. Similarly, FDD’s concepts of organizing use cases into features and planning by feature can be applied to an ICONIX Process project.

FDD is described in lots more detail in the book A Practical Guide to Feature-Driven Development by Stephen R. Palmer and John M. Felsing (Prentice-Hall, 2002).

Agile ICONIX

To provide a fair comparison with the agile methodologies described here, we finish up with a brief description of Agile ICONIX, described in a similar style to the previous methodologies.

Overview of Agile ICONIX

Agile ICONIX takes ICONIX Process (a minimal object-modeling process) and describes how to use it in an agile project. Agile ICONIX retains the core philosophy of ICONIX Process, which is that less is more. The less we have to do to safely achieve our goal, the quicker and more effectively we’ll do it. The agile practices that Agile ICONIX uses are a combination of current agile thinking and some practices of its own, which create a core subset of agile practices—the bare minimum that you’d need to do in order to achieve the goals of agility.

How Do ICONIX Process’s Agile Practices Differ?

In Chapter 4, we describe ICONIX Process’s agile practices, and then in the rest of the book we show how they’re applied by example. For now, here’s an overview of how ICONIX Process differs from the agile methods described here.

The key differences are as follows:

• With ICONIX Process, an emphasis is placed on getting the requirements right up front. This is done through a variety of practices that root out ambiguity in the requirements and uncover gaps in the use cases and domain model.

• With ICONIX Process, design is not something that you “quickly do a bit of” before coding (where the real design work happens). Instead, design modeling and coding are tightly interleaved, each providing feedback to the other. Essentially, in ICONIX Process, we “model like we mean it”—that is, we intend to drive the coding of the system from the model. This in turn means that…

• With ICONIX Process, the design and the code don’t diverge over time. Instead, they converge, with the design model and the code becoming more in sync.

For more about Agile ICONIX, see … well, the book you’re reading at the moment.

Modeling Question 

Does this mean that you do all of your use cases, then go back and reanalyze the use cases, and then write all the code? Although we place a big emphasis on getting the requirements right up front, ICONIX Process isn’t by any means a “waterfall” process. It’s actually highly iterative (as we describe in Chapter 4). One way to describe it is with an example of iteratively painting your house. You can primer the whole house, then sand the whole house, then primer it again, then sand it again, then primer it again, then sand it again, then primer it again. That’s one type of iterative process: doing the whole thing over and over. The other way is to primer the front of the house, sand the front, primer the front, paint the front, and then move to the right side of the house and repeat. The first way is more of a repetitive waterfall; the second way is repetitive, but in small chunks. ICONIX Process (and most other agile methods) take the “repetitive small chunks” approach.

^[6.]jUnit, nUnit, cppUnit, and so on.

^[7.]For a different perspective, see Matt and Doug’s other collaboration, Extreme Programming Refactored: The Case Against XP (Apress, 2003).

^[8.]See www.objectmentor.com/resources/articles/Continuous_Care.pdf.

^[9.]Okay, we know, it’s really “code smells”….

^[10.]Scott W. Ambler, Agile Modeling: Effective Practices for eXtreme Programming and the Unified Process (Hoboken, NJ: John Wiley & Sons, 2002).

^[11.]The term chaordic describes a self-organizing, adaptive system, the behavior of which exhibits characteristics of both order and chaos.

^[12.]See Alistair Cockburn’s Crystal Clear: A Human-Powered Methodology for Small Teams (New York: Addison-Wesley, 2004).

^[13.]Alistair Cockburn, Agile Software Development (New York: Addison-Wesley, 2001), p. 202.