Chapter 1. In the Beginning

The phone on my desk startled me, breaking my concentration, demanding attention. "This is Gary," I mumbled into the receiver, somewhat miffed that this interruption took me away from dreaming about the "good old days" when life was simple. No, I don't mean that I walked 20 miles in tattered shoes in a foot of snow so I could get to school. I was thinking about how simple programming seemed when I started in the computer industry. I was lamenting the complexities we seem to add for little or no reason today. We do this even for the small projects to which I have always gravitated.

"Gary, this is Russell. Have I got a deal for you!" Russell and I have been friends for years. We worked together in those very same good old days that I was just dreaming about. He is now the development manager of a small company that has been around for a few years and has become successful in its niche. His company builds computer peripherals and the software that goes with them.

Somewhat leery of the "deal," and remembering some of the pranks we've played on each other over the years, I asked, "What do you have in mind?"

"Remember our conversations last year about process and tools and how they can improve a development organization? Well, we've started using the Rational Unified Process and it seems to be helping us. Now my programmers have agreed to try some of the techniques in the Personal Software Process you pointed out to me. My lead programmer looked at PSP and tried it on some very small tasks . He is on board and thinks it will help the organization be more predictable."

"That's great, Russell, but what's the deal?" He certainly had my attention ”he'd just mentioned some of the topics I had been thinking about for years. And it turned out that his call hadn't interrupted my reverie after all; my daydreaming had, in fact, been a nice preamble to our conversation.

"Well," Russell continued , "now that the programmers are willing to try PSP, I don't want them to spend a lot of time recording the data and massaging it. We need tools to help automate PSP. I want you and some of your colleagues to build the tools for us."

Now I was really interested. I thought back to some of my conversations with Russell over the last year. One of my beliefs is that a small team can achieve outstanding results building software if it applies the right combination of process and tools. Programmers and other technical practitioners have typically challenged me to prove this assertion. What data did I have to support my belief?

Over the course of my career, I've seen many projects that successfully married process and tools. Most recently, I've worked with, and helped others work with, the Rational Unified Process (RUP). While many of the customers I've worked with are from large organizations, many of them are trying to apply RUP to small teams working on small projects within their organizations. They typically run into trouble applying RUP to smaller projects because they think RUP just does not apply to such projects. I am able to help them get started, but I've always wanted to show them a complete example. An example would be like the proverbial picture worth 10,000 words. (See Appendix A for a brief introduction to RUP.)

Today there are several emerging "agile" processes such as eXtreme Programming (XP) (see Appendix C), SCRUM, Dynamic System Development Method (DSDM), and Feature-Driven Development (FDD). Each of these processes has good advice for practitioners, just as RUP does. They each can point to success stories. They are designed mainly for small teams, but most of them constrain the type of team that should adopt the process (for example, some processes require the team to be co-located). See the accompanying sidebar on Agility.

The challenge for project managers and others working in software development is to decide which parts of all of the different methods work best for them. The best way to accumulate the knowledge is to have personal experience working on projects using each of the processes. Unfortunately, this is not practical. The next best option is to learn how other people have benefited, or not benefited, from applying the different practices. This is similar to programmers learning how to write better code by reading other programmers' code.

We are still missing much hard data on projects. Mostly, people are too busy building software to record the results in detail. Many measurements are done in laboratory-like conditions that don't capture the chaos of the real world. Lab results are often unrealistic because too many variables must be controlled in order to collect a large data set. Perhaps the best data we can get are in the form of well-documented examples.

Agility

Sometime before 2000, the term agility began to appear in software development circles. The concepts and principles of agility came from many sources, and different words were used to describe the concepts. In February 2001, the leading proponents of the methods met to discuss differences and similarities. The attendees found they had much in common.

The original Agile Alliance emerged from this meeting and produced the Agile Manifesto (see agilemanifesto.org/). There are four core values expressed in the manifesto, and emphasis is placed upon one of two properties at different ends of a conceptual spectrum. The manifesto asserts that the Alliance values:

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan

The signatories to the manifesto explicitly state that they believe the items on the right have value, but not as much as those on the left. Some people miss this when they talk about agility and assert that the only things of value are those on the left. This is not the intent of the authors of the manifesto.

If you consider the four value statements as Boolean values ”either you have the value or you do not ”you end up with sixteen possible combinations. Agility represents one of the combinations.

You might also think of each of the value statements as representing a continuous spectrum. In this case, you have an infinite number of possible combinations. We think this is closer to reality, but it's much easier to characterize a process when you work with sixteen combinations rather than an infinite number of combinations.

Several methodologies claim to be agile. Two of the best known are eXtreme Programming and SCRUM. Our belief is that, when applied according to the spirit in which it was developed, RUP can easily be configured to both embrace the agile values and complement other Agile processes.

I knew that to create a realistic example, I would have to find both an application to build and a small group of colleagues to work on it with me. While creating the example, our group would record what we did and what we decided not to do. After we were done, we could look back and see what worked and what didn't work. An example like this would not be the perfect classroom case study, but it would be a real project, with real results and experiences. I could show it to people to demonstrate how one team applied the spirit of RUP to a small project.

I remember telling Russell about my desire to work on a sample project where we could provide the rationale for our decisions and capture real examples along with all the intermediate artifacts and the final product. And now he was calling to offer me the opportunity I was looking for.

I worried about over-committing myself . "Russell, that sounds great, but you know I have a real job that pays the bills. I'm sure you aren't going to pay my bills for me."

"I knew you were going to say that," Russell said. "That's why I'm calling you now. We aren't starting our next project for at least six months. That should allow you to work a little in your spare time, if you are willing to help your old buddy out. I want the tools ready for my team when we start the new project. It would be great to have the tools before that, so we can try them out, but you have at least six months before we need the first 'real' release." (It turns out that we actually had more than six months. This was helpful because it took us a while to get started.)

Russell continued, "I have some ideas about what the tools need to do for us. I started with the information from the PSP book and the information from the PSP supporting tools specification that was on the SEI Web site. ^[1] I want to add some features to support the way our team works. We don't need all of PSP implemented right away. We probably need PSP level 1 in the first release." ^[2]

^[1] The Personal Software Process (PSP) book is A Discipline for Software Engineering by Watts S. Humphrey. The SEI is the Software Engineering Institute at Carnegie Mellon University (see www.sei.cmu.edu/). The specification for tool support on the SEI Web site has been removed since Russell called.

^[2] There are several levels of the PSP corresponding to individual engineering maturity. It is similar to the Capability Maturity Model (CMM) defined by the SEI. See Appendix B for an overview of PSP.

"Russell, I'll get back to you next week and let you know. I need to talk to some colleagues here and put together a team that can get the job done for you." When I hung up, my mind raced to put together a plan that would allow me to help my friend and, at the same time, see if I was right about trying to make software development fun, like it was in the "good old days."