Change Examples

I can't stand the status quo and prefer flexibility over any other mode of work. I try to always find better ways to do things and new ways to think about problems. I do this because I thrive on learning; my first priority is getting things done, and I prefer to get things done in the most efficient manner possible. I also like the element of surprise and producing an unexpected result or an unexpected analysis. Hence, I have been involved in many change efforts in my career at the personal, team, and organization levels. Not all have been successful, of course, but I've been fortunate that most of them have.

In this section I'll describe two successful changes. The first was the introduction of a controversial practice into a small development team, and the other was the introduction of agile development into a relatively large (approximately 200 people) mature software organization. I think they're both good examples of leadership and creating positive lasting change.

The Introduction of a Controversial Practice

The small-scale change was the introduction of the root-cause analysis practice I've described several times earlier in this book. At this time, I was a programmer working in a small and highly motivated team. We had a lot of good practices, and we thought we were doing pretty well. When our team leader first introduced the idea to the team of meeting once a week to review all our bugs, we resisted with all the predictable excuses: that it would take too much time, that the return on investment wouldn't be there, etc. But our team leader had some bug history data that convinced us enough to try it out for several months.

As predicted, the first few meetings were painful and took too long. However, I think it clicked for us after about a month when we realized that our drive to fix defects properly before the next meeting was paying dividends in increased quality. Around that time, we also developed an unproven belief that if we stuck with it, our overall productivity would increase. It took a few more months, but eventually we did reach the point where we were driven to reduce our defects through prevention (we got truly serious about automated testing), we always had a working product, and we held each other to increasingly high standards. Our productivity did increase, and we were able to add more features and deal with more complexity in a team that was constant in size.

In this case, our team leader deserves the credit for helping us take the initial leap and then essentially getting out of the way as we built on the early successes. I also acknowledge him for not taking any of the credit for the success of this change, although he certainly deserved it. For that team at that particular time, this was the right change that on hindsight took us from being a good (above average) to a great team. I was a member of that team as a programmer and team leader, and I am extremely proud of what we were able to accomplish.

Introducing Agile Development Into a Mature Software Organization

A few years ago the company I worked for closed down the office I worked at. The team I described in the previous example was dispersed, and we each went off and found different jobs. I joined a smaller company (approximately 400 people, about half of which are in product development) with a completely different software development culture. Its approach to software development was mostly the waterfall method, and it had been used and evolved for over a decade. I was there for about a year, left, and then came back because of the bright people, products, challenging problems, and amazing customers. When I came back, I was promoted into a new position where part of my mandate consisted of technology strategy, software architecture, and software development processes. Attention was needed in all three areas, but I believe that often process changes have the highest return, so that is where I decided to focus initially.

There was a general recognition that the software development processes were not what they should be. Some people were experimenting with different methods, but not with any noticeable success overall. There had been a few attempts to introduce new processes, but they had all failed. I spent a bit of time looking at each one and talking to the people who had been involved. The one attempt that left the biggest impression on me was one where the proposed process was meticulously documented with diagrams and loads of text. It was essentially a proposal to add more bureaucracy, with mandatory documents, sign-offs, and decision-making groups. Nobody really bought into it, for obvious reasons.

I was working with a woman from our HR group, and we decided that we wanted to bring in outside training and focus initially on project management. We started by obtaining an approved budget for some project management training. We spent a few weeks talking to consultants and various training groups. All but one of the groups we spoke to offered what I'll call traditional training in waterfall methods: Gantt charts, PERT charts, project tracking, etc. The one exception was Jim Highsmith. At the time, I had been reading anything I could get my hands on related to software development and development processes, from the CMM to agile. The books that had gotten me the most excited were the Extreme Programming books [Beck 2004] [Auer and Miller 2001] [Jeffries et al 2000], Jim's Adaptive Software Development [Highsmith 1999] book, and the Scrum book [Schwaber and Beedle 2001].

Initially, I wasn't content with any of the groups we spoke to about training. The traditional project management groups made me uncomfortable because, based on my previous work experience, I had minimal confidence that what they taught worked, except perhaps for building bridges. I was intrigued by agile development but didn't feel that any of the methods I had read about were complete. However, I realized that what really resonated with me about agile development were its principles (see www.agilemanifesto.org) and I was able to connect most of these principles to my previous job. We had a really good talk with Jim Highsmith, and he agreed to tailor a course for us that provided a balanced view across what I viewed at the time as project management and technical practices, with practices combined from ASD, XP, and Scrum.

Once we communicated our decision to provide agile development training, we encountered resistance. Mostly it was because at the time agile was new and viewed as largely untried. We were able to convince most people that the worst that could happen from such a course would be that people would have a few new ideas and skills to take away. Luckily, my boss and the majority of my peers were supportive. There was, however, one executive who was openly hostile to the idea and thought it was a waste of money and time. I spent an afternoon with him and outlined my arguments against the traditional methods (some are in the Introduction) and described my experience from my previous job. In the end, we agreed to disagree but, to his credit, he converted more to skepticism and a "wait-and-see" attitude. That experience prompted a phone call to Jim Highsmith, where we outlined a short course for the directors and select executives to introduce them to agile development and to give them a chance to ask detailed questions.

The next step was the actual training course. Our budget allowed us to initially run two 2-day courses plus a 2-hour course for directors and executives. We spent a fair bit of time identifying key people to get into the course (leaders at all levels), ensuring that we had a cross-section of all the various functions (team leaders, software developers, QA people, documentation writers, usability specialists, product managers, etc.) within our department, that the courses had a blend of vocal and quiet people, and that the skeptics were balanced by those we felt had a more open mind. Then we issued invitations and followed up if people didn't respond. A few people said no, but luckily we had a long list.

We held a kickoff prior to the first training session where we got all the course attendees together the day before the first course for 30 minutes. We had arranged for one of the executives to be there to give a short talk, but this person had to travel at the last minute, so it was left to me. I scribbled down some notes, rehearsed beforehand, and then spent some time "on my soapbox." Generally, I enjoy speaking in public, but I was very nervous that day because I knew this was a risk, and it was on me. I tried to speak from the heart and described my perspective on where we were and where we could be. For me, that little talk is when the change started.

The training courses went extremely well and the majority of the responses were positive. One of the reasons they went well is that quite a few people had been experimenting with approaches that were agile in spirit, but what had been lacking, perhaps, was the discipline, a framework, and a broader set of tried-and-true practices.

The next step was followup. One thing I always worry about with training courses is that it's easy for people to get enthused during the course, but the moment they get back to their desks and are confronted with their e-mail, voice mail, feature lists, bug backlog, etc. that reality can quickly overwhelm them. Later, they'll say they never had a chance to try out any of the new ideas. I also worry about situations where a select few people attend a course, come back enthusiastic, and then are beaten down by the pessimism of those who didn't attend the course. Therefore, as a followup, I decided to form an agile guiding council that consisted of the more enthusiastic course participants. The purpose of the group was to get as many people as possible as quickly as possible involved in leading the change and deciding what our next steps were.

The people in the guiding council are the ones who really made the change happen. We met weekly for the first year, then bi-weekly, and then sporadically for about another year after that. The council decided to bring in more training, so we got some more budget approved. This round of training would be a few one-day courses taught by Jim Highsmith and also some technical training that we thought should focus on test-first development, simple design, and refactoring. The council worked in subteams that did things like creating a vision statement and some nifty posters and sourcing and organizing the training. We also spent a lot of time talking about next steps, challenges, and successes. For the technical training, we chose Object Mentor, which eventually also ran some other courses for us. Jim Highsmith also visited us for a day once a month or so to talk to various people and groups. He definitely helped lend credibility to the initiative, because he could speak from his experiences at other companies.

Our first real success came about two months after the initial training. One of the team leaders had left the course and essentially thought "Hmm, that was interesting. I wonder if the company is going to do anything about it?" Then, through his participation in the council and with some encouragement from his peers on the council, he decided to try it out with his team. His team was enthusiastic about the Extreme Programming books, so members built out a quick project with test-first development, etc. The quality was amazing as was the speed that the results were achieved. It definitely turned some heads and got more people interested.

Shortly after the first success, the technical training started paying dividends. Some of the programmers had been adding unit tests to their new code, and as these tests started failing when other programmers modified the code, more people started noticing. The interesting thing is that our products prior to the training did have some automated tests. We had a unit test framework, for example, but it was only used by a few individuals and only sporadically. One of my favorite stories is that we had what were essentially system-level automated tests for one of our products. But everyone ignored the tests because they always failed, and they hadn't been updated in a long time. One day, one of the programmers called a meeting with all the team leaders for the product and, in a very nice way, outlined why the tests should be taken seriously! It was a turning point for that project.

There have been many interesting lessons learned to date. For example, we didn't really understand the importance of the physical workspace to agile development. Pair programming turned out to be impossible on most of our desks because they are too small for two people to sit side by side. For the curious, we don't mandate pair programming, and most of the people who do pair up only do so for an hour or two at a time. Another thing that was lacking was enough whiteboards. I'll never forget one day when I received an e-mail saying that we would have to erase the whiteboards when customers were touring the building so they wouldn't see anything confidential. My reply was a short NO, and luckily I managed to convince people that productivity was more important than customers seeing our building!

A few years later, I can safely say we have achieved lasting change. It isn't uniform across all teams, because there is still a divide between our new products developed since agile was introduced and those that existed before. We have rapidly developed one amazing new product, and we're developing other new products using the same methods. We have built up several new areas now with new desks, and we have happily converted many of our former formal sit-down meeting rooms into agile project rooms where the walls are plastered with magnetic whiteboards and corkboards. As a result, a number of large tables have been relegated to the basement to collect dust so we can stand up and move around.

The challenges for our existing projects have been quite large. The teams have adopted as many agile practices as they can, and they've added many new automated tests. But the amount of code that isn't covered by automated tests is quite large, and in many cases not enough attention was paid in the past to issues such as code duplication and design, so the teams are still paying off accumulated technical debt. Still, huge strides have been made and the teams are motivated to continually improve. I hope that eventually we can close the gap with our other teams.