Principles and Practices

Principles are underlying truths that don't change over time or space, while practices are the application of principles to a particular situation. Practices can and should differ as you move from one environment to the next, and they also change as a situation evolves.

Let's say that you want to change your software development practices because they aren't working very well, but it's not clear what new practices you should adopt. In other words, you would like to move away from something, but it's not clear where you are heading. As you look for a new approach to software development, is it better to spend your time understanding principles or studying practices?

There is a learn-by-doing approach: Adopt a coherent set of practices with confidence that this will eventually lead to understanding of the principles behind them. Then there is an understand-before-doing approach: Understand the underlying principles, and use them to develop practices for specific situations. We observe that the best results come from combining the two approaches. Copying practices without understanding the underlying principles has a long history of mediocre results. But when the underlying principles are understood, it is useful to copy practices that work for similar organizations and modify them to fit your environment. This can provide a jump-start to implementing the principles.

For example, when Mary's video tape plant implemented Just-in-Time, it was not possible to simply copy Toyota's practices because her plant was a process plant, not an assembly plant. The management team had to think carefully about what Just-in-Time meant in their world. They decided to use the Toyota Kanban system and implement pull scheduling, which were specific practices. But they knew that inventory reduction alone was not enough to achieve breakthrough cost reduction, so they developed unique approaches to engage production workers and create a stop-the-line culture. This combination of understanding principles and adapting practices led to dramatic success.

Many companies copied Toyota's Kanban system during the 1990s with mediocre results. Those companies didn't see lean as a management system aimed at eliminating waste. We suspect that many companies with Kanban systems left plenty of waste in their value streams. Similarly, many companies will implement some of the agile software development practices discussed later in this book, but if they don't recognize waste and manage the value stream so as to eliminate it, if they don't possess a deep respect for their workers and partners, results may be similarly mediocre.

Software Development

Lean practices from manufacturing and supply chain management don't translate easily to software development, because both software and development are individually quite different than operations and logistics. Let's take a look at each of these wordssoftware and developmentand analyze just where their uniqueness lies.

Software

Embedded software is the part of a product that is expected to change. If it didn't need to change, then it might as well be hardware. Enterprise software is the part of a business process that bears the brunt of its complexity. If there is a messy calculation or complicated information flow, software gets assigned the job. Software gets the user interaction jobs, the last minute jobs, the "figure it out in the future" jobs. Because of this, almost everything we know about good software architecture has to do with making software easy to change.^[1] And that's not a surprise because well over half of all software is developed after first release to production.^[2]

^[1] For example, in the article "Quality With a Name," Jim Shore defines a high-quality software architecture this way: "A good software design minimizes the time required to create, modify, and maintain the software while achieving acceptable run-time performance." See: www.jamesshore.com/Articles/Quality-With-a-Name.html

^[2] The percentage of software lifecycle cost attributed to "maintenance" ranges between 40 percent and 90 percent. See Kajko-Mattsson, Mira, Ulf Westblom, Stefan Forssander, Gunnar Andersson, Mats Medin, Sari Ebarasi, Tord Fahlgren, Sven-Erik Johansson, Stefan Törnquist, and Margareta Holmgren, "Taxonomy of Problem Management Activities." Proceedings of the Fifth European Conference on Software Maintenance and Reengineering, March 2001, 110.

As time goes on, modifying production software tends to get more difficult and expensive. Changes add complexity, and complexity usually calcifies the code base, making it brittle and easy to break. Far too often companies are startled to discover that their software investment has turned into a tangle of unmanageable code. But this doesn't have to happen. The best software products have been around for a decade or more, and every useful product of that age has been changed regularly over its lifetime. These products have architectures and development processes that build change tolerance into the code. All code worthy of the name software should be designed and built with change tolerance in mind.

Development

Development is the process of transforming ideas into products. There are two schools of thought about how go about this transformation: We might call one the deterministic school of thought and the second the empirical school of thought. The deterministic school starts by creating a complete product definition, and then creates a realization of that definition. The empirical school starts with a high-level product concept and then establishes well-defined feedback loops that adjust activities so as to create an optimal interpretation of the concept.

The Toyota Product Development System sits squarely in the empirical school, starting with a vehicle concept rather than a definition and empirically fleshing out the concept into a product throughout the development process. For example, the product concept of the Prius did not mention a hybrid engine; it set a fuel economy target of 20 kilometers per liter (47.5 miles per gallon). The Prius product concept also called for a roomy cabin for passengers, but did not set vehicle dimensions. Only during development did the team settle on the hybrid engine, barely out of the research lab, as the best way to achieve the aggressive fuel economy target.^[3]

^[3] Jeffrey Liker, The Toyota Way, McGraw-Hill, 2004. See Chapter 6 on the development of the Prius.

We believe that any development process that deals with a changing environment should be an empirical process, because it provides the best known approach for adapting to change. As we noted above, software by its very nature should be designed to adapt to change both during initial development and over its lifecycle. Thus, software development should be an empirical process.

What Is This Thing Called "Waterfall?"

I didn't run into the term "waterfall" until 1999 when I started working on a government project. I had been a (very good) programmer in the 1970s, writing software that controlled machines. This might be called embedded software today, but in those days the computers were hardly small enough to be embedded. I worked on some large projects that involved building and starting up complex process lines to make tape.^[4] My colleagues on these projects were engineers with years of experience developing large, complex tape-making lines. The projects were managed by seasoned experts, who knew how to get approval for an overall budget and schedule, then turn things over to the experts to adapt the plan as needed to deliver a working line. Everyone was well aware of the fact that while budget and schedule were important, the overriding goal was to deliver a line that made excellent product. And we always did.

When I became the information systems manager of a video cassette plant, I used the approach I had learned as an engineer to manage the development of new software, always keeping in mind that my department's primary job was to support production. Later I moved into product development, where we used a rigorous but very adaptable stage-gate process to commercialize new products.

Thus, I managed to escape working with the waterfall methodology until I bumped into it in 1999 on that government project. I was puzzled by the waterfall approach because I couldn't understand how it could possibly work; and as a matter of fact, it didn't work. As I compared my experience working on complex, successful projects to the prescribed waterfall approach which failed on a relatively small project, I decided to write a book about what really works.^[5] In that book we outlined seven principles of software development, which are summarized below.

Mary Poppendieck

^[4] The equipment needed for making tape bears some resemblance to the automated looms manufactured by Toyoda Automatic Loom in the 1920s.

^[5] Mary and Tom Poppendieck, Lean Software Development: An Agile Toolkit, Addison-Wesley, 2003.