Hypothesis

Once you have decided where you want to go and considered what you already know about change initiatives, the next step is to develop a hypothesis about how to embark on a lean journey. Our hypothesis is this:

The most effective way to get started with lean development is:

• Train team leaders and supervisors (in preference to change agents).

• Emphasize on-the-job thinking (rather than documentation).

• Measure a few key indicators (as opposed to many data points).

Training

At the start of the 20th century, while Sakichi Toyoda was making looms in Japan, Henry Ford was starting up his automobile company, and Frederick Taylor was publishing his ideas on Scientific Management, a scientist named Charles Allen was developing an industrial education program in New Bedford, Massachusetts. He took a novel approachhe felt that most learning happens on the job, so he taught people how to learn through hands-on experience.

By 1917, Massachusetts had developed a model for American vocational education that was subsequently copied throughout the country, and Charles Allen was its director. There was a war going on, and the US Shipping Board decided to build a thousand new ships. Faced with an urgent need for a tenfold increase in the number of skilled ship builders, the Shipping Board tapped Charles Allen to oversee their training. During the next two years, 88,000 workers went through the training programs Allen developed, which were regarded as a great success. He summarized his experience and philosophy in the book The Instructor, the Man and the Job.^[16]

^[16] Charles Allen, The Instructor, the Man and the Job: A Handbook for Instructors of Industrial and Vocational Subjects, J. B. Lippincott Co., Philadelphia, 1919.

Charles Allen felt that most training happened on the job, so he focused on teaching supervisors how to train workers. His first assumption was that supervisors were highly skilled at the work they supervised. Allen's training programs taught supervisors how to transfer this knowledge to new workers using four steps: preparation, presentation, application, and testing. At the same time that the automobile industry was teaching industrial engineers to find the "one best way" and tell workers what to do, Allen was teaching shipbuilding supervisors to cooperate with workers to solve problems. His philosophy was, "If the learner hasn't learned, the teacher hasn't taught."

Fast forward 20 years to 1940. The US War Production Board faced a similar problem: A rapid escalation in the production of aircraft and munitions created a huge demand for skilled workers, which was exacerbated by the departure of most of the available young men to fight the war. Drawing upon Allen's success, the War Production Board developed Training Within Industry (TWI), a program focused on training supervisors how to train workers. The basic modules of this training program were:^[17]

^[17] See Jim Huntzinger, "The Roots of Lean," www.lean.org/Community/Resources/ThinkersCorner.cfm.

• Job Instruction (JI): This training program was based on the assumption that experienced workers might be very good at what they do, but they may have little experience teaching others how to do the same thing. Using Allen's four steps, supervisors are taught to 1) prepare the worker, 2) present the operation, 3) have the worker try the operation, 4) follow-up by checking frequently, encouraging questions, and gradually tapering off coaching as the worker becomes proficient.

• Job Methods (JM): This training program was based on the assumption that workers have many good ideas about how to improve things, but they may not be encouraged to act on these ideas. Supervisors were taught to help workers: 1) break down the job, 2) question every detail, 3) develop a new method along with colleagues, 4) sell the new method broadly, get approval and apply it.

• Job Relations (JR): This training program taught supervisors to treat people as individuals and helped them resolve "people" problems effectively and fairly. Supervisors were taught to: 1) get the whole story; 2) weigh the issues, don't jump to conclusions; 3) take action, don't pass the buck; 4) follow-up and check results.

Following Allen's lead, TWI training assumes that first line supervisors know how to do the job of the people reporting to them, and therefore can act as a teachers. It is interesting to note that Deming's sixth and seventh points address the same issues as the TWI training program:

6.	Institute training. Managers should know how to do the job they supervise and be able to train workers.
7.	Institute leadership. The job of managers is to help people do a better job and remove barriers in the system that keep them from doing their job with pride.

Perhaps 2 million US supervisors received TWI training, which was credited with large increases in productivity and a huge drop in scrap and grievances. As the war came to an end in 1945, so did the TWI programat least in the US. TWI training was offered in countries recovering from the war and was especially well received in Japan. As it did with Deming's ideas, Toyota brought the TWI ideas inside, added its own experiences and made it a uniquely Toyota program. To this day, supervisors at Toyota receive training based on the ideas of Charles Allen as implemented in the TWI program.^[18]

^[18] See TPS vs. Lean and the Law of Unintended Consequences, by Art Smalley, Ibid.

Thinking

Toyota executive Teruyuki Minoura says that the "T" in TPS stands for "thinking," and the greatest strength of Toyota's "Thinking Production System" is the way it develops people. "Under a 'push' system, there is little opportunity for workers to gain wisdom because they just produce according to the instructions they are given," he says. "In contrast, a 'pull' system asks the worker to use his or her head to come up with a manufacturing process where he or she alone must decide what needs to be made and how quickly it needs to be made."^[19]

^[19] October 8, 2003, Public Affairs Report, Toyota Motor Corporation, www.toyota.co.jp/en/special/tps/tps.html.

Most improvement programs put far too much emphasis on documentation and not nearly enough emphasis on encouraging every person to think every minute about how her or his work situation might be improved. Of course, most companies believe that they encourage workers to think about improvements. For years we have had suggestion systems that encourage workers to write down their good ideas and submit them to the suggestion system, and for years suggestion programs have turned in a dismal track record of improvements. In software development, we use retrospectives to generate lists of things that should change, but too often the same list seems to get generated time and again.

The problem is, most of the ideas from our suggestion programs and brainstorming sessions are turned over to someone else for evaluation and occasionally for implementation. This is a mistake. The people with ideas, working with their teams, should be the implementers of their ideas. They should not be asked to drop their ideas into a suggestion system to be implemented by others.

They should not just add their ideas to a long list of good ideas. When good improvement ideas become someone else's problem, the originators have less personal investment in pursuing the idea, and tacit knowledge about the problem is left behind as well.

All workers, all the time, should be expected to question how their current process works and actively encouraged to use effective problem-solving techniques to try out new ideas and implement those that work. Documentation should exist as a basis for this problem-solving process. It should be developed by work teams, used by work teams, maintained by work teams, and readily changed by work teams.

If we value engaged, thinking people, we have to take a hard look at assessment programs that are based on documentation. Typically these programs measure the maturity of an organization by its conformance to documented procedures. While the objective seems innocuous enough on the surface, in practice the assessment process usually places pressure on an organization to freeze its documentation and therefore not change its processes. The result is a disturbing tendency to remove the "right to think" from front-line workers. Quite often workers are encouraged to do exactly what is documented, while the proper emphasis would be to encourage workers to constantly question what is documented. Documentation-focused assessment programs tend to value the stability of the documentation. In fact, they should view frequently changing documentation as a sign of an organization that has learned how to think.

Measurement

Finding effective measurements for development teams has always been challenging, because results are often not apparent until some time after the development effort is finished. This has led to the proliferation of measurements based on the premise that if each piece of a process is optimized, the results of the process will also be optimized. This premise is a fundamentally flawed view of the way systems work. While it is true that optimizing every piece of a process will show benefits if you start with a totally out-of-control process, it is also true that local optimization will eventually sabotage optimization of the overall system.

Trying to improve the wrong measures creates the wrong incentivesoften leading to unintended consequences. For example, project managers are often measured based on earned value. However, earned value is a measurement of adherence to plan; it ignores the question of whether or not the plan is the best way to achieve the desired results. This is how we get software delivered on time, on budget, with planned scope complete, but still have dissatisfied customers.

Dysfunctional Measurements

A colleague whom I'll call Michelle (not her real name) took over a dysfunctional software development organization a couple of years ago. When she arrived, developer performance was measured in lines of code per hour. Tester performance was measured in number of defects foundmore defects indicating better performance.

I can't think of two measurements more likely to lead to dysfunctional behavior. Measuring lines of code per hour encourages developers to produce quantity without regard for value. Measuring the number of defects found creates a huge disincentive for testers to collaborate with developers to produce defect-free code.

The first thing Michelle did was change the measurements. Lines of code were no longer measured, and teams (developers and testers together) were rewarded for not finding defects. Over the last two years, through these and other initiatives, Michelle has largely transformed the organization into a productive and appreciated contributor to the company's bottom line.

Tom Poppendieck

So what measurements should we use to encourage the right behavior? We propose that instead of proliferating measurements, it is best to reduce the number of measurements and find system-level measurements that drive the right behavior at the subsystem level. In lean organizations, it is well known what these measurements are: cycle time, financial results, and customer satisfaction. Let's consider how they might be used in software development.

Cycle Time

The most fundamental lean measurement is cycle time: how longon the averagedoes it take to go from concept to cash or from customer "order" to deployed software? This single measurement provides a system-level gauge of your process capability. In addition, it exposes every waste in the system: Every missing skill, weak capability, and defective implementation increases cycle time. Trying to do too much at once increases cycle time, as does complexity, unnecessary dependencies, and intolerance for change.

When you measure cycle time, you should not measure the shortest time through the system. It is a bad idea to measure how good you are at expediting, because in a lean environment, expediting should be neither necessary nor acceptable. The question is not how fast can you deliver, but how fast do you repeatedly and reliably deliver a new capability or respond to a customer request.

The objective of a development organization is to first of all establish a repeatable, reliable cycle time for each classification of work, and then to reduce that cycle time through continuous improvement. This single measurement drives all manner of good behavior in every area of the organization, because it aligns everyone in making the right tradeoffs.

Focusing on cycle time reduction requires the full engagement of the people adding value. It drives collaboration, it drives world-class quality, and it drives standards. Almost everything else you might measureability to meet deadlines, quantity of work produced, speed to market, even utilizationwill improve with cycle time reduction. However, the opposite is not the case; if you focus on optimizing the subsystem measurements, cycle time will probably move in the wrong direction.

I want to see some examples of what I should measure for cycle time.

When a defect goes on your defect list, give it a date. When it gets resolved, calculate how long it was on the list. Keep track of these numbers: average cycle time and standard deviation of resolved defects and average age of items on the defect list.

When an item goes on the product backlog, give it a date. If it is broken out into smaller pieces, each piece keeps the original date. If two items are combined, the new item gets the older date. When an item is deployed, subtract the original date from the deployment date. This is its cycle time. Compute the average cycle time and the standard deviation for the items in each release. Also compute the average waiting time of items still in the backlog.

But I put big items on the product backlog. No one has spent any time on them, and customers don't consider them an "order."

1. In this case you might divide backlog items into three categories:

   1. Features customers think they have "ordered" or items that have had any measurable investment of time.

   2. Stuff you need to break out in order to think about architecture, estimate overall schedule, etc.

   3. Really big bullet product roadmap items.

   Items go into the highest category that fitsso if there are customers that are waiting for a roadmap item, it's an A.

   Date everything as it goes on the backlog. Add a new date if an item moves from C to B. Add another date when it moves to A.

When an item is deployed, compute three categories of cycle times:

1. Elapsed time since being assigned to Category A

2. Elapsed time since being assigned to Category B

3. Elapsed time since being assigned to Category C

Every release, compute the average cycle time and standard deviation for each of the three categories. See what the numbers tell you. Do you need some different categories?

While you are at it, what is the average age of the A items left in the backlog? Is it within a release or two of the average cycle time of the A items in the release? If the average age of A items is significantly longer than two release cycles, you are not limiting work to capacity.

Financial Return

Most development efforts receive funding based on a business case. Even government and nonprofit organizations make investments based on business cases. We recommend that the primary measurement of development success should be the realization of the business case. This means, of course, than only realistic business cases should be used to justify development, and that it will be necessary to follow-up by measuring the actual results. True, it takes time for the business case to be realized, and true, there are other factors involved. But the basic principle here is that if you measure what you really want, you are much more likely to get it.

If the ultimate goal of the development effort is to create a profitable product, then the development team should understand the profit and loss (P&L) model for the product so they can gear their work to create the most profitable product. For internal development, contracted development, or nonprofit development, the team should be challenged to realize an appropriate return on investment (ROI) or whatever other business metric (e.g., throughput) is used to justify the investment.^[20]

^[20] For examples of P&L and ROI models for development teams, see our first book, Lean Software Development: An Agile Toolkit by Mary and Tom Poppendieck, Addison-Wesley, pp. 8391.

We have found that most companies do not expose their development teams to the financial implications of their work. However, every development team we talk to would be delighted to understand the financial objectives of their efforts, make tradeoff decisions with those objectives in mind, and experience the sense of accomplishment that comes from having met those objectives.

Sometimes the objective will not be financial: it may be increased audience for a public radio station, number of children receiving vaccinations, etc. In any case, the first step in chartering a development team is to communicate a clear and compelling statement of what constitutes success, and the final step is to measure that successor lack thereofand make it visible.

Customer Satisfaction

Great solutions delight customers. True, basic needs of customers must be met, and performance must be in line with competitors. But the goal of lean development should be to find ways to delight customers by understanding their situation deeply and solving their problem completely. In the book The Ultimate Question,^[21] Fred Reichheld claims that companies that delight customers gain a sustainable competitive advantage, while companies that annoy customers will lose these customers the moment a more attractive alternative is available.

^[21] See The Ultimate Question: Driving Good Profits and True Growth, by Fred Reichheld, Harvard Business School Press, 2006.

Reichheld proposes a single, simple measurement to find out if customers are delighted: the net promoter score. Ask customers one simple question: How likely are you to recommend our product or service? Customers respond on a scale of 0 (would not recommend) to 10 (will definitely recommend). The responses are grouped thus: Scores of 9 and 10 indicate a "promoter." Scores of 06 indicate a "detractor." Scores of 7 or 8 indicate a passive customer. These scores are similar to grading in school, where 90100 is an A or B while scores below 70 are failing scores.

To calculate a net promoter score, subtract the percentage of detractors from the percentage of promoters. You will get a number between -100 percent and 100 percent. Average companies might have a score around 10 percent. Really good companies are in the 50 percent or higher range. Negative scores are cause for serious concern. Reichheld presents evidence that the net promoter score is highly correlated to both market share and profitability, making it a good high-level view of customer satisfaction in a single number.