Selecting and Testing Solutions

Overview

Purpose of these tools

• To generate solution ideas targeted at identified problems
• To select the best solution from among multiple options
• To successfully implement solutions

Deciding which tool to use

• If you need help generating solution ideas…
  • Review Sources of Solution Ideas (p. 254) and Benchmarking (p. 254)
• If you want to compare alternative solutions…
  • Tips on Solution Selection, p. 255
  • Developing and Using Evaluation Criteria, p. 256
  • Solution selection matrix, p. 258
  • Pairwise ranking, p. 261
  • Cost Evaluation, p. 264
  • Impact/Effort Matrix, p. 264
  • Pugh Matrix, p. 265
  • Other evaluation techniques, p. 268
• If you want to identify and counteract risks for likely solutions…

  These tools are generally used on solution ideas that have made it through the initial assessments:

  • Controls Assessment Matrix, p. 269
  • Failure Modes and Effects Analysis (FMEA), p. 270
• If you want to plan and test solutions…
  • Piloting testing, p. 273

Sources of solution ideas

• Always start with a confirmed root cause, verified by data collection, process observation, and/or experimentation
• Look to the following for ideas on how to counteract the root cause:
  • Ideas sparked by discoveries made during process analysis
  • Best practices
  • Other projects that have faced similar or related challenges
  • Brainstorming (p. 27) (be sure to explore the many advanced creativity techniques published in many sources)
  • Performance targets
  • Benchmarks

Tips 

• Focus on one root cause at a time
• Start with the root causes determined to have the biggest contribution to the problem statement

Benchmarking

Highlights

• Benchmarks are measures (of quality, time, or cost) that have already been achieved by some company, somewhere
• They tell you what' s possible so you can set goals for your own operations
• Benchmarking can be very helpful to inject new ideas into the process and borrow the good ideas from other companies/industries

Sources of benchmarking data

• Surveys or interviews with industry experts
• Trade or professional organizations (check their databases, seminars, publications, websites, experts)
• Published articles (research, trade)
• Company tours
• Prior experience of current staff
• Conversations

Types of benchmarks

	Pros	Cons
Internal/Company	Establishes a baseline for external benchmarking Identifies differences within the company Provides rapid and easy-to-adapt improvements	Opportunities for improvement are limited to the company' s internal best practices
Direct Competition	Prioritizes areas of improvement according to competition Initial area of interest to most companies Best used for in-depth studies as supplement to competitive intelligence studies	Often a limited pool of participants Opportunities for improvement are limited to "known" competitive practices Potential antitrust issues
Industry	Provides industry trend information Provides management with a conventional basis for quantitative and process-based comparison	Opportunities for improvement may be limited by industry paradigms
Best-in-Class	Examines multiple industries Provides the best opportunity for identifying radically innovative practices and processes Provides a brand new perspective Free exchange of information more likely to occur	Often difficult to identify best-in-class companies Sometimes difficult to get best-in-class companies to participate

Tips on solution selection

1. Generate potential solutions: It is best to exhaust all ideas that the team can produce. Often, the best potential solutions emerge toward the end of intense creativity efforts, when everybody has many details fresh in mind related to the problem at hand.
   • The more potential solutions available to explore, the more opportunities for uncovering ideas for improvements
2. Narrow the list and synthesize: If you have more ideas than you can reasonably act on, use affinity diagrams (p. 30), multivoting (p. 31), or other techniques to identify themes and trim the list. Whenever possible, synthesize by combining the best characteristics from alternative options to generate a single stronger solution. (Be sure to develop and use evaluation criteria, see below.)
3. Select the best solution: Selection involves deciding which solution improvements to implement.

Tip 

• The synthesis and selection process is iterative. Work through it once to generate some ideas, evaluate those ideas, then brainstorm off the best options or newer ideas to see if you can develop even better solutions.

Developing and using evaluation criteria

Highlights

• Identifying and documenting criteria takes the guesswork out of selecting solutions
• Use all sources of information to determine evaluation criteria
  • Talk to project sponsors, stakeholders, customers, process staff
  • Review your team charter and customer interview notes

Example criteria

• Customer requirements (CTQs)
  • Process or Output Impact—How well will this solve the problem?
  • Customer satisfaction
• Business needs
  • Alignment with strategy
  • Impact on time, operating costs, marketshare, etc.
  • Capital investment
  • Risk to implement
  • Penetration of new markets
  • Brand recognition or enhancement
• Regulatory/Other
  • Adherence to regulatory requirements
  • Safety
  • Environmental and political constraints

Weighting criteria

Not all criteria are created equal. When evaluating alternatives, it' s best to weight the criteria by assigning numerical values that indicate relative importance.

1. Identify the criteria you want to use
2. Assign a numeric value to each criteria that indicates its relative contribution towards achieving your goals
   • You can use pairwise comparisons (p. 261) to develop weights if you want
   • Make sure there is enough spread in the values to allow you to distinguish between truly good and truly poor ideas

     Ex: If "Providing on-time delivery" is a lot more important than "using existing software," give delivery a value of 10 and software a 3 or 4

     Ex: If on-time delivery is only slightly more important than using existing software, give delivery a 10 and software a 7 or 8

Tip 

Questions to help determine the evaluation criteria:

• What will the best solutions look like?
• What are the barriers to implementation?
• Which type of solutions will be the cheapest to implement?
• Which type of solutions will be the most dramatic? The most visible?
• Which type will show the fastest results and deliver the most "bang for the buck?"
• Which type will meet the least resistance and be the easiest to put in place?
• What factors are most likely to affect your department and other departments?

Solution selection matrix

Purpose

• To document the solution selection process and criteria
• To make sure the solution selected for implementation provides the best chance for meeting the project goals

When to use a solution selection matrix

• Use a solution selection matrix whenever you have two or more alternative solutions to compare (which will be most of the time)

How to create and use a solution selection matrix

1. Remove show stoppers from your list of alternative solutions. Solutions with components that would prohibit their implementation should be removed prior to performing additional analysis.
   • Ex: addresses a defect but results in a large adverse impact on customers
   • Ex: directly conflicts with the organization' s strategy
   • Ex: goes beyond the scope of the charter
2. Consider organization fit for each remaining idea. The solution must be capable of obtaining management commitment, and fit with customer needs, strategic objectives, organizational values, and the organization culture. Eliminate any options that do poorly on questions such as:
   • Management commitment—can you develop support for this idea?
   • Strategic factors and organizational values—is this idea consistent with our one-and two-year goals? five-year goals?
   • Operating and management systems—does the potential solution complement or conflict with our decision making, accounting, communication, and reward systems?
3. Determine project goal impact for each remaining idea. Each potential solution must be evaluated on its ability to reduce and eliminate the root causes of poor performance. The solution must have a sufficient impact on the process to achieve the desired performance levels. Solutions that can' t produce the desired results must either be altered to meet the goals or removed from consideration.
   • Standalone (independent) solution ideas: These are solutions that by themselves are capable of satisfying the project goals or, due to their uniqueness, can' t be combined with other solutions

     

   • Coupled solutions These are solutions that in isolation are not capable of satisfying the project goals, but are capable of being combined with other solutions

     

4. Narrow the list. Alternative techniques include:
   • Multivoting—Use to trim a list down to a manageable number (4 to 7). See p. 31 for instructions.
   • CDAM—Evaluate the alternatives and see if you can Combine, Delete, Add, or Modify them.
   • Pairwise Ranking—Qualitatively rank solutions against each other. See p. 261 for instructions.
   • Force Field Analysis—Evaluate forces working for and against the various alternatives.
5. Enter criteria and top solution alternatives into a solution selection matrix

   	Process Impact	Time	Cost vs. Benefit	Other	Total Score	Rank
   Weight	2	2	3	1
   Opt 1	8	8	10.5	4	26.5	2
   Opt 2	14	18	22.5	7	32.5	1
   Opt 3	2	4	21.0	1	28	3

6. Score alternatives on each criteria
   • Process impact: Score the impact the solution will have on the problem (the ‘Y’), versus the other solutions.
   • Evaluate time impact: Decide whether it' s important to understand (a) The total amount of time needed to design and implement the solution, or (b) How long it will be before a process is actually performing at desired levels (weeks, months, etc.). Rank or score accordingly. An alternative is to express time impact in terms of the expected fulltime resources (FTEs) necessary to implement the solution.
   • Evaluate cost/benefit impact—What is the relationship between the total costs and the expected benefits the business will realize as a result of the implementation?
   • Evaluate other impacts— What are the impacts that the organization wants to keep visible during the decision-making process? Common examples include safety, business risk, and morale.
7. Use FMEA (p. 270) or any risk-evaluation technique commonly used in your company, as appropriate

Tips 

• After a lot of discussion on each of the solutions, team members may find themselves gravitating toward one or two favorites. Remaining objective is important—a bias or uninformed preference may cause the team to overlook excellent cost-beneficial solutions.
• Remember, nothing gets thrown away. These tools help the team focus on likely solutions—but later you may find it helpful to revisit ideas that were set aside previously in order to spark more creative thinking.

Pairwise ranking

Highlights

• Pairwise ranking techniques can be used by individuals or teams to prioritize a list of items qualitatively.
• While there are many different variations of this technique, all of them force you to rank items against each other. The combined results of these paired rankings help to clarify priorities.

  

To use pairwise ranking to select solutions…

1. Identify the solution ideas you want to compare
   • List them on a flip chart or white board so that everyone can see them
   • Label them with numbers or letters (Ex: A to E if you have five options)
2. Draw a matrix with as many columns and rows as there are items (Ex: five rows and five columns if you have five items)
   • This matrix will be used to record the results, so it should be visible to everyone (on a flip chart, white board, overhead projector, etc.)
   • Label the columns and rows with the assigned numbers or letters
   • Color in or otherwise mark off the diagonal boxes that represent comparing an item to itself
3. Review or develop criteria
   • During the comparison, you' ll be asking how one solution compares to another, which means you all need to agree on what criteria will be used
4. Compare each item to every other item, one at a time, until you' ve filled out the upper or lower half the matrix (doing both sides would be repetitive)
   • For each comparison, ask which solution is better
   • Use a show of hands or other voting method to select which is better

   

5. Record the results
   • Simply put the letter or number of the preferred option in the box

     Ex: In one comparison, Option A was voted as being better than option B

6. After completing all comparisons, tally the number of times each option won out. Interpret the results.
   • In the example above, A = 3, B = 1, C = 0, D = 4, and E = 2
   • This would indicate an overall preference for item D.
   • Explore ways to improve option D by incorporating elements of the other strong options. Here, for example, ask why Options A and E are stronger than B or C. Can you work elements of A and E into the solution?
7. OPTIONAL: Sometimes the interpretation is easier if you tally the number of votes each item gets instead of the number of times it "won." Here' s a portion of the above example, this time with the number of votes recorded:

   

   • This increases the spread between items, emphasizing strong differences and giving added weight to items that people feel strongly about
   • In this example, the tally might end up something like: A = 12, B = 5, C = 0, D = 18, E = 8

Using pairwise ranking to weight criteria

Uses the same method above except compares potential criteria for evaluating solutions, not the solutions themselves.

1. Identify and label (letter or number) the criteria.
2. Create a table and compare each criterion against each other criterion. In this case you' re asking "Which of these criteria is more important?"
3. Tally the number of times each criterion "wins" then add one point (to make sure there are no 0s in the table).
4. Tally the total number of points.
5. Determine the % of points each criterion received.
6. Assign weights (usually on a scale of 1 to 5 or 1 to 10, with higher numbers being better)

Cost evaluation

Most solutions will need to be evaluated based on the cost of implementation.

• Sources of costs: direct vs. indirect, one-time vs. ongoing
• Timing of cost and benefits
• Financial analysis using company-preferred measures (talk to your financial experts to find out which of these are used in your company):
  • Economic Profit (EP)
  • Return on Invested Capitol (ROIC)
  • Working Capital
  • Type I, II & III Savings
• Sensitivity analysis of involved factors
• Qualitative factors (cultural fit, potential shifts in jobs, etc.)

Impact effort matrix

Impacts to consider

• Anticipated changes to the overall flow of the work
• Changes in job responsibilities
• New ways to accomplish key tasks—Do you have access to new technology? Can you outsource?
• New ways required to deliver on customer needs
• How will you avoid problems/defects?
• How will you reduce cycle time?
  • Reduce or eliminate rework, sequential processing, batch work, handoffs, approvals/inspections, delays or bottlenecks
  • Increase or improve customer involvement, employee skills/knowledge, parallel processing, one-at-a-time processing (start to finish), non-stop processing, appreciation/accountability for entire job

Pugh matrix

Purpose

A decision-making tool to formally compare concepts (processes, services, products) based on customer needs and functional criteria.

• It quickly identifies strengths and weaknesses for each potential solution so that the strengths can be preserved and the weaknesses corrected or at least addressed
• The objective is to improve upon the initial ideas and eventually converge on an optimal solution

When to use a Pugh matrix

• Primarily used when designing or redesigning products, services, or processes

How to create a Pugh matrix

Note 

A Pugh matrix analysis is performed iteratively, with each time through called a "run." The first time through ("first run") you want to develop a small number of strong solutions. Then you repeat the full analysis to confirm the initial results.

1. Develop potential solutions (alternative concepts). Capture each, as appropriate, in…
   • Drawings
   • Word descriptions
   • Other characterizations of potential solutions
2. Identify criteria
   • See p. 256 for general instructions. Focus on those most relevant to the purpose of your current effort (customer requirements, design parameters, project goals, etc.)
3. Weight the criteria (see p. 257)
4. Select one alternative as the baseline
   • Often the current method or model, but could be any of the alternatives
5. Prepare an evaluation matrix
   • List alternative solutions across top of matrix
   • List criteria in first column
   • Importance ratings in second column or in last column
   • Enter "B" in all cells under the Baseline alternative

     

6. Score each of the alternative solutions against the baseline:

   + or ++ means better or significantly better than baseline

   − or −− means worse or significantly worse than baseline S means about the same

7. Sum the pluses (+), negatives (−), and "sames" (S)
8. Multiply the counts of the +' s,−'s, and S's by the Importance Rating (weights) and sum vertically to compute the Weighted Total for each concept
   • Do not treat the numbers as absolute
9. Focus first on the alternative(s) with the most pluses and the fewest minuses
10. Look for strengths and weaknesses and determine how to attack the weaknesses
    • What is needed to reverse the negatives?
    • Does the change reverse any of the positives?
    • Can strengths of other alternatives be incorporated to address or reverse the negatives?
    • If another alternative is "S" or "+", perhaps that alternative contains an idea (solution) that could improve the selected alternative
    • If a modified solution emerges, enter it into the matrix
    • Eliminate truly weak concepts from the matrix
11. Examine positives
    • If you find a uniform strength exhibited across a number of solutions, either (a) the criteria are too ambiguous to discriminate between solutions or (b) some solutions are strongly related (are subsets) to the others
      • In the first case, decompose the criteria (divide them into subcomponents)
      • In the second case, decompose the solutions, then look for ways to recombine them
12. Identify the strongest possible solutions and repeat analysis to confirm first-run results
    • If the first run is not confirmed, continue to analyze and improve the alternatives until a strong solution emerges
    • Conduct a confirmation run using the strongest solution as the baseline
    • If the second run confirms the first, proceed to Controlled Convergence
13. Perform Controlled Convergence analysis: enhance positives and eliminate negatives to get a better solution than any of the original ideas
    • List product service and process attributes
    • List alternative solutions
    • Pick strong solution as baseline
    • Evaluate solutions against defined criteria
    • Evaluate solutions as better (+), worse (-), or same (S) compared to baseline
    • Assess the individual solution scores
    • Identify strong and weak solutions
    • Attack the negatives and enhance the positives
      • Attempt to reverse the negatives by improving designs
    • Enhance the positives of the weaker solutions
      • Attempt to improve the solution designs. Strengthen strong solutions with strong aspects of the weaker solutions
    • Abandon solutions that remain weak
      • Add new Solutions if warranted. Matrix should get smaller.
    • Rerun the matrix using the strongest Solution as a new baseline
    • Repeat the process until strong solutions persist through several runs
14. Perform product and process design work if necessary

Tip 

Pitfalls that often occur in the first run

• Desire to cut-and-run prematurely terminates process
• Doubts about the validity of process arise, especially as strong solutions emerge and stay strong
• Disruptions by those who see the process as a competitive winlose contest, rather than a collaborative win-win effort

Other evaluation techniques

Simulation models—Ask for help from experts to develop computer simulations that model the current process and demonstrate how an improved process might operate.

Benchmarking—Identify the "Best-in-Class" organization in any process or capability you' re studying, regardless of what industry that company is in. For example, Memorial Hospital might look at Hilton Hotel' s guest registration process when evaluating its patient admission process. (See more on benchmarking on p. 254.)

Industry standards—Research organizations conduct comparative studies of industries and publish performance data. A project team at a refinery might reference research data on refinery capacities to assist their decision.

Internet research—Using the numerous search engines, teams can conduct competitive analysis or benchmarking with organizations worldwide.

Conducting a limited pilot—The team may elect to implement the solution through a limited pilot. Before and after measurements may be used to extrapolate the value if implemented on a larger scale. Consider the time to implement as well as the associated risks to the normal operation.

Controls assessment matrix

Project Objective: Accurate receipt of contracts via EDI

Risk: Information is lost in transaction

Sufficiency of Control: (x = excessive, a = adequate, i = inadequate)

Controls	Existing Y/N	Is it sufficient? If not, why?	Next Steps
a) Training	Y	a-Training conducted	What? Who? When?
b) Procedures not specific	N	a-Configured to prevent submission of transactions without required data
c) No Process for customer master	N	i-Customer master verification

Date:__ Prepared by:__ Reviewed by:__ Next Update:__

Highlights

• Helps identify the activities designed to protect you and your business unit from high-priority risks

To create a controls assessment matrix…

1. List your highest priority risk
2. List the controls that would help mitigate or eliminate the risk
3. Determine if those controls exist
4. Discuss the appropriateness and sufficiency of each control
5. Identify actions needed to address any control deficiencies

Tip 

• Use the results as an input to the Process Control Plan

Failure Modes and Effects Analysis (FMEA)

Purpose

A structured approach to:

• Identify the ways in which a product, service, or process can fail
• Estimate risk associated with specific failure causes
• Prioritize the actions to reduce risk of failure
• Evaluate design validation plan (product/service) or current control plan (process)

When to use FMEA

• When designing new systems, products, and processes
• When changing existing designs or processes
• When carry-over designs are used in new applications
• After system, product, or process functions are defined, but before beginning detailed final design
• In Define: To understand the risks of a project
• In Measure: To understand how process steps or KPIVs relate to risk, and to prioritize KPIVs
• In Analyze: To understand the improvement implementation risks
• In Improve: To assess the effectiveness of a Control Plan

Types of FMEA

1. Design FMEA: Analyzes a new process, product, or service design before rollout to understand how it could fail once released. Exposes problems that may result in safety hazards, malfunctions, shortened product life, or decreased satisfaction.
2. Process FMEA: Used to improve existing transactional and operational processes to understand how people, materials, equipment, methods, and environment cause process problems. Exposes process problems that may result in safety hazards, defects in product or service production processes, or reduced process efficiency.
3. System FMEA: Analyzes systems and subsystems in the early stages of concept and design.

   Process or Product Name:				Prepared by:
   Responsible:				FMEA Date (Orig) ____(Rev)
   Process Step/Input	Potential Failure Mode	Potential Failure Effects	SEVERITY	Potential Causes	OCCURRENCE	Current Controls	DETECTION	RPN
   What is the process step and Input under investigation?	In what ways does the Key Input go wrong?	What is the impact on the Key Output Variables (Customer Requirements)		What causes the Key Input to go wrong?		What are the existing controls & procedures (inspection and test) that prevent the cause of the Failure Mode?
   								0
   								0

How to perform FMEA

1. Review the product, service or process
   • If working on a process, start with the steps that contribute the most value
2. Brainstorm then sort possible failure modes
   • A failure mode is the way in which the component, subassembly, product, input, or process could fail to perform its intended function. They may be the result of upstream operations or may cause downstream operations to fail.
3. List one or more potential effects for each failure mode
   • Answer the question: If the failure occurs, what are the consequences?
4. Assign ratings for severity and occurrence
   • Severity of failure: 1-10, with 10 representing most severe impact on customers
   • Likeliness a failure will occur: 1-10, with 10 representing most likely to occur
5. List current monitoring and controls for each failure then assign a detection rating to each
   • Detectability of failure: 1-10, with 10 representing least likely to be noticed given your current control methods
6. Calculate a risk priority number (RPN) for each effect by multiplying the three numbers (severity * occurrence * detection)
7. Use the RPNs to select high-priority failure modes
   • Prioritize actions so the highest RPNs are attended to first
   • Exception: Any failure with a severity rating of 10 must be worked on immediately because of impact on customers, even if it does not score a high RPN overall
8. Plan to reduce or eliminate the risk associated with high-priority failure modes
   • Identify potential causes of the selected failure modes
   • Develop recommended actions, assign responsible persons
   • Look for both:
     • Preventive action: Steps that reduce the likelihood of a problem occurring at all, focused on reducing/eliminating root causes prior to occurrence
     • Contingent action: Measures implemented to limit the damage caused by a potential problem should it occur; focused on achieving the goal in spite of difficulties
9. Carry out the plans. Document actions taken
10. Recompute RPN

    Process or Product Name:						Prepared by Page__of:__						Process/Product FMEA Form
    Responsible:						FMEA Date (Orig) ____(Rev)
    Process Step/Input	Potential Failure Mode	Potential Failure Effects	SEVERITY	Potential Causes	OCCURRENCE	Current Controls	DETECTION	RPN	Actions Recommended	Resp.	Actions Taken	SEVERITY	OCCURRENCE	DETECTION	RPN
    What is the process step and Input under investigation?	In what ways does the Key Input go wrong?	What is the impact on the Key Output Variables (Customer Requirements)		What causes the Key Input to go wrong?		What are the existing controls & procedures (inspection and test) that prevent either the cause of the Failure Mode?			What are the actions for reducing the occurrence of the cause, or improving detection?		What are the completed actions taken with the recalculated RPN?
    Fill carafe with water	Wrong amount of water	Coffee too strong or too weak	8	Faded level marks on carafe	4	Visual inspection	4	128	Replace Carafe	Mel	Carafe replaced	8	1	3	24
    			8	Water spilled from carafe	5	None	9	360	Train employees	Flo	Employees trained	8	2	7	112
    	Water too warm	Coffee too strong	8	Faucet not allowed to run and cool	8	Finger	4	256	Train employees	Flo	Employees trained	8	2	6	96
    			8	Employee not aware of new need for cool water	7	None	10	560	Train employees	Flo	Employees trained	8	1	8	64
    	Carafe not clean	Foreign objects in coffee	10	Carafe not washed	4	Visual inspection	4	160	Appoint inspector before storage	Alice	Vera is the new inspector	10	1	4	40
    		Bad taste	10	Carafe stored improperly	7	training	5	350	Create storage bin & train employees	Alice	New storage bin & employees trained	10	2	3	60

Tips on scoring systems 

• There are a wide variety of quantitative and qualitative scoring "anchors" that can form the basis for your rating scales. If you have difficulty, ask data experts to help you find examples you can use as models.
• Two rating scales typically used:
  • 1 to 5 scale makes it easier for the teams to decide on scores
  • 1 to 10 scale allows for better precision in estimates and a wider variation in scores; it is more commonly used

Pilot testing

Purpose

To identify practical problems and failures in a chosen solution so those can be addressed before full-scale implementation

Key characteristics of a pilot test

A pilot is a test of a selected solution. This type of test has the following properties:

• Performed on a small scale (limited in scope, budget, and/or time)
• Used to evaluate both the solution and the implementation of the solution
• Used to make full-scale implementation more effective
• Gives data about expected results and exposes issues in the implementation plan

The pilot should test if the process meets both the design specifications and customer expectations.

How to pilot a solution

Phase 1: Plan

• What needs to be piloted
• Where will the pilots be run
• Who will be involved
• When and for how long will the pilots run
• How will the pilots be conducted

Phase 2: Review design

Before conducting the pilot, review your plans to…

• Make sure all the elements of the design are complete
• Make sure all the elements are well integrated and that interfaces between different parts of the design are tight
• Identify possible failure points and areas of vulnerability to be tested in the pilot
• Evaluate predicted design capability (sigma or capability level)
• Review the pilot and implementation plans

  • Tips 

    • Establish review objectives and agenda in advance
    • Complete all prework in preparatory meetings
    • Keep documentation clear and consistent
    • Index documentation to make it easy to find and reference
    • Distribute documentation in advance of the meeting
    • Set follow-up procedures to confirm completion of action items
  • Outcomes
    • List of key issues raised, identifying who raised them
    • List of proposed actions for modifying the pilot, including who is responsible for making the changes
    • List of changes needed in documentation
    • Schedule of future meetings to assess completion of proposed actions
    • Schedule of future design review meetings as appropriate

Phase 3: Finalize design and implement

• Implement design changes identified in Phase 2. If necessary, perform another design review.
• Move to pilot testing and implementation.

  • Tips 

    • Carefully observe all activities, effects, and interactions during the pilot
    • Actively manage the implementation plan
    • Manage expectations and perceptions of customers, management, and staff
    • Continue the pilot long enough to establish reliable baseline performance data
    • Check for ripple effects and unintended consequences

Phase 4: Evaluate the test and verify results

• Use applicable statistical tools to properly evaluate design predictions, process control, and capabilities
• Celebrate success
  • Communicate small victories
  • Celebrate initial successes
  • Celebrate final results
• Improve on the design if the pilot demonstrates any weaknesses

Tips