Chapter 9: Design of Experiments

SETTING THE STAGE FOR DOE

Design of Experiments (DOE) is a way to efficiently plan and structure an investigatory testing program. Although DOE is often perceived to be a problem-solving tool, its greatest benefit can come as a problem avoidance tool. In fact, it is this avoidance that we emphasize in design for six sigma (DFSS).

This chapter is organized into nine sections. The user who is looking for a basic DOE introduction in order to participate with some understanding in a problem-solving group is urged to study and understand the first two sections or go back and review Volume V of this series. The remaining sections discuss more complex topics including problem avoidance in product and process design, more advanced experimental layouts, and understanding the analysis in more detail.

WHY DOE (DESIGN OF EXPERIMENTS) IS A VALUABLE TOOL

DOE is a valuable tool because:

1. DOE helps the responsible group plan, conduct, and analyze test programs more efficiently.

2. DOE is an effective way to reduce cost.

Usually the term DOE brings to mind only the analysis of experimental data. The application of DOE necessitates a much broader approach that encompasses the total process involved in testing. The skills required to conduct an effective test program fall into three main categories:

1. Planning/organizational

2. Technical

3. Analytical/statistical

The planning of the experiment is a critical phase. If the groundwork laid in the planning phase is faulty, even the best analytic techniques will not salvage the disaster. The tendency to run off and conduct tests as soon as a problem is found, without planning the outcome, should be resisted. The benefits from up-front planning almost always outweigh the small investment of time and effort. Too often, time and resources are wasted running down blind alleys that could have been avoided. Section 2 of this chapter contains a more detailed discussion of planning and the techniques used to ensure a well-planned experiment.

DOE can be a powerful tool in situations where the effect on a measured output of several factors, each at two or more levels, must be determined. In the traditional "one factor at a time" approach, each test result is used in a small number of comparisons. In DOE, each test is used in every comparison. A simplified example follows .

A problem-solving brainstorming group suspects 7 factors (named A, B, C, D, E, F, and G), each at two levels (level 1 and level 2), of influencing a critical, measurable function of the design. The group wants to determine the best settings of these factors to maximize the measured test results ” see Table 9.1. Two evaluations (a and b) are run at each test configuration rather than a single evaluation in order to attain a higher confidence in the difference between factor levels (this assumes no need for a "tie breaker"). The group makes comparisons as shown in Table 9.2. Sixteen total tests are run, and four tests are used to determine the difference between levels for each factor. The best combination of factors is (1, 2, 1, 2, 2, 1, 1) for factors A through G.

Table 9.1: One Factor at a Time

The group tests configurations containing the following combinations of the factors:
Test Number	Level of Factor (1 and 2 Indicate the Different Levels)							Results
	A	B	C	D	E	F	C	a	b
1	1	1	1	1	1	1	1	271.4	266.3
2	2	1	1	1	1	1	1	215.0	211.2
3	1	2	1	1	1	1	1	275.3	271.1
4	1	2	2	1	1	1	1	235.2	231.5
5	1	2	1	2	1	1	1	296.6	301.6
6	1	2	1	2	2	1	1	305.2	301.1
7	1	2	1	2	1	2	1	278.8	275.3
8	1	2	1	2	1	1	2	251.9	254.3

Table 9.2: Test Numbers for Comparison

	Test Numbers Used to Determine:		Difference Level 1 - Level 2
Factor	Level 1	Level 2
A	1a, 1b	2a, 2b	55.8
B	1a, 1b	3a, 3b	-4.4
C	3a, 3b	4a, 4b	39.9
D	3a, 3b	5a, 5b	-25.7
E	5a, 5b	6a, 6b	-4.3
F	6a, 6b	7a, 7b	26.1
G	6a, 6b	8a, 8b	50.1

However, using DOE the group runs test configurations as shown in Table 9.3. The group makes comparisons as shown in Table 9.4. Eight total tests are run, and eight tests are used to determine the difference between levels for each factor. This can be done because each level of every factor equally impacts the determination of the average response at all levels of all of the other factors (i.e., of the four tests run at A = 1, two were run at B = 1 and two were run at B = 2; this is also true of the four tests run at A = 2). This relationship is called orthogonality. This concept is very important, and the reader should work through the relationships between the levels of at least two other factors to better understand the use of orthogonality in this testing matrix. The best level is [1, (1 or 2), 1, 2, (1 or 2), 1, 1] for A through G. Factors B and E are not significant and may be set to the least expensive level.

Table 9.3: The Group Runs Using DOE Configurations

Test Number	Level of Factor (1 and 2 Indicate the Different Levels)
	A	B	C	D	E	F	C	Result
1	1	1	1	1	1	1	1	270.7
2	1	1	1	2	2	2	2	223.8
3	1	2	2	1	1	2	2	158.2
4	1	2	2	2	2	1	1	263.1
5	2	1	2	1	2	1	2	129.3
6	2	1	2	2	1	2	1	175.1
7	2	2	1	1	2	2	1	195.4
8	2	2	1	2	1	1	2	194.6

Table 9.4: Comparisons Using DOE

Factor	Test Numbers Used to Determine		Difference Level 1 - Level 2
	Level 1	Level 2
A	1, 2, 3, 4	5, 6, 7, 8	55.4
B	1, 2, 5, 6	3, 4, 7, 8	-3.1
C	1, 2, 7, 8	3, 4, 5, 6	39.7
D	1, 3, 5, 7	2, 4, 6, 8	-25.8
E	1, 3, 6, 8	2, 4, 5, 7	-3.3
F	1, 4, 5, 8	2, 3, 5, 8	26.3
G	1, 4, 6, 7	2, 3, 5, 8	49.6

For a comparison of the two methods , see Table 9.5. Half as many tests are required using a DOE approach and the estimate at each level is better (four tests per factor level versus two). This is almost like getting something for nothing. The only thing that is required is that the group plan out what is to be learned before running any of the tests. The savings in time and testing resources can be significant. Direct benefits include reduced product development time, improved problem correction response, and more satisfied customers. And that is exactly what DFSS should be aiming at.

Table 9.5: Comparison of the Two Means

	Number of Tests	Estimate at the Best Levels	Confidence Interval at 90% Confidence
One factor at a time	16	301.1	± 3.7
DOE	8	299.6	± 3.3

 

This approach to DOE is also very flexible and can accommodate known or suspected interactions and factors with more than two levels. A properly structured experiment will give the maximum amount of information possible. An experiment that is less well designed will be an inefficient use of scarce resources.

TAGUCHI'S APPROACH

Here it is appropriate to summarize Dr. Taguchi's approach, which is to minimize the total cost to society. He uses the "Loss Function" (Section 4) to evaluate the total cost impact of alternative quality improvement actions. In Dr. Taguchi's view, we all have an important societal responsibility to minimize the sum of the internal cost of producing a product and the external cost the customer incurs in using the product. The customer's cost includes the cost of dissatisfaction. This responsibility should be in harmony with every company's objectives when the long-term view of survival and customer satisfaction is considered . Profits may be maximized in the short run by deceiving today's customers or trading away the future.

Traditionally, the next quarter's or next year's "bottom line" has been the driving force in most corporations. Times have changed, however. Worldwide competition has grown, and customers have become more concerned with the total product cost. In this environment, survival becomes a real issue, and customer satisfaction must be a part of the cost equation that drives the decision process.

Dr. Taguchi uses the signal-to-noise (S/N) ratio as the operational way of incorporating the loss function into experimental design. Experiment S/N is analogous to the S/N measurement developed in the audio/electronics industry. S/N is used to ensure that designs and processes give desired responses over different conditions of uncontrollable "noise" factors. S/N is introduced in Section 4 and developed in examples in later sections.

There are three basic types of product design activity in Dr. Taguchi's approach:

1. System design

2. Parameter design

3. Tolerance design

System design involves basic research to understand nature. System design involves scientific principles, their extension to unknown situations, and the development of highly structured basic relationships. Parameter and tolerance design involves optimizing the system design using empirical methods. Taguchi's methods are most useful in parameter and tolerance designs. The rest of this chapter will discuss these applications.

Parameter design optimizes the product or process design to reach the target value with minimum possible variability with the cheapest available components . Note the emphasis on striving to satisfy the requirements in the least costly manner. Parameter design is discussed in Section 8.

Tolerance design only occurs if the variability achieved with the least costly components is too large to meet product goals. In tolerance design, the sensitivity of the design to changes in component tolerances is investigated. The goal is to determine which components should be more tightly controlled and which are not as crucial. Again, the driving force is cost. Tolerance design is discussed in Section 9.

Problem resolution might appear to be another type of product design. If targets are set correctly, however, and parameter and tolerance design occur, there will be little need for problem resolution. When problems do arise, they are attacked using elements of both parameter and tolerance design, as the situation warrants .

MISCELLANEOUS THOUGHTS

A tremendous opportunity exists when the basic relationships between components are defined in equation form in the system design phase. This occurs in electrical circuit design, finite element analysis, and other situations. In these cases, once the equations are known, testing can be simulated on a computer and the "best" component values and appropriate tolerances obtained. It might be argued that the true best values would not be located using this technique; only the local maxima would be obtained. The equations involved are generally too complex to solve to the true best values using calculus. Determining the local best values in the region that the experienced design engineer considers most promising is generally the best available approach. It definitely has merit over choosing several values and solving for the remaining ones. The cost involved is computation time, and the benefit is a robust design using the widest possible tolerances.

Those readers who have some experience in classical statistics may wonder about the differences between the classical and Taguchi approaches. Although there are some operational differences, the biggest difference is in philosophical emphasis ” see Volume V of this series. Classical statistics emphasizes the producer's risk. This means a factor's effect must be shown to be significantly different from zero at a high confidence level to warrant a choice between levels. Taguchi uses percent contribution as a way to evaluate test results from a consumer's risk standpoint. The reasoning is that if a factor has a high percent contribution, more often than not it is worth pursuing. In this respect, the Taguchi approach is less conservative than the classical approach. Dr. Taguchi uses orthogonal arrays extensively in his approach and has formulated them into a "cookbook" approach that is relatively easy to learn and apply. Classical statistics has several different ways of designing experiments including orthogonal arrays. In some cases, another approach may be more efficient than the orthogonal array. However, the application of these methods may be complex and is usually left to statisticians. Dr. Taguchi also approaches uncontrollable "noise" differently. He emphasizes developing a design that is robust over the levels of noise factors. This means that the design will perform at or near target regardless of what is happening with the uncontrollable factors. Classical statistics seeks to remove the noise factors from consideration by "blocking" the noise factors.

In certain cases, the approaches Taguchi recommends may be more complicated than other statistical approaches or may be questioned by classical statisticians. In these cases, alternative approaches are presented as supplemental information at the end of the appropriate section. Additional analysis techniques are also presented in section supplements.

The reader is encouraged to thoroughly analyze the data using all appropriate tools. Incomplete analysis can result in incorrect conclusions.