This section discusses:
What dynamic test situations are
How a test plan should be set up in a dynamic situation
The analysis of test data
In many instances, the experimenter knows that the optimum response for a system changes with levels of an input signal. Using the signal-to-noise techniques described in the previous sections would yield incorrect results. These techniques emphasize repeatability across all levels of the noise factors. In a dynamic situation, the experimenter wants different responses depending upon the level of an input signal factor. Two examples are:
If two or more length measurement devices are compared, the standard lengths to be measured become signal factor levels for comparison. The experimenter would want a measurement device that gives a reading that is relative to the different standard lengths measured and is repeatable at each standard measured.
Several factors are to be included in an experiment to determine the combination that optimizes vehicle braking distance. The tests are run at two different vehicle speeds. The vehicle speed would be treated as a signal factor. The experimenter would want the braking distance to be repeatable at each vehicle speed and reflect the customers' needs and desires for braking distance at each vehicle speed. These needs and desires would not be the same for all vehicle speeds. (Note: It might seem that the goal should be to minimize the braking distance at each vehicle speed; however, if the braking were too abrupt, the driver might lose control of the vehicle.)
The analysis of dynamic test data can be complicated. The following conditions exist in the examples that follow. If these conditions are not present in a dynamic experiment, help from a statistician should be sought before setting up the experiment.
Signal factors will be assigned to an outer array in the experimental setup.
The signal factor(s) will have either two or three levels.
If there are three levels for a signal factor, the intervals between the adjacent levels will be equal.
The experimental test includes either noise factors in an outer array or repetitions so that for each inner array control factor setup, two or more runs are made for each signal factor.
The general approach that is used to analyze the data is:
The test results for each inner array setup (test number) are separately analyzed using analysis of variance (ANOVA). The ANOVA table for these analyses will be shown in a typical format as Table 9.45.
Source | df | SS | V |
---|---|---|---|
Signal | df s | SS s | V s |
error | df 1 - df s | SS 1 - SS s | V e |
Total | df 1 | SS 1 |
A nominal-the-best signal-to-noise ratio is calculated for each inner array setup from these ANOVA tables as follows :
S / N = 10 log 10
where r = the number of data in each level of the signal factor for this inner array setup; s = 0.5 if the signal factor has two levels or s = 2.0 if the signal factor has three levels; and h = the interval between the adjacent levels of the signal factor.
The calculated S/N ratio for each inner array setup is then used in a nominal-the-best (NTB) S/N analysis of variance to determine which control factor settings should be used to reduce variability and which should be used to tune the response to the desired output.
The application of these steps will be developed more fully through the following examples.
Two different types of automatic optical measurement devices are to be compared. Two orientations of the devices are possible, horizontal or vertical. These are assigned to an L4 inner array as follows:
Factor | Column |
---|---|
Type (T) | 1 |
Orientation (O) | 2 |
T — O Interaction | 3 |
Items with two different surface finishes will be measured by the devices. Surface finish (F) will be a noise factor. Two standard lengths of 10 and 20 mm will be evaluated. These will be the two levels of the signal factor (S). The test matrix and test results for the experiment are shown in Table 9.46.
Test Number | Test Matrix | S 1 | S 2 | NTB S/N | ||||
---|---|---|---|---|---|---|---|---|
T | O | T — O | F 1 | F 2 | F 1 | F 2 | ||
1 | 1 | 1 | 1 | 9.8 | 9.7 | 20.4 | 20.2 | 19.33 |
2 | 1 | 2 | 2 | 10.2 | 9.9 | 20.3 | 20.1 | 14.94 |
3 | 2 | 1 | 2 | 9.6 | 9.9 | 19.6 | 20.0 | 12.05 |
4 | 2 | 2 | 1 | 10.2 | 9.8 | 19.7 | 19.5 | 12.65 |
For test number 1, the S/N ratio is calculated from the ANOVA table for just the runs in test number 1.
S | F | Test Result |
---|---|---|
1 | 1 | 9.8 |
1 | 2 | 9.7 |
2 | 1 | 20.4 |
2 | 2 | 20.2 |
The ANOVA table for these data is shown in Table 9.47. The S/N ratio is calculated as:
S /N = 10 log 10
S / N = 10 log 10
S / N = 19.33
Source | df | SS | V |
---|---|---|---|
Signal | 1 | 111.303 | 111.303 |
error | 2 | 0.026 | 0.013 |
Total | 3 | 111.328 |
An S/N ratio for each of the other test setups is calculated in a similar manner. These S/N ratios are then analyzed using the S/N ratio as a single response for each test setup ” see Table 9.48. The level averages for the data are:
O 1 | O 2 | Overall | |
---|---|---|---|
T 1 | 19.33 | 14.98 | 17.16 |
T 2 | 12.05 | 12.65 | 12.35 |
Overall | 15.69 | 13.82 | 14.76 |
Source | df | SS | MS | F Ratio | S' | % |
---|---|---|---|---|---|---|
T | 1 | 20.794 | 20.794 | 4.311 | 15.970 | 52.46 |
O | 1* | 4.494 | 4.494 | 14.471 | 47.54 | |
T — O | 1* | 5.153 | 5.153 | |||
Error | ” | 9.647 | 4.824 | |||
(pooled error) | 2 | |||||
Total | 3 | 30.647 | 10.147 |
Inspection of the data shows that the setting of T that gives the highest S/N is level 1. Although there are not enough test setups to allow the statistical identification of level 1 of factor O as the optimum, the data suggests that orientation 1 might work the best with device 1 and should be further investigated.
The level averages of the raw data are shown in Table 9.49.
S 1 | S 2 | Overall | |
---|---|---|---|
T 1 | 9.90 | 20.25 | 15.08 |
T 2 | 9.88 | 19.70 | 14.79 |
O 1 | 9.75 | 20.05 | 14.90 |
O 2 | 10.03 | 19.90 | 14.97 |
Average of S | 9.89 | 19.98 | |
Overall | 14.93 | ||
Average at T 1 O 1 | 15.03 |
The predicted averages are calculated using the techniques given in Section 5 using the interaction of T 1 and O 1 (assumed) as the optimum setting.
for S = 1
for S = 2
Note that the readings at the optimum do not average out to the standard exactly. This assumes that the output reading can be calibrated to reflect the standard measured. The emphasis in the approach is to provide readings with low variability at each standard level output.
This example was very simple and it may seem that the ANOVA was not really necessary. In many cases, the inner array will be more complicated than an L4, and the technique shown in this example will help the experimenter make an informed choice.
The effect of several factors on vehicle braking distance is to be investigated. The control factors to be investigated are assigned to an L8 orthogonal inner array as follows:
Column | |
---|---|
1 | A ” Content of material "Z" in the brake pads |
2 | B ” Content of material "Y" in the brake rotors |
3 | A — B interaction |
4 | C ” Hydraulic fluid type |
5 | Unassigned |
6 | Unassigned |
7 | D ” Brake pad design |
Noise and signal factors are assigned to an L8 outer array as follows:
Column | |
---|---|
1 | S ” Vehicle speed (30 mi/h or 60 mi/h) |
2 | T ” Tire size |
3 | Unassigned |
4 | P ” Pavement type (asphalt or concrete) |
5 | Unassigned |
6 | Unassigned |
7 | Unassigned |
In this example, vehicle speed is a signal factor. It is not possible that the braking distance would be the same when starting from 30 mi/h vs. 60 mi/h and therefore, different responses are expected. The experimenter has determined through market research that for this type of vehicle, the customer would prefer that the braking distance be 35 feet from 30 mi/h and 130 feet from 60 mi/h.
The test setup and results are shown in Table 9.50.
A | S 1 | S 2 | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
X | T 1 | T 2 | T 1 | T 2 | NTB S/N | ||||||||
A | B | B | C | D | P1 | P 2 | P1 | P 2 | P1 | P 2 | P1 | P 2 | |
1 | 1 | 1 | 1 | 1 | 39.9 | 40.6 | 40.4 | 40.6 | 140.9 | 141.2 | 140.7 | 139.6 | 15.73 |
1 | 1 | 1 | 2 | 2 | 42.5 | 42.8 | 42.4 | 42.7 | 143.0 | 142.4 | 141.1 | 142.8 | 14.62 |
1 | 2 | 2 | 1 | 2 | 45.0 | 41.3 | 41.0 | 44.8 | 141.2 | 143.1 | 143.4 | 142.7 | 5.90 |
1 | 2 | 2 | 2 | 1 | 40.7 | 39.7 | 40.5 | 40.7 | 141.3 | 140.7 | 140.9 | 139.7 | 15.25 |
2 | 1 | 2 | 1 | 2 | 39.4 | 40.1 | 39.7 | 38.1 | 139.9 | 139.7 | 141.1 | 139.7 | 12.74 |
2 | 1 | 2 | 2 | 1 | 37.5 | 37.3 | 37.6 | 37.3 | 137.6 | 138.0 | 137.1 | 137.2 | 20.64 |
2 | 2 | 1 | 1 | 1 | 36.0 | 38.4 | 36.9 | 37.9 | 135.1 | 139.3 | 138.4 | 136.1 | 6.58 |
2 | 2 | 1 | 2 | 2 | 39.6 | 40.4 | 40.5 | 40.3 | 139.7 | 140.1 | 142.0 | 138.5 | 9.89 |
The unassigned columns are not shown to conserve space and to make the table more presentable. The outer array is also shown somewhat differently, with column 1 (factor S) at the top, column 2 (factor T) in the middle, and column 4 (factor P) at the bottom. Although this arrangement can be used to present the data, the unassigned columns should be added back to the arrays to aid the experimenter's understanding of the analysis and the application of the inner and outer L8 orthogonal arrays.
For the first test setup, the S/N ratio is calculated from the ANOVA table for the data in the first row. The ANOVA table for these data is:
Source | df | SS | V |
---|---|---|---|
Signal | 1 | 20090.101 | 20090.101 |
Error | 6 | 1.786 | 0.298 |
Total | 7 | 20091.888 |
The S/N ratio is calculated as follows:
S/N = 10 log 10
S / N = 10 log 10
S / N = 15.73
An S/N ratio for each of the other test setups is calculated in a similar manner. These S/N ratios are then analyzed using the S/N ratio as a single response for each test setup ” see Table 9.51.
Source | df | SS | MS | F Ratio | S ' | % |
---|---|---|---|---|---|---|
A | 1* | 0.340 | 0.340 | |||
B | 1 | 85.217 | 85.217 | 44.453 | 83.300 | 47.87 |
A — B | 1 | 7.431 | 7.431 | 3.876 | 5.514 | 3.17 |
C | 1 | 47.288 | 47.288 | 24.668 | 45.371 | 26.08 |
Unassigned | 1* | 1.103 | 1.103 | |||
Unassigned | 1* | 4.307 | 4.307 | |||
D | 1 | 28.313 | 28.313 | 14.769 | 26.396 | 15.17 |
Error | ||||||
(pooled error) | 3 | 5.750 | 1.917 | 13.418 | 7.71 | |
Total | 7 | 173.998 | 24.857 |
The ANOVA table indicates that factors B, C, D, and the interaction of factors A and B are significant. The level averages for these factors are:
B 1 | B 2 | ||||||
---|---|---|---|---|---|---|---|
A 1 | A 2 | A 1 | A 2 | C 1 | C 2 | D 1 | D 2 |
15.18 | 16.69 | 10.58 | 8.24 | 10.24 | 15.10 | 14.55 | 10.79 |
The ANOVA table and the level averages indicate that B 1 , C 2 and D l are the optimal choices from an S/N standpoint. These are the factor choices that should result in the minimum variance of the response.
An analysis of the raw data would identify the signal factor as the most significant contributor to the variation of the data. However, this information is not useful. To increase the ability of the analysis to clearly show the significant control factors, the target braking distance for each of the signal factor levels is subtracted from all of the data collected at that signal factor level. This reduces the percent level of contribution of the signal factor and increases the percent level of contribution of the control factors while maintaining their relative order of contribution. This transformation makes the effects of the control factors more visible but does not affect their significance. The transformed data are shown in Table 9.52.
A | S | S 2 | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
X | T 1 | T 2 | T 1 | T 2 | ||||||||
A | B | B | C | D | P 1 | P 2 | P 1 | P 2 | P 1 | P 2 | P 1 | P 2 |
1 | 1 | 1 | 1 | 1 | 4.9 | 5.6 | 5.4 | 5.6 | 10.9 | 11.2 | 10.7 | 9.6 |
1 | 1 | 1 | 2 | 2 | 7.5 | 7.8 | 7.4 | 7.7 | 13.0 | 12.4 | 11.1 | 12.8 |
1 | 2 | 2 | 1 | 2 | 10.0 | 6.3 | 6.0 | 9.8 | 11.2 | 13.1 | 13.4 | 12.7 |
1 | 2 | 2 | 2 | 1 | 5.7 | 4.7 | 5.5 | 5.7 | 11.3 | 10.7 | 10.9 | 9.7 |
2 | 1 | 2 | 1 | 2 | 4.4 | 5.0 | 4.7 | 3.1 | 9.9 | 9.7 | 11.1 | 9.7 |
2 | 1 | 2 | 2 | 1 | 2.5 | 2.3 | 2.6 | 2.3 | 7.6 | 8.0 | 7.1 | 7.2 |
2 | 2 | 1 | 1 | 1 | 1.1 | 3.4 | 1.9 | 2.9 | 5.1 | 9.3 | 8.4 | 6.1 |
2 | 2 | 1 | 2 | 2 | 4.6 | 5.4 | 5.5 | 5.3 | 9.7 | 10.1 | 12.0 | 8.5 |
The ANOVA table for these data is shown in Table 9.53.
Source | df | SS | MS | F Ratio | S' | % |
---|---|---|---|---|---|---|
A | 1 | 150.369 | 150.369 | 155.340 | 149.401 | 21.56 |
B | 1* | 1.56E-4 | 1.56E-4 | |||
A — B | 1* | 0.473 | 0.473 | |||
C | 1* | 0.833 | 0.833 | |||
Unassigned | 1* | 2.600 | 2.600 | |||
Unassigned | 1* | 2.213 | 2.213 | |||
D | 1 | 101.758 | 101.758 | 105.122 | 100.790 | 14.55 |
S | 1 | 382.691 | 382.691 | 395.342 | 381.723 | 55.09 |
T | 1* | 0.083 | 0.083 | |||
Unassigned | 1* | 0.170 | 0.170 | |||
P | 1* | 0.375 | 0.375 | |||
Unassigned | 1* | 1.658 | 1.658 | |||
Unassigned | 1* | 0.508 | 0.508 | |||
Unassigned | 1* | 2.520 | 2.520 | |||
A — S | 1* | 0.083 | 0.083 | |||
D — S | 1* | 0.098 | 0.098 | |||
Error | 47* | 16.441 | 0.988 | 60.959 | 8.80 | |
(pooled error) | 30 | 58.055 | 0.968 | |||
Total | 63 | 692.874 | 10.998 |
The interactions between all the columns of the inner array and all the columns of the outer array are available for investigation. For this example, only the A — S and D — S interactions are investigated to give an indication of whether factors A and D "behave" consistently at the two levels of the signal factor. The analysis indicates that control factors A and D are significant contributors to the variation of the data. The difference in responses between the two levels of these factors is independent of the signal level. The analysis also identified the signal factor, S, as an important contributor to the data variation. This, of course, was already known.
The level averages are:
S 1 | S 2 | Overall | |
---|---|---|---|
A 1 | 6.58 | 11.54 | 9.06 |
A 2 | 3.59 | 8.41 | 6.00 |
D 1 | 3.86 | 8.68 | 6.27 |
D 2 | 6.30 | 11.28 | 8.79 |
Average of S | 5.08 | 9.98 | |
Overall Average | 7.53 |
The predicted averages are calculated using the techniques given in Section 5 using A 2 and D 1 as the optimum settings and adding the values that were subtracted prior to the ANOVA.
for S =1
for S = 2
Factor B should be set to level 1 and factor C should be set to level 2 to maximize the S/N ratio.
Since the target values were not obtained at the optimum settings, the experimenter must either continue to investigate other ways of reducing the stopping distance or accept the consequences of failing to fully satisfy the customer's requirements.
Let us close this section with a discussion of the two examples dealing with the NTB II S/N approach. The NTB S/N ratio for a dynamic situation is:
NTB S / N = 10 log 10
This equation was explained earlier. Using the same terminology, the NTB II S/N = NTB II S/N = -10 log (V e ) which equals -20 log (error standard deviation).
The calculations for the NTB S/Ns were discussed earlier. The same steps are followed for the NTB approach until the final S/N calculation. The two sets of S/N ratios are contrasted below:
Test Number | NTB S/N | NTB II S/N |
---|---|---|
1 | 19.33 | 19.03 |
2 | 14.94 | 14.88 |
3 | 12.05 | 12.04 |
4 | 12.65 | 13.01 |
When the NTB II S/N ratios were analyzed, the ANOVA table and the interpretation of the level averages were essentially the same as those for the NTB S/N.
The calculations for the NTB S/N were discussed earlier. The NTB II analysis had suggested that the standard deviation of the data might be related to the average of the data. In other words, the spread of the stopping distances might be greater at standard one (30 mi/h) than at standard two (60 mi/h). Using the procedure given in pages 395-397, the averages and standard deviations were compared as follows:
For each vehicle speed, the average stopping distance and the standard deviation were calculated (16 averages and 16 standard deviations total).
The log (standard deviation) was plotted versus the log (average).
The slope was estimated to be in the range of 0.2 to 0.3 with large scatter in the data. By comparing this value to Item 4 on page 396, it was determined that there was not a strong need to transform the data.
The NTB II S/N ratios were calculated for the untransformed data. The two sets of S/N ratios are compared below:
Test Number | NTB S/N | NTB II S/N |
---|---|---|
1 | 15.73 | 5.26 |
2 | 14.62 | 4.19 |
3 | 5.90 | -4.51 |
4 | 15.25 | 4.62 |
5 | 12.74 | 2.21 |
6 | 20.64 | 10.18 |
7 | 6.58 | -3.87 |
8 | 9.89 | -0.56 |
When the NTB II S/N ratios were analyzed, the ANOVA table and the interpretation of the level averages were essentially the same as those for the NTB S/N. The reader is encouraged to run the analysis to confirm this. The analysis of the raw data did not change. The conclusions also remained the same as before.
For these two examples, the NTB and NTB II methods give equivalent results. However, this does not prove equivalency of the methods. On other sets of data, differences in the results obtained have been demonstrated. Of the two methods, the NTB II approach is easier to understand since maximizing the NTB II S/N is the same as minimizing the error variance for the chosen combination of factor levels. (The experimenter should always analyze the data completely, plot the data, compare the results to the data plots, and run confirmation tests.)