OBSERVED SIGNIFICANCE LEVELS


OBSERVED SIGNIFICANCE LEVELS

When your observed significance level is small, its interpretation is fairly straightforward: the two means seem to be unequal in the population. The observed significance level tells you the probability that the observed difference could be due to chance. The observed significance level is the probability that your sample could show a difference at least as large as the one that you observed if the means are really equal.

So what is a small significance level? Most of the time, significance levels are considered small if they are less than .05; sometimes, if they are less than .01. Rather than just rejecting or not rejecting the null hypothesis, look at the actual significance level as well. An observed significance level of .06 is not the same as an observed significance level of .92, though both may not be statistically significant. When reporting your results, give the exact observed significance level. It will help the reader evaluate your results. Treat the observed significance level as a guide to whether or not the difference could be due to chance alone.

If your observed significance level is too large to reject the hypothesis that the means are equal, more than one explanation is possible. The first explanation is that no difference may exist between the two means or that it may be so small that you cannot detect it. If the true difference is very small, it may not matter that you cannot find it. Who really cares about a tiny difference (such as a difference in annual income of ten dollars)? Little, if anything, is lost by your failure to establish such tiny differences.

The second explanation is more troublesome . Perhaps an important difference does exist, and you cannot find it. This can occur if the sample size is small.

If you flip a coin only twice, you cannot establish whether it is fair. A fair coin has a 50% chance of coming up heads twice or tails twice in two flips and a 50% chance of coming up with one of each. Any outcome that you see is consistent with the coin's being fair. As the number of flips increases , so does your ability to detect differences. To detect a small difference, you need a big sample so that the difference would clearly be outside the expected degree of sample variation.

The variability of the responses (in the population) also affects your ability to detect differences. If the observations vary a great deal, the sample means will vary a lot as well. Even large differences in observed means can be attributed to variability among the samples.

To wrap up all of this: if you do not find evidence to reject the hypothesis that two means are equal in the population, one of two possibilities is true:

  • The means are equal or very similar.

  • The means are unequal, but you cannot detect the difference because of small sample size, large variability, or both.




Six Sigma and Beyond. Statistics and Probability
Six Sigma and Beyond: Statistics and Probability, Volume III
ISBN: 1574443127
EAN: 2147483647
Year: 2003
Pages: 252

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net