An observed significance level printed on a computerized t test output is labeled 2-TAIL PROB, standing for two-tailed probability. This value
If you do not know which of the two groups should have the larger mean, that is what you have to ask. Differences in either direction cast doubt on the null hypothesis that in the population the two groups have the same means. If you do know in advance which group will have the larger mean if they
Suppose you know that a new drug for insomnia will either leave the length of time you need to fall asleep unchanged or decrease it. You take two random samples of people and perform an experiment. One group gets the drug, and the other gets a placebo (a fake drug just to make the subjects think they are being treated). Then you find the average time it takes each group to fall asleep. You calculate the difference between the two means, along with its standard error. To find out how often you would get a difference of this magnitude by chance when the drug and placebo are equally effective, you need only calculate the probability that you see a decrease at least as large as the one observed. You are confident that people treated with the drug will not take longer to fall asleep, so you decide in advance not even to test for that possibility.
Think back to the coin analogy. Suppose your friend tells you, as he is handing you the coin, that he
If you know in advance which of two means should be larger, you can convert the two-tailed significance level to a one-tailed level. All you do is divide the two-tailed probability by two. The result tells you the percentage of the t distribution in one of the tails.
In the previous example, we used a statistical technique called the t test to test the hypothesis that two groups have the same mean in the population. We did the following:
For each of the groups, we calculated the mean of the variable we were interested in comparing.
We subtracted one mean from the other to determine the difference between the two.
We calculated a t statistic by dividing the difference of the two sample means by its standard error.
We calculated the observed significance level. This told us how often we would expect to see a difference as large as the one we
If the observed significance level was small (less than .05), we rejected the hypothesis that the two means are equal in the population.
Otherwise, we did not reject the null hypothesis, and we did not accept it either. We remained undecided. That is because we did not know whether no difference was present or whether our sample was simply too small to detect the difference.
This procedure is the same for tests of most hypotheses:
You calculate the probability of observing a difference of a particular magnitude in the sample when the null hypothesis is true.
If this probability (the observed significance level) is small enough, you reject the null hypothesis.
If the probability is not small enough, you
The only part of this
Several groups have the same means.
No linear relationship exists among several variables.
If you make sure now that you understand the way hypothesis testing works, the rest of this book will be easy to understand.