# TAILS AND SIGNIFICANCE TESTS

## TAILS AND SIGNIFICANCE TESTS

An observed significance level printed on a computerized t test output is labeled 2-TAIL PROB, standing for two-tailed probability. This value tells you the probability that you would see in either direction a difference at least as large as the one you would observe if no difference existed in the population. Either the first group has a mean larger than that of the second group by at least the observed size , or the second group has a mean larger than that of the first group by at least the observed size.

If you do not know which of the two groups should have the larger mean, that is what you have to ask. Differences in either direction cast doubt on the null hypothesis that in the population the two groups have the same means. If you do know in advance which group will have the larger mean if they differ , then you use a one-tailed significance level.

Suppose you know that a new drug for insomnia will either leave the length of time you need to fall asleep unchanged or decrease it. You take two random samples of people and perform an experiment. One group gets the drug, and the other gets a placebo (a fake drug just to make the subjects think they are being treated). Then you find the average time it takes each group to fall asleep. You calculate the difference between the two means, along with its standard error. To find out how often you would get a difference of this magnitude by chance when the drug and placebo are equally effective, you need only calculate the probability that you see a decrease at least as large as the one observed. You are confident that people treated with the drug will not take longer to fall asleep, so you decide in advance not even to test for that possibility.

Think back to the coin analogy. Suppose your friend tells you, as he is handing you the coin, that he suspects the coin is biased. Perhaps his wife always uses it to settle disputes, and she always bets heads and wins. You have a pretty good suspicion that the coin is biased in favor of heads and no reason at all to suspect bias in favor of tails. If you toss the coin ten times, and it comes up all heads, you just want to know what the probability is that a fair coin comes up ten heads out of ten flips . You do not worry that it might have come up with ten tails, since the only situations that will cause you to doubt your coin are excesses of heads.

If you know in advance which of two means should be larger, you can convert the two-tailed significance level to a one-tailed level. All you do is divide the two-tailed probability by two. The result tells you the percentage of the t distribution in one of the tails.

## THE HYPOTHESIS-TESTING PROCESS

In the previous example, we used a statistical technique called the t test to test the hypothesis that two groups have the same mean in the population. We did the following:

• For each of the groups, we calculated the mean of the variable we were interested in comparing.

• We subtracted one mean from the other to determine the difference between the two.

• We calculated a t statistic by dividing the difference of the two sample means by its standard error.

• We calculated the observed significance level. This told us how often we would expect to see a difference as large as the one we observed if no difference existed between the groups in the population.

• If the observed significance level was small (less than .05), we rejected the hypothesis that the two means are equal in the population.

• Otherwise, we did not reject the null hypothesis, and we did not accept it either. We remained undecided. That is because we did not know whether no difference was present or whether our sample was simply too small to detect the difference.

This procedure is the same for tests of most hypotheses:

• You formulate a null hypothesis and its alternative.

• You calculate the probability of observing a difference of a particular magnitude in the sample when the null hypothesis is true.

• If this probability (the observed significance level) is small enough, you reject the null hypothesis.

• If the probability is not small enough, you remain undecided.

The only part of this ritual that changes for different situations is the actual statistic used to evaluate the probability of the observed difference. In the chapters that follow, we will use different types of statistics to test hypotheses that have these characteristics:

• Variables are independent.

• Several groups have the same means.

• No linear relationship exists among several variables.

If you make sure now that you understand the way hypothesis testing works, the rest of this book will be easy to understand.