# IS THE DIFFERENCE REAL?

## IS THE DIFFERENCE REAL?

So now the question is, "Is the difference real?" Usually when you conduct a study, you have some ideas that you want to explore. These ideas, often called hypotheses, typically involve comparisons of several groups, such as "Do men and women find life equally exciting?" "Does income differ between people who find life exciting and those who do not?" "Is a new machine making a real difference?"

Chances are if you compare two or more things, you are going to find some differences. After all, no two things are exactly the same. The question is not so much if things are different but rather, what can you make of the difference?

So far we have seen that different samples from the same population give different results. The real issue is, how much will they differ? How can you decide whether a difference in sample means can be attributed to their natural variability or to a real difference between groups in the population?

## EVALUATING A DIFFERENCE BETWEEN MEANS

How can you decide when a difference between two means is big enough for you to believe that the two samples are from a population with different means? It depends on how willing you are to be wrong. Look at Figure 5.1, which is the real distribution of differences for samples of size 20 from a distribution with a standard deviation of 15. (You can calculate the standard deviation of the distribution of differences. It is called the standard error of the difference.) Since the distribution is normal, you can find out what percentage of the samples falls into each of the intervals.

Figure 5.1: Theoretical distribution of differences of means.

The scale on the distribution is marked with the actual values and with "standardized" values, which are computed by dividing the differences by the standard error. Looking at standardized distances is convenient , since the percentage of cases within a standardized distance from the mean is always the same. For example, 34% of all samples are between zero and one standardized unit greater than the mean, and another 34% are between zero and one standardized unit less than the mean. If you always express your distances in standardized units, you can use the same normal distribution for evaluating the likelihood of a particular difference.

From Figure 5.1, you can see that about 13% of the time, you would expect to have at least a 7-point difference in the sample means when two population means are equal. Why? You just found out that the 7-point difference is 1.5 standard errors. Look on the figure to see what percentage of the differences is that big. You should look at the area to the right of +1.5 and the area to the left of -1.5. Since the distribution is symmetric, the two areas are equal. Each one is about 6.7% of the total, and together they make up a little over 13% of the total. So about 13% of differences in means are going to be as big as 1.5 standard errors (or 7 points) if the real difference in means is zero.

## WHY THE ENTIRE AREA?

You may wonder why you do not find the probability of getting a difference of just seven. Think of the following analogy. You are tired of the life of a poor student and have decided that the quickest (legal) way to upgrade your status is to marry rich. You settle on a definition of rich. Perhaps you need an income of \$250,000 a year. Now you want to see how likely it is that you can achieve your goal. You go to the university library and ask the reference librarian to find some facts. Would you ask about just the number of eligible singles of the opposite sex with incomes of \$250,000? No, you would ask for incomes of \$250,000 or more, since they all satisfy your criterion of richness. In evaluating your chances of marrying rich, you would include all incomes of \$250,000 or more. Similarly, when you are trying to decide whether seven is a likely outcome for a difference, your interest is not just in the number seven but in all differences that are at least that large.

Since both outcomes were possible and you did not know the outcome before you actually ran your experiment, when you evaluate the chances of seeing a difference at least as large as seven points, you have to look in both directions. Both of the extreme regions of the distribution are atypical. Sometimes, though, you can look in just one direction. It really depends on how you stated your initial hypothesis. If you hypothesized that you expect higher values you would be looking at the right tail, if lower you would be looking at the left tail of the distribution. This type of test is called a one-tailed test . Your decision to use a one-tailed test is a very important one because once you see the results you cannot go back and switch sides and apply the one-tailed test for a difference that is in the other direction. Use a one-tailed test only if you definitely expect one specific group to be higher. Otherwise, use a two-tailed test and look at both sides of the distribution.