Hack 34. Make Wise Medical Decisions

Medical tests provide diagnostic screening information that is often misunderstood by patients and, sometimes, even by doctors. Understanding the probability characteristics called "sensitivity" and "specificity" can provide a more accurate and (sometimes) reassuring picture.

As a consumer of medical information, you have to make decisions about behavior, treatment, seeking a second opinion, and so on. You likely rely on medical informationnewspaper stories, your doctor's advice, test resultsto make those decisions. However, much of the medical information you get from your doctor has a known amount of error. This is especially true about diagnostic test results that indicate the probability that you have a certain condition.

This hack is all about using information about the characteristics of those medical tests to get a more accurate picture of reality and, hopefully, make better decisions about treatment.

Statistics and Medical Screening

To use medical test information wisely, we have to learn just a bit about what the concept of accuracy means for these tests. The four possible outcomes of medical tests, in terms of accuracy, are shown in Table 3-12.

Table Possible medical test outcomes

	Patient actually has the condition (A)	Patient actually does not have condition (B)
Test result indicates patient has condition	True positive (score is correct)	False positive (score is wrong)
Test result indicates patient does not have condition	False negative(score is wrong)	True negative (score is correct)

The reliability [Hack #6] of medical screening tests is summarized by two proportions called sensitivity and specificity. Essentially, those who rely on these tests are concerned with three questions of accuracy:

• If a person has the disease, how likely is the person to score a positive test result? This likelihood is sensitivity. Of those people in column A, what percent will receive a positive test result?

• If the person does not have the disease, how likely is the person to score a negative test result? This likelihood is specificity. Of those people in column B, what percent will receive a negative test result?

• If a person scores a positive test result, how likely is the person to have the disease? From the patient's perspective, this is the ultimate question, and it can be thought of as the basic validity concern with these tests. Doctor, can I trust these test results, or could there be some mistake?

Notice in Table 3-12 that there are different people in columns A and B. People with the disease are in column A and people without the disease are in column B. If you are in column A, you cannot score a false positive on the test, because a positive result is correct. If you are in column B, you cannot score a false negative, because a negative result is correct.

Which column anyone is in depends on the natural distribution of the disease. The chance that someone will be in column A (the chance the person actually has the disease) depends on the base rate of the disease. If 5 percent of the population has the disease, 5 percent of the population would find themselves in column A.

Understanding Breast Cancer Screening

Breast cancer is an example of a serious condition for which there are diagnostic screening tests. Breast cancer screening begins with a mammogram test. A positive result on this test results in further testing: another mammogram, ultrasound, or biopsy.

We are first interested in answering the questions regarding the sensitivity and specificity of breast cancer screening. With that information and knowledge of the base rate for breast cancer, we can answer the most important question:

If a woman scores a positive test result, how likely is she to have breast cancer?

By asking your doctor or doing some research, you might discover that sensitivity for mammograms is about 90 percent. Specificity is about 92 percent.

The exact sensitivity and specificity for breast cancer screenings change over time as different populations take the test. Younger women now have mammograms more commonly than in the past, and the test is less sensitive and less specific for younger women. Of course, you should check with a physician or expert for current levels of precision.

Table 3-13 shows those numbers in the layout used in Table 3-12. Because columns A and B must both independently add to 100 percent, we can also estimate the rate of false negatives and rate of false positives.

Table Theoretical mammogram results for 10,000 women

	Patient actually has breast cancer (A) N=120	Patient actually does not have breast cancer (B) N=9,880
Mammogram indicates cancer	Sensitivity90 percentN=108	False positives8 percentN=790
Mammogram does not indicate cancer	False negatives10 percentN=12	Specificity92 percentN=9,090

Table 3-13 also shows the outcomes for 10,000 hypothetical women, based on the base rate of breast cancer in the population, which is about 1.2 percent.

It turns out that it is difficult to identify an accurate incident rate for breast cancer because of the different ways one can define the relevant population and, of course, limitations in the accuracy of breast cancer testing. I'm using an often-reported and fairly well-accepted estimate of the current percentage of women aged 40 to 84 that have breast cancer.

Let's return now to the third question in our list of important questions to ask before interpreting the results of a medical test. If a person scores a positive test result, how likely is the person to have the disease? Out of 10,000 women who have a breast cancer screening, 898 will receive a positive score. For 790 of those women, the score is wrong; they do not actually have breast cancer. For 108 of those women, the test was right; they do have cancer. In other words, if a person scores a positive result, it is only 12 percent likely that they have the disease. The most common result for follow-up testing to a positive mammogram is that the patient is, in fact, cancer free.

What about the accuracy of a negative result? Of the 9,102 women who will score negative on the screening, 12 actually have cancer. This is a relatively small 1/10 of 1 percent, but the testing will miss those people altogether, and they will not receive treatment.

Why It Works

Medical screening accuracy uses a specific application of a generalized approach to conditional probability attributed to Thomas Bayes, a philosopher and mathematician in the 1700s. "If this, then what are the chances that..." is a conditional probability question.

Bayes's approach to conditional probabilities was to look at the naturally occurring frequencies of events. The basic formula for estimating the chance that one has a disease if one has a positive test result is:

Expressed as conditional probabilities, the formula is:

To answer the all-important question in our breast cancer example ("If a woman scores a positive test result, how likely is she to have breast cancer?"), the mammogram equation takes on these values:

Making Informed Decisions

Medical tests are used to indicate whether patients might have a disease or be at risk for getting one. Identifying the presence or absence of a disease such as cancer is a process that usually has at least two steps. In step one, a patient is administered a screening test, typically a relatively simple and noninvasive test that looks for indications that a person might have a certain medical condition. If the result is positive, the second step is to conduct a second test (or series of tests) that is typically more complex, invasive, and expensive, but also much more accurate, to confirm or disconfirm the original finding.

Medical tests are not perfectly reliable and valid. Test results can be wrong. There are four possibilities for anyone who undergoes medical testing. A patient might have the disease and the test indicates this, or the patient does not have the disease and the test finds no presence of it. In these cases, the test worked right and the scores are valid.

Conversely, the test results might reflect the opposite of the true medical condition, with a positive result wrongly indicating presence of a disease that is not there, or a negative result wrongly indicating that the patient is disease-free. In these cases, the test did not work right and the results are not valid. This table of outcomes is similar to the possibilities when one accepts or rejects a hypothesis in statistical decision making [Hack #4].

Breast cancer screening is very good at finding breast cancer when it is there to find. However, one drawback to such a sensitive test for a low-incidence disease is that many more people will be told that they might have the disease than actually do. There is a trade-off in medical testing between test sensitivity and test specificity. More sensitive tests tend to result in more false positives, but in serious situations like life and death, this seems to be a result we can live with.

See Also

• Gigerenzer, G. (2002). Calculated risks. How to know when numbers deceive you. New York: Simon and Schuster.