CONTROL GROUPS

If you are conducting a study to evaluate a new experimental method or treatment, make sure you include a group that does not receive the new treatment. This control group will provide you with measurements to which the results of the new treatment can be compared. If you are evaluating a new instructional method, for example, the appropriate control treatment may be the standard instructional method. If you are doing a medical experiment, the appropriate control treatment may be the standard medication or procedure for a particular ailment. If you are doing a study of a new component for a new sub-assembly, the control group may be an old design of that product.

Do not compare the new treatment's results just to historical information or commonly held beliefs. Experimenters may be tempted to do so, but then they run into a variety of problems. For example, a surgeon who is pioneering a new technique cannot simply compare the survival rates of patients who were given the new operation with those of patients from previous years . An engineer pioneering a new catalytic converter cannot afford to evaluate that new technology only by comparing it to past catalytic converters. Differences may occur for many reasons. Current patients may have been diagnosed earlier than previous patients , so they have a better chance of surviving. Another possibility is that the surgeon's skills may have improved with time, making the newer patients more likely to survive. In the case of the catalytic converter, it may be that the new one is "better" because it is positioned closer to the manifold or because it includes more precious metal.

All kinds of things may be different between groups that are treated at different times. You do not know ” you cannot know ” what all of these things are and how they affect a study. To avoid this problem, make sure that a control group is part of your study's design, and do not rely on historical controls.