Focus Groups

Many industries have discovered the value of focus groups for learning what customers like and don't like about various products. However useful focus groups are for gaining insight into what customers think about most consumer goods, they are troublesome when used in the software business. The biggest problem is simply that most people, even professional software users, are ignorant of what software is and what it can and cannot do. So when a focus group participant asks for a feature, the request is made from a shortsighted point of view. The user is asking for what he or she thinks is likely, possible, and reasonable. To consciously ask for something unlikely, impossible, or unreasonable would be to voluntarily seem stupid, and people don't willingly do that.

Stanford University scientists Nass and Reeves have studied people's reactions to computers, and they see conclusive evidence that people's own evaluation of their reactions to computers is unreliable. They say, "Many popular methods, especially focus group techniques, rely heavily on the assumption that people can be introspective about [interactive] experiences. This is an assumption that we think is frequently wrong."

Larry Keeley says that "users will reject new ideas if you ask them." This makes focus-group techniques suspect for any significantly innovative products. Today, most software-based products are sufficiently innovative to be un-focus-groupable.

Focus groups can be effective for some product categories, but it is a mistake to trust them for a reliable evaluation of high-cognitive-friction products.