When Focus Groups Are Appropriate

The purpose of focus groups is not to infer, but to understand, not to generalize but to determine a range, not to make statements about the population but to provide insights about how people perceive a situation.

—Richard A. Krueger, Focus Groups, p. 87

Knowing when to use focus groups is one of the keys to using them successfully. Although the technique is straightforward and flexible, it's not applicable to all cases or at all stages in the development of a product.

What Focus Groups Are Good For

Focus groups are good at finding desires, motivations, values, and first-hand experiences; in other words, they're a tool to get at people's attitudes and perceptions. A focus group is an environment where people (ideally) feel comfortable revealing their thoughts and feelings. This allows them to share their view of the issues and assumptions that lie at the core of an experience and to relate them to real-world situations.

When coupled with contextual inquiry and task analysis, a complete picture of how and why people are behaving right now can be built, before a single line of code is written or a single screen layout is sketched on a whiteboard.

As tools for competitive analysis, focus groups come in early in product development, though they can also be done during redesign or update cycles. Understanding fundamental issues and perceptions is generally most needed near the beginning of a product development cycle. That's also when the development team is trying to nail down what problems their product is supposed to solve, how it's supposed to solve them, and why it's valuable for consumers to use their solutions versus all others. Likewise, by bringing users of competitive products in early in the process, it's possible to find out why people value the competition, what they feel are the most critical features, what regularly bothers them, and where they feel the competitors fail. Apart from the obvious marketing angle, this information can immediately drive feature and interaction development, defining a user experience that's closer to what the target audience wants before resources have been committed.

Later in the development cycle, the technique can help identify and prioritize features. Knowing why people value certain features can help determine what gets developed and in what order. Moreover, since focus groups can act as brainstorming sessions, it's possible to achieve a synergy in which participants generate more ideas together than they could have come up with on their own.

One example from my practice was a site for Web novices that concentrated on news and information about developments in the Web world—which companies were doing what, what software was coming out when, and so forth. In conducting a series of focus groups with their target audience, we discovered that most of the people who were new to the Web were also new to computers in general. They weren't interested in Web industry developments nearly as much as they were interested in knowing the basics of how their software worked and where to turn for help. Moreover, since the process of getting on the Web was difficult for nearly all of them, they were tired of dealing with it and uninterested in installing or learning new software. They wanted to know what they could do with it and how they could use it to help the rest of their lives. Based on this information, the site decided to de-emphasize its Web news and information and to emphasize the self-help nature of the Web, collecting information about popular software products in one place.

Note

Focus groups uncover people's perceptions about their needs and their values. This does not mean that they uncover what people actually need or what really is valuable to them (in some objective sense); it's just what they think they need and what they claim is valuable. However, relativism (and italics) aside, knowing perceptions of needs is as important as knowing the needs themselves. Ultimately, people are going to judge a product based on what they think it does and how well that matches what they think they need. It doesn't replace accurate functionality or needs assessment, but the closer a product's presentation of its services matches people's perceptions of their needs, the more they're likely to use it.

What Focus Groups Are Not Good For

First and foremost, focus groups are not a way to get usability information. There's no good way for a group of people to tell you whether they will be able to use a certain interface or a certain feature. They can tell you whether they like the idea of it, but they can't show you whether they can use it in practice.

Second, focus group results are impossible to numerically generalize to a larger population, so they can't replace surveys. Specifically, focus groups create generalized models based on observations by an analyst. These models are assumed to apply to groups similar to the ones interviewed, but because they are not statistically significant samples, there's no guarantee that the proportion of responses in the group matches that of the larger population of users. This is an important distinction between focus groups and surveys. Because surveys are statistically significant (or should be), then the proportions observed in them can be extrapolated to larger populations. With focus groups, there is no such assurance.

Thus, although it's an inadequate technique for investigating the prevalence of a phenomenon in the entire target audience, focus groups can give you a really good idea of why the audience behaves how it does. Once the "why" has been determined, it can be verified through statistically significant research, such as a survey. However, identifying trends is often enough to act upon, making specific quantification unnecessary. Just as it can be sufficient to know the block a business is on without knowing the specific address, it's often enough to know that a phenomenon is widespread without knowing its exact magnitude.

Focus groups can create situations that are deceptive both to the participants and to analysts who literally interpret statements made in focus groups rather than extracting their underlying attitudes. This can be seen in the plethora of products (such as many feature films) that are made worse, rather than better, by misinterpreted focus group results. An amusing, if somewhat apocryphal, example of this comes from comic books: in an attempt to give Superman fans what they wanted, a focus group of comics consumers (10- to 12-year-old boys) was asked what kinds of figures they admired. Their replies were interpreted literally, and for a while in the 1960s, Superman did whatever the focus groups decided, leading to a string of surreal stories of the Man of Steel working as a police chief, dressing up as an Indian, or meeting George Washington (and to Jimmy Olsen, a meek supporting character, turning into a giant space turtle). It led to a kind of creative bankruptcy and an impossibly convoluted storyline that had to be eventually scrapped entirely, the comic starting over as if none of those stories had happened.

Finally, focus groups are not useful in situations where it's important to prove a point or to justify a position. The data collection process and analysis is based on multiple levels of human judgment. As such, the results can be called into question when used as a basis for proof, and rightly so. This is an excellent exploratory procedure; it produces deep insight into motivations and thinking processes, but it does not (and cannot) be used to unequivocally prove or disprove anything.

Four Types of Focus Groups

There are four common types of focus groups in software or Web development. The type of group you choose depends on the types of questions you want to answer, which in turn will likely depend on the stage of development the product is in. Don't feel limited by these categories; they're provided only as rough guides.

Exploratory. These groups get at general attitudes on a given topic, helping developers see how the eventual users of the product will understand it, what words they use to talk about it, and what criteria they will use to judge it. For example, a furniture company is interested in what criteria people use to buy furniture and how they buy similar items online. At the beginning of their development process, they run focus groups and find out that, at first, people insist on seeing furniture "in real life" before buying it (thus negating their entire business plan for selling it online). Further discussion reveals that it is only certain classes of products such as couches and beds that are mistrusted without direct experience. With other things (chairs, tables), most people have no problem buying based solely on pictures and descriptions and, in fact, would prefer to do so.

Feature prioritization. These groups focus on what features are most attractive to the group and why. They are held, in general, near the beginning of the development cycle, when it's already clear what the general outlines of the product are going to be. In these types of groups, the assumption is that the participants are interested in a certain kind of product, and the discussion centers on what kinds of things they would like that product to do for them. For example, the participants in a focus group for a homepage creation service were not nearly as interested in community services as they were in tools to help them build and market their own home-page. The "community feeling" that the site was trying to communicate and the tone with which it promoted itself meant little. For them the value in the site lay in the tools and free disk space.

Competitive analysis. Just as it's important to know what people value in the feature set that a product provides, it's important to know what attracts and repels them with respect to competitor's sites. Often held anonymously (with the commissioning client left unmentioned), these focus groups attempt to understand what associations people have with a competitor, what aspects of the competitor's user experience they find valuable, and where it doesn't satisfy their needs and desires. For example, a competitive focus group of online technology news sites revealed that most of the content that wasn't explicitly news was considered superfluous. Most people read only one news site for a few minutes a day. What they valued most were daily updates and links to other sites' content. Opinions, in-depth background stories, and thematic collections of older stories were not seen as valuable.

Trend explanation. When a trend in behavior is spotted, whether it's driven by survey responses, customer service feedback, or log analysis, it's often difficult to determine which of the many potential causes are primary. Focus groups can help explain the behavior by investigating the users' motivations and expectations. These types of focus groups are generally held either as part of a redesign cycle or in response to specific issues. For example, a survey showed that when people used search services, they would use two or three different ones with no particular pattern, but most people used Yahoo! regardless of what or how many other services they used. Focus groups with people who had taken the survey were called, and it was discovered that Yahoo! was consulted first because people felt that it did not contain much, but when it did, its results were of a significantly higher quality than the competitors. Thus, it made sense for them to always check it first, just in case.