Ethnographic Interviews: Interviewing and Observing Users

Drawing on years of design research in practice, the authors believe that a combination of one-on-one interviews and work/lifestyle observation is the most effective and efficient tool in a designer's arsenal for gathering qualitative data about users and their goals. The technique of ethnographic interviews (Wood, 1996) is a combination of immersive observation and directed interview techniques.

Hugh Beyer and Karen Holtzblatt have pioneered an ethnographic interviewing technique that they call contextual inquiry. Their method has, for good reason, rapidly gained traction in the industry, and provides a sound basis for qualitative user research. It is described in detail in the first four chapters of their book, Contextual Design (1998). Contextual inquiry methods closely parallel the methods described here, but with some subtle and important differences.

Contextual Inquiry

Contextual inquiry, according to Beyer and Holtzblatt, is based on a master-apprentice model of learning: observing and asking questions of the user as if she is the master craftsman and the interviewer the new apprentice. Beyer and Holtzblatt also enumerate four basic principles for engaging in ethnographic interviews:

• Context: Rather than interviewing the user in a clean white room, it is important to interact with and observe the user in their normal work environment, or whatever physical context is appropriate for the product. Observing users as they perform activities and questioning them in their own environments, filled with the artifacts they use each day, can bring the all-important details of their behaviors to light.

• Partnership: The interview and observation should take the tone of a collaborative exploration with the user, alternating between observation of work and discussion of its structure and details.

• Interpretation: Much of the work of the designer is reading between the lines of facts gathered about users' behaviors, their environment, and what they say. These facts must be taken together as a whole, and analyzed by the designer to uncover the design implications. Interviewers must be careful, however, to avoid assumptions based on their own interpretation of the facts without verifying these assumptions with users.

• Focus: Rather than coming to interviews with a set questionnaire or letting the interview wander aimlessly, the designer needs to subtly direct the interview so as to capture data relevant to design issues.

Improving on Contextual Inquiry

Contextual inquiry forms a solid theoretical foundation for qualitative research, but as a specific method it has some limitations and inefficiencies, at least in the opinion of the authors. The following process improvements, in the authors' experience, result in a more highly-leveraged research phase that better sets the stage for successful design:

• Shortening the interview process: Contextual inquiry assumes full day interviews with users. The authors have found that interviews as short as one hour in duration are sufficient to gather the necessary user data, provided that a sufficient number of interviews (about six well-selected users for each hypothesized role or type) are scheduled. It is much easier and more effective to find a diverse set of users who will consent to an hour with a designer than it is to find users who will agree to spend an entire day.

• Using smaller design teams: Contextual inquiry assumes a large design team that conducts multiple interviews in parallel, followed by debriefing sessions in which the full team participates. The authors have found that it is more effective to conduct interviews sequentially with the same designers in each interview. This allows the design team to remain small (two or three designers), but even more important, it means that the entire team interacts with all interviewed users directly, allowing the members to most effectively analyze and synthesize the user data.

• Identifying goals first: Contextual inquiry, as described by Beyer and Holtzblatt, feeds a design process that is fundamentally task-focused. We propose that ethnographic interviews first identify and prioritize user goals before determining the tasks that relate to these goals.

• Looking beyond business contexts: The vocabulary of contextual inquiry assumes a business product and a corporate environment. Ethnographic interviews are also possible in consumer domains, though the focus of questioning is somewhat different, as we describe later in this chapter.

The remainder of this chapter provides general methods and tips for preparing for and conducting ethnographic interviews.

Preparing for ethnographic interviews

Ethnography is a term borrowed from anthropology, meaning the systematic and immersive study of human cultures. In anthropology, ethnographic researchers spend years living immersed in the cultures they study and record. Ethnographic interviews take the spirit of this type of research and apply it on a micro level. Rather than trying to understand behaviors and social rituals of an entire culture, the goal is to understand the behaviors and rituals of people interacting with individual products.

IDENTIFYING CANDIDATES

Because the designers must capture an entire range of user behaviors regarding a product, it is critical that the designers identify an appropriately diverse sample of users and user types when planning a series of interviews. Based on information gleaned from stakeholders, SMEs, and literature reviews, designers need to create a hypothesis that serves as a starting point in determining what sorts of users and potential users to interview.

Kim Goodwin (2002a) has coined this the persona hypothesis, because it is the first step towards identifying and synthesizing personas, the user archetypes we will discuss in detail in the next chapter. The persona hypothesis is based on likely behavioral differences, not demographics, but takes into consideration identified target markets and demographics. The nature of a product's domain makes a significant difference in how a persona hypothesis is constructed. Business users are often quite different than consumer users in their behavior patterns and motivations, and different techniques are used to build the persona hypothesis in each case.

THE PERSONA HYPOTHESIS The persona hypothesis is a first cut at defining the different kinds of users (and sometimes customers) for a product in a particular domain. The hypothesis serves as a basis for an initial set of interviews; as interviews proceed, new interviews may be required if the data indicates the existence of user types not originally identified.

The persona hypothesis attempts to address, at a high level, these three questions:

• What different sorts of people might use this product?

• How might their needs and behaviors vary?

• What ranges of behavior and types of environments need to be explored?

ROLES IN BUSINESS AND CONSUMER DOMAINS Patterns of needs and behavior, and therefore types of users, vary significantly between business and technical, and consumer products. For business products, roles—common sets of tasks and information needs related to distinct classes of users—provide an important initial organizing principle. For example, in an enterprise portal, we might find these rough roles:

• People who search for content on the portal

• People who upload and update content on the portal

• People who technically administer the portal

In business and technical contexts, roles often map roughly to job descriptions, so it is relatively easy to get a reasonable first cut of user types to interview by understanding the kind of jobs held by users (or potential users) of the system.

Unlike business users, consumers don't have concrete job descriptions, and their use of products tends to cross multiple contexts. Their roles map more closely to lifestyle choices, and it is possible for consumer users to assume multiple roles even for a single product in this sense. For consumers, roles can usually better be expressed by behavioral variables.

BEHAVIORAL AND DEMOGRAPHIC VARIABLES Beyond roles, a persona hypothesis seeks to identify variables that might distinguish users based on their needs and behaviors. The most useful, but most difficult to anticipate without research, are behavioral variables: types of behavior that vary across a spectrum. For example, for an e-commerce application, there are several ranges of behavior concerning shopping that we might identify:

• Frequency of shopping (frequent—infrequent)

• Desire to shop (loves to shop—hates to shop)

• Motivation to shop (bargain hunting—searching for just the right item)

Although consumer user types can often be roughly defined by the combination of behavioral variables they map to, behavioral variables are also important for identifying types of business and technical users. People within a single business-role definition may have different motivations for being there and aspirations for what they plan to do in the future. Behavioral variables can capture this, though usually not until user data has been gathered.

Given the difficulty in accurately anticipating behavioral variables before user data is gathered, another helpful approach in building a persona hypothesis is making use of demographic variables. When planning your interviews, you can use market research to identify ages, locations, gender, and incomes of the target markets for the product. Interviewees should be distributed across these demographic ranges.

DOMAIN EXPERTISE VERSUS TECHNICAL EXPERTISE One important type of behavioral distinction to note is the difference between technical expertise (knowledge of digital technology) and domain expertise (knowledge of a specialized subject area pertaining to a product). Different users will have varying amounts of technical expertise; similarly, some users of a product may be less expert in their knowledge of the product's domain (for example, accounting knowledge in the case of a general ledger application). Thus, depending on who the design target of the product is, domain support may be a necessary part of the product's design, as well as technical ease of use. A relatively naive user will likely never be able to use more than a small subset of a domain-specific product's functions without domain support provided in the interface. If naive users are part of the target market for a domain-specific product, care must be taken to support domain-naive behaviors.

ENVIRONMENTAL CONSIDERATIONS A final consideration, especially in the case of business products, is the cultural differences between organizations in which the users are employed. Small companies, for example, tend to have more interpersonal contact between workers; huge companies have layers of bureaucracy. These environmental variables also fall into ranges:

• Company size (small—multinational)

• IT presence (ad hoc—draconian)

• Security level (lax—tight)

Like behavioral variables, these may be difficult to identify without some domain research, because patterns do vary significantly by industry and geographic region.

PUTTING A PLAN TOGETHER

After you have created a persona hypothesis, complete with potential roles, behavioral, demographic, and environmental variables, you then need to create an interview plan that can be communicated to the person in charge of providing access to users.

Each identified role, behavioral variable, demographic variable, and environmental variable identified in the persona hypothesis should be explored in four to six interviews (sometimes more if a domain is particularly complex). However, these interviews can overlap: It is perfectly acceptable to interview a female in her twenties who loves to shop; this would count as an interview for each of three different variables: gender, age group, and desire to shop. By being clever about mapping variables to interviewee screening profiles, you can keep the number of interviews to a manageable number.

Conducting ethnographic interviews

After the persona hypothesis has been formulated and an interview plan has been derived from it, you are ready to interview—assuming you get access to interviewees! While formulating the interview plan, designers should work closely with project stakeholders who have access to users. Stakeholder involvement is generally the best way to make interviews happen, especially for business and technical products.

If stakeholders can't help you get in touch with users, you can contact a market or usability research firm that specializes in finding people for surveys and focus groups. These firms are useful for reaching consumers with diverse demographics. The difficulty with this approach is that it can sometimes be challenging to get interviewees who will permit you to interview them in their homes or places of work.

As a last alternative for consumer products, designers can recruit friends and relatives. This makes it easier to observe the interviewees in a natural environment, but also is quite limiting as far as diversity of demographic and behavioral variables are concerned.

INTERVIEW TEAMS AND TIMING

The authors favor a team of two designers per interview, one to drive the interview and take light notes, and the other to take detailed notes (these roles can switch halfway through the interview). One hour per user interviewed should be sufficient, except in the case of interviewing consumers in their homes or traveling with them as they interact with a product (for which you should budget extra time). Teams should try to limit interviews to six per day, so that there is adequate time for debriefing and strategizing between interviews, and so that the interviewers do not get fatigued.

PHASES OF ETHNOGRAPHIC INTERVIEWS

A complete set of ethnographic interviews for a project can be grouped into three distinct, chronological phases. The approach of the interviews in each successive phase is subtly different from the last, reflecting the growing knowledge of user behaviors that results from each additional interview. Focus tends to be broad at the start, aimed at gross structural and goal-oriented issues; and more narrow for interviews at the end of the cycle, zooming in on specific functions and task-oriented issues.

• Early-phase interviews are exploratory in nature, and focused on gathering domain knowledge from the point of view of the user. Broad, open-ended questions are common, with a lesser degree of drill-down into details.

• Mid-phase interviews are where designers begin to see patterns of use and ask open-ended and clarifying questions to help connect the dots. Questions in general are more focused on domain specifics, now that the designers have absorbed the basic rules, structures, and vocabularies of the domain.

• Late-phase interviews confirm previously observed patterns, further clarifying user roles and behaviors and making fine adjustments to assumptions about task and information needs. Closed-ended questions are used in greater number, tying up loose ends in the data.

After you have an idea who your actual interviewees will be, it can be helpful, if you have the opportunity, to work with stakeholders to schedule individuals most appropriate for each phase in the interview cycle. In some cases, you may also want to loop back and interview a particularly knowledgeable and articulate subject both at the beginning and the end of the interview cycle.

BASIC METHODS

The basic methods of ethnographic interviewing are simple, straightforward, and very low-tech. Although the nuances of interviewing subjects takes some time to master, any practitioner should, if they follow the suggestions below, be rewarded with a wealth of useful qualitative data:

• Interview where the interaction happens

• Avoid a fixed set of questions

• Focus on goals first, tasks second

• Avoid making the user a designer

• Avoid discussion of technology

• Encourage storytelling

• Ask for a show and tell

• Avoid leading questions

We describe each of these methods in more detail in the following sections.

INTERVIEW WHERE THE INTERACTION HAPPENS Following the first principle of contextual inquiry, it is of critical importance that subjects be interviewed in the places where they actually use the products. Not only does this give the interviewers the opportunity to witness the product being used, but it also gives the interview team access to the environment in which the interaction occurs. This can give tremendous insight into product constraints and user needs and goals.

Observe the environment closely: It is likely to be crawling with clues about tasks the interviewee might not have mentioned. Notice, for example, the kind of information they need (papers on desks or adhesive notes on screen borders); inadequate systems (cheat sheets and user manuals); frequency and priority of tasks (inbox and outbox); and the kind of workflows they follow (memos, charts, calendars). Don't snoop without permission, but if you see something that looks interesting, ask your interviewee to discuss it.

AVOID A FIXED SET OF QUESTIONS If you approach ethnographic interviews with a fixed questionnaire, you not only run the risk of alienating the interview subject, but you can also cause the interviewers to miss out on a wealth of valuable user data. The entire premise of ethnographic interviews (and contextual inquiry) is that we as interviewers don't know enough about the domain to presuppose the questions that need asking: We must learn what is important from the people we talk to. This said, it's certainly useful to have types of questions in mind.

Here are some goal-oriented questions to consider:

• Opportunity: What activities currently waste your time?

• Goals: What makes a good day? A bad day?

• Priorities: What is most important to you?

• Information: What helps you make decisions?

Another useful type of question is the system-oriented question:

• Function: What are the most common things you do with the product?

• Frequency: What parts of the product do you use most?

• Preference: What are your favorite aspects of the product? What drives you crazy?

• Failure: How do you work around problems?

• Expertise: What shortcuts do you employ?

For business products, workflow-oriented questions can be helpful:

• Process: What did you do when you first came in today? And after that?

• Occurrence and recurrence: How often do you do this? What things do you do weekly or monthly, but not every day?

• Exception: What constitutes a typical day? What would be an unusual event?

To better understand user motivations, you can employ attitude-oriented questions:

• Aspiration: What do you see yourself doing five years from now?

• Avoidance: What would you prefer not to do? What do you procrastinate on?

• Motivation: What do you enjoy most about your job (or lifestyle)? What do you always tackle first?

FOCUS ON GOALS FIRST, TASKS SECOND Unlike contextual inquiry and the majority of other qualitative research methods, the first priority of ethnographic interviewing is understanding the why of users—what motivates the behaviors of individuals in different roles, and how they hope to ultimately accomplish this goal—not the what of the tasks they perform. Understanding the tasks is important, and the tasks must be diligently recorded. But these tasks will ultimately be restructured to better match user goals in the final design.

AVOID MAKING THE USER A DESIGNER Guide the interviewee towards examining problems and away from expressing solutions. Most of the time, those solutions sound good to him, but are either not well considered, or they represent idiosyncratic solutions that the user-modeling tool of personas actively seeks to avoid. That said, if a user blurts out an interesting idea, by all means record it for later reference, thank the user for his input, and move on.

AVOID DISCUSSIONS OF TECHNOLOGY Just as you don't want to treat the user as a designer, you also don't want to treat him as a programmer or engineer. In the case of technical or scientific products, where technology is always an issue, distinguish between domain-related technology and product-related technology, and steer away from the latter.

ENCOURAGE STORYTELLING Far more useful than asking users for design advice is encouraging them to tell specific stories about their experiences with a product (whether an old version of the one you're redesigning, or an analogous product): how they use it, what they think of it, who else they interact with when using it, where they go with it, and so forth. Detailed stories of this kind are some of the best ways to understand how users relate to and interact with products. Try to encourage stories that deal with both typical cases and more exceptional ones.

ASK FOR A SHOW-AND-TELL After you have a good idea of the flow and structure of a user's activities and interactions and you have exhausted other questions, it is often useful to ask the interviewee for a show-and-tell or grand tour of artifacts related to the design problem. These can be domain-related artifacts, software interfaces, paper systems, tours of the work environment, and ideally all the above. Be careful to not only record the artifacts themselves (digital or video cameras are very handy at this stage), but also pay attention to how the interviewee describes them. Be sure to ask plenty of clarifying questions as well.

AVOID LEADING QUESTIONS One important thing to avoid in interviews is the use of leading questions. Just as in a courtroom, where lawyers can, by virtue of their authority, bias witnesses by suggesting answers to them, designers can inadvertently bias interview subjects by implicitly (or explicitly) suggesting solutions or opinions about behaviors. Examples of leading questions include:

• Would feature X help you?

• You like X, don't you?

• Do you think you'd use X if it were available?

AFTER THE INTERVIEWS

After each interview, teams compare notes and discuss any particularly interesting trends observed or specific points brought up in the most recent interview. If they have the time, they should also look back at old notes to see whether unanswered questions from other interviews and research have been properly answered. This information should be used to strategize about the approach to take in subsequent interviews.

After the interview process is finished, it is useful to once again make a pass through all the notes, marking or highlighting trends and patterns in the data. This is very useful for the next step, creating personas from the cumulative research. If it is helpful, the team can create a binder of the notes, review any videotapes, and print out artifact images to place in the binder or on a public surface like a wall where they are all visible simultaneously. This will be useful in later design phases.