|
|
|
Although a Web site is many things—entertainment, advertising, information—every Web site is also a tool. It's a tool to help people do something that,
Tools solve problems, and to build the right tool, you need to know what the problem is. There are a number of ways to do that. You can guess, using your knowledge of the target audience and what they're trying to do. This is fast, but it's fraught with danger: if you're not a member of the target audience (which, as a developer, you rarely are), your understanding of the nature and severity of your users' problems will not be the same as theirs. You could decide that someone needs a bigger
Another method is to ask representatives of the target audience what their problems are, but this too can lead you in the wrong direction. People tend to idealize their needs and desires, and their statements often don't
Moreover, the obvious problem isn't always the real problem. The person who wants a new hammer and smaller nails? Maybe she really just needs a cheap birdhouse, so she's making one. Once she's done, maybe she'll never need the hammer and nails again. The best way to find out what people's real problems and needs are, and how they really do things, is to discover them for yourself. The techniques in this chapter are designed to reveal how your target audience lives, how they think, and what problems they run into.
These procedures are best done before specific solutions have been created, which is
|
|
|
|
|
|
The definitive book on
contextual inquiry
—and one that this chapter is deeply indebted to—is Hugh Beyer and Karen Holtzblatt's
Contextual Design: Defining Customer-Centered Systems
. They define this procedure as
Contextual inquiry is a field data-gathering technique that studies a few
carefully selected individuals in depth toarrive at afuller understanding of the work practice across all customers. Through inquiry and interpretation, it reveals commonalties across a system's customer base.
In other words, contextual inquiry is a technique that helps you understand the
real
environment people live in and work in, and it reveals their needs within that environment. It uncovers what people really do and how they define what is actually
As a technique based in anthropology and ethnography, the basic method of research involves visiting people and observing them as they go about their work. In watching them carefully and
Since the focus of this book is information
Most projects begin with an idea about what problems must be
It can also be done in between development cycles or as part of a redesign. In those situations, it can tell you how people are using the product, when they're using it, and what they're using it for. This serves as a check of your initial assumptions and as a method of discovering areas into which the product can naturally expand (Table 8.1).
|
Timing |
Activity |
|---|---|
|
t - 2 weeks |
Organize and schedule
|
|
t |
Begin interviews. Begin analysis scheduling process for development team. |
|
t + 1 week |
Complete interviews. Review videotapes and take notes. Complete analysis scheduling. |
|
t + 2 weeks |
Prepare space for
|
|
t + 2 weeks |
Analyze affinity diagram (one day). |
|
t + 2 weeks + 1 day |
Begin results analysis. Write report and present to stakeholders. |
Since you'll be going out of your office and into the workplaces and
Choosing an appropriate target audience is described in detail in Chapter 6 of this book, but the short version is that you should pick people like the people you think will want to use your product. Maybe they use the product already. Maybe they use a competitor's product. Regardless, they should have the same profile as the target audience that will eventually use what you have to offer.
You should specify this target audience in as much detail as you can,
What is their demographic
What is their Web use profile?
What
What tools do they regularly use?
Are there tools they must occasionally use to solve specific problems?
How do they use them?
Concentrate on the most important customers. Your product may
Once you have your profile, you need to find people that match it. A complete description of recruiting is in Chapter 6, but here are some things to consider when recruiting for contextual inquiry.
First, decide how many people you want to visit. The number will depend on how much time you have allocated to the research and the resources available. Beyer and Holtzblatt suggest observing 15 to 20 people, but that can be prohibitive because of the amount of interview and analysis time it requires. Five to eight people should give you a pretty good idea of how a big
Once you've found your group of candidates, you need to schedule them. Contextual inquiry sessions can last from a couple of hours to a full
Interviews can last from an
Since the research is going to be onsite, give the participants some idea of what to expect. Tell them the general goals of the research, how long it will take, what equipment is going to be used, and what kinds of things you're going to be looking at. You don't have to be specific (and, in fact, leaving some specifics out can produce a more
When studying people in office environments, it's often necessary to get releases and to sign
The incentive payment should reflect the length of the observation and should be between $100 and $200 for most research. However, companies may have policies restricting such payments to their
In order to be able to understand what people are doing and to properly analyze your data, you need to be familiar with what they do. This means familiarizing yourself with the terminology, the tools, and the techniques that they are likely to be using in their work. You don't have to know all the details of their job, but you should be somewhat familiar with the domain.
If you know nothing about a task, before you begin your research, you can have someone familiar with it walk you through a session. If you have the time, you can also use the "sportscaster method," having one expert explain what another one is doing, as in a play-by-play
If possible, try the task yourself. If it's something that is done with software, ask to
In general, the more you know about the tasks your target audience does, the better you'll be able to interpret their behavior when you observe it.
As part of your preparation, you should be explicit about your expectations. Attempt to write down how and when you expect the people you're observing to do certain things that are important to your product, and what attitudes you expect they will have toward certain elements. You can do this with other
| Warning |
Be careful not to let your scenarios bias your observations. Use them as a framework to structure your interview, not to create expectations of how people do or don't behave in general. |
When you're in the interview, keep these scenarios in mind while watching people. Keep an
In addition to all the contextual inquiry-related preparation, there are a number of things you should do just because you're leaving the
Make a list of everything you're going to bring—every pencil, videotape, and notebook. Start the list a week before you're going to do your first interview. Then, whenever you realize there's something else you should bring, add it to the list. A day before the interview, make sure you have everything on the list (I put a check
If you're going to get releases (either to allow you to observe or for the participants to participate), make sure you have twice as many as you expect to need.
Bring everything you need for the incentive payment. This could be a disbursement form from accounting, a check, or an envelope with cash. Blank receipt forms are useful, too (though some accounting departments will accept a participant release as proof of receipt of payment).
Know how to
Have more than enough
Make plans for meal breaks. Closely watching someone for several hours can be draining, and you don't want to run around an office frantically looking for a drinking fountain while worried that you're missing a key moment. Bring bottled water and plan to eat in between sessions, or have
| Note |
Sometimes real-life situations unfold very differently from how you may have expected. You may be expecting to find one kind of situation—say, a typical day using the typical tools—and you find something completely different, a crisis where the main system is down or they're
|
One of the keys to getting good feedback in a contextual inquiry situation is establishing rapport with the participant. Since you want to watch him or her working as "naturally" as possible, it's important to set out expectations about each other's roles. Beyer and Holtzblatt define several kinds of relationships you can strive for.
The
master/apprentice
model introduces you as the apprentice and the person who you'll be watching as the master. You learn his or her craft by watching. Occasionally, the apprentice can ask a question or the master can explain a key point, but the master's primary role is to do his or her job, narrating what he or she is doing while doing it (without having to think about it or explain why). This keeps the "master craftsman" focused on details, avoiding the
Partnership
is an extension of the master/apprentice model where the interviewer
Beyer and Holtzblatt also point out several relationships to avoid.
The
interviewer/interviewee.
Normally, an interviewee is prompted by an interviewer's questions into
The
expert/novice.
Although you may be the expert in creating software for helping them, the participants are experts in their own domain. As Beyer and Holtzblatt suggest, "Set the customer's expectation correctly at the beginning by explaining that you are there to hear about and see their work because only they know their work practice. You aren't there to help them with problems or answer questions." They suggest that it should be clear that the goal is not to solve the problems then and there, but to know what the problems are and how they solve them on their own. If you are asked to behave as an expert, use nondirected interviewing techniques and
Don't be a
guest.
Your comfort should not be the focus of attention. You are there to understand how they do their work, not to bask in the light of their hospitality. Be sensitive to the protocol of the situation. If good manners
Another role to avoid is
big brother.
You are not there to evaluate or critique the performance of the people you are observing, and that should be clear. If they feel that way, then they're not likely to behave in typical ways. Moreover, if participation in your research is at the request of their managers, it can seem that this is a sneaky way to check up on them.
| Warning |
Sometimes management
may
want you to report on specific employees and their performance without telling them ahead of time that you'll be doing so. In such situations, explain the ethical problems with doing so—that it
|
The structure of the inquiry is similar to the structure of most interviews, except that the majority of it is driven by the interviewee's work rather than the interviewer's questions. The structure follows the general interview structure described in Chapter 6: introduction, warm-up, general issues, deep focus, retrospective,
The introduction and warm-up should be times for the participant and the interviewer to get comfortable with each other and to set up expectations for how the observation will proceed. This is the time to get all the nondisclosure forms signed, describe the project in broad detail, and set up the equipment. Check that the image and sound recording is good, and then don't fuss with the equipment again until the interview is over since it'll just distract people from their work. Describe the master/apprentice model and emphasize your role as an observer and learner. Remind the participant to narrate what he or she is doing and not to go for deep explanations.
| Note |
I'm using the word
action
to refer to a single operation during a task. In most cases, it's something that takes a couple of seconds and can be described with a single, simple idea. Actions are then grouped into
tasks,
which are things that
|
Once you're in position, you may want to ask some
general questions
to gain an understanding of who the person is, what his or her job is, and what tasks he or she is going to be doing. Ask the participant to provide a description of a typical day: what kinds of things does he or she do regularly? What are
This will begin the
main observation period.
This phase should comprise at least two-
| Warning |
Maintaining authenticity is a crucial part of observation. If you sense that the person you're watching is not doing a task in the way that they would do it if you were not watching, ask him or her about it. Ask whether how he or she is doing it is how it should be done, or how it is done. If the former, ask him or her to show you the latter even if it's "really messy." |
When either the task is done or time is up, the main interview period is over. An immediate
follow-up interview
with
| Note |
Provide privacy when people need it. Tell the people you're observing to let you know if a phone call or meeting is private—or if information being discussed is secret—and that you'll stop observing until they tell you it's OK to start again. Pick a place to go in such a situation (maybe a nearby conference room or the cafeteria) and have them come and get you when they're finished. |
Wrap-up the interview by asking the participant about the contextual inquiry experience from his or her perspective. Was there anything about it that made him or her anxious? Is there anything the participant would like to do differently? Are there things that you, as the apprentice, could do differently?
Beyer and Holtzblatt summarize the spirit of the interviewing process as follows:
Running a good interview is less about following specific rules than it is about being a certain kind of person for the duration of the interview. The apprentice model is a good starting point for how to behave. The four principles of Contextual Inquiry modify the behavior to better get design data: context, go where the work is and watch it happen; partnership, talk about the work while it happens; interpretation, find the meaning behind the customer's words and actions; and focus, challenge your entering assumptions.
So what you want most is to come in
There are four kinds of information you should pay attention to when observing people at work. Each of these elements can be improvised or formal, shared or used alone, specific or flexible.
The tools they use. This can be a formal tool, such as a specialized piece of software, or it can be an informal tool, such as a scribbled note. Note whether the tools are being used as they're designed, or if they're being repurposed. How do the tools interact? What are the brands? Are the Post-its on the bezel of the monitor or on the outside flap of the Palm Pilot?
The
sequences
in which actions occur. The order of actions is important in terms of understanding how the participant is thinking about the task. Is there a set order that's dictated by the tools or by office culture? When does the order matter? Are there things that are done in parallel? Is it done continuously, or
Their
What kinds of interactions they have. What are the important parties in the transfer of knowledge? Are they people? Are they processes? What kinds of information are shared (what are the inputs and outputs)? What is the nature of the interaction (informational, technical, social, etc.)?
The influences of all four of these things will, of course, be intertwined, and sometimes it may be hard to unravel the threads. The participant may be choosing a sequence for working on data, or the organization of the data may force a certain sequence. Note the situations where behaviors may involve many constraints. These are the situations you can clarify with a carefully placed question or during the follow-up interview.
|
|
Artifacts
are—for the purposes of contextual inquiry—the nondigital tools people use to help them accomplish the tasks they're trying to do. Documenting and collecting people's artifacts can be extremely enlightening. For example, if you're interested in how people schedule, it may be appropriate to photograph their calendars to see what kinds of annotations they make, or to videotape them using the office whiteboard that serves as the group calendar. If you're interested in how they shop for food, you may want to collect their shopping lists and videotape them at the
Always make sure to get permission when you copy or collect artifacts.
|
|
Here is a snippet of the notes from an observation of a health insurance broker using an existing online system to create an RFP (request for proposal):
|
|
Looks at paper [needs summary] form for coverage desired. Circles coverage section with pen.
|
|
|
|
Goes to Plan Search screen.
Opens "New Search" window.
"I know I want a 90/70 with a 5/10 drug, but I'm going to get all of the 90/70 plans no matter what."
Types in plan details without looking back at form.
Looks at search results page.
Points at top plan: "Aetna has a 90/70 that covers chiro, so I'm looking at their plan as a benchmark, which is enough to give me an idea of what to expect from the RFP."
Clicks on Aetna plan for full details.
Prints out plan details on printer in hall (about three cubes away) using browser Print button. Retrieves printout and places it on top of needs summary form.
Would like to get details on similar recent quotes.
Goes back to search results. Scrolls through results and clicks on Blue Shield plan.
|
|
|
|
I recommend
However, recording good interviews on video takes some practice, especially when working in the field. The twin dragons that can keep you from a good recording are
insufficient light
and
Light
. Most video cameras aren't as sensitive as human eyes, so a room that looks
Sound
. The hum of a computer fan. The hiss of an air conditioner. Laughing from the cafeteria next door. It's amazing what ends up recorded instead of what you wanted. Good sound is probably the most technically important part of the whole recording process. Getting it can be tricky. Most video cameras have omnidirectional microphones that pick up a broad arc of sound in front (and sometimes behind) the camera. This is great for taping the family at Disney World, but not so great when you're trying to isolate a participant's voice from the background noise. Get a camera with a good built-in microphone, an external microphone jack, and a headphone jack. Get a decent set of headphones, plug them into the
Even the best equipment is sometimes insufficient or superfluous. Some situations are
|
|
The output from Contextual Inquiry is not a neat hierarchy; rather, it is narratives of successes and breakdowns, examples of use that entail context, and messy use artifacts.
—Dave Hendry, Assistant Professor, University of Washington Information School, personal email
How data should be interpreted should
Understanding the
mental models
people build. People don't like black boxes. They don't like the unknown. They want to understand how something works in order to be able to predict what it will do. When the operation of a process or tool isn't apparent, people create their own model for it. This model helps them explain the results they see and
Understanding the
tools
they use. Since you're building a tool that's supposed to replace the tools people use now, it's important to know what those tools are and how they're used. Rather than leisurely browsing a catalog as you would expect them to do, they may just check the specials page and let an online comparison engine find the cheapest price. They may keep addresses in a Palm Pilot, a carefully organized address book, or they may just have a pocketful of scribbled-on
Understanding the
terminology
they use to describe what they do. Words reveal a lot about people's models and thought processes. When shopping for food, people may talk in terms of meals,
Understanding their methods. The flow of work is important to understanding what people's needs are and where existing tools are failing them. Unraveling the approach people take to solving a task reveals a lot about the strengths and weaknesses of the tools they use. If someone composes a message in a word processor and then cuts and pastes it into an email program, that says something about how he or she perceives the strengths of each product. If he or she is scribbling URLs on a pad while looking through a search engine, it says something about the weaknesses of the search engine and the browser and something about the strength of paper.
Understanding their
goals.
Every action has a reason. Understanding why people perform certain actions reveals an underlying structure to their work that they may not be aware of themselves. Although a goal may seem straightforward ("I need to find and print a TV listing for tonight"), the reasons behind it may reveal a lot about the system people are using ("I want to watch
The Simpsons
at
Understanding their
values.
People's value systems are part of their mental model. We're often driven by our social and cultural contexts as much as by our rational decisions. What is the context for the use of these tools? Are they
There are several ways to do a full analysis of the data you collect. Beyer and Holtzblatt recommend what they call the
affinity diagram
method (which is loosely
Watch the observation videotapes to create 50–100 notes from each 2-hour interview (longer interviews will produce more notes, though probably not in proportion to the length of the interview). Notes are singular observations about tools, sequences, interactions, mental models—anything. Number the notes and identify the user whose behavior inspired it (they recommend using
Get a group of people together in a room with a blank wall, a large window, or a big whiteboard. Beyer and Holtzblatt recommend 1 person per 100 notes. Preferably, these are members of the development team. By making the development team do the analysis, a group understanding of the customer's needs is
Divide the group into pairs of analysts. Give each pair an equal number of notes (ideally, each pair should have 100–200 notes).
Write a note on a yellow Post-it (yes, yellow; Beyer and Holtzblatt are very specific about Post-it colors) and put it on the wall/window/board.
Tell the groups to put notes that relate to that note around it one at a time. It doesn't matter how the notes relate, just as long as the group feels they relate.
If no more notes relate to a given note cluster, put a blue note next to the group. Write a label on the blue note, summarizing and naming the cluster. They recommend avoiding technical terminology in the labels and using simple
Repeat the process with the other notes, labeling groups in blue as they occur.
Try to keep the number of yellow notes per blue group between two and four. One note cannot be a group, and often groups of more than four notes can be broken into smaller clusters. However, there's no upper bound on how many notes may be in a group if there's no obvious way to break it up.
As the groups accumulate, they recommend using pink notes to label groups of blue notes, and green notes to label groups of pink notes.
Eventually, you run out of yellow notes, and the group of analysts
Figure 8.1:
A portion of an affinity diagram.
The insurance broker observation might produce an affinity diagram with the following
|
|
RFPs are tools that collect most of the information
RFPs are flexible
RFPs are read by people. (note 35, U2)
"I don't normally compare plans. I just ask for exactly what I want." (note 20, U1)
Plan comparisons provide critical information
Query specificity is important (and largely absent)
"I know I want a 90/70 with a 5/10 drug, but I'm going to get all the 90/70 plans no matter what." (note 55, U2)
Some plans exclude certain industries (legal). No way to filter on that. (note 43, U3)
Carrier ratings are important to customers. (note 74, U3)
Query output requires a lot of filtering
"Most of my time is
|
|
In addition to the affinity diagram method, it's also possible to do a more traditional kind of analysis based on an expert reading of the data. The observers can all meet and discuss their observations and hypothesize about how those observations are linked and what they mean. Although not as
In many situations, this will be enough to begin building models of where the
Flow models are representations of "the communication between people to get work done." They show what information, knowledge, and artifacts get passed among the members of the development team. Their elements can be formal or informal, written or verbal. They seek to capture the interaction, strategy, roles, and informal structures within the communication that happens in a product's life cycle.
Sequence models represent "the steps by which work is done, the triggers that kick off a set of steps, and the intents that are being accomplished." They show the order that things are done, what causes ("triggers") certain steps, the purpose ("intent") of each step, and how they depend on each other. They are sufficiently detailed to allow the team to understand, step by step, how a task is accomplished.
Artifact models represent how people use real-world tools to accomplish their goals. Starting with a simple photograph, drawing, or photocopy of the object, artifact models "extend the information on the artifact to show structure, strategy, and intent." They provide insight into the tools people use, how they use them, what problems they have, and most important, why they're necessary.
Physical models represent the actual physical environment that users do their work in. They provide an understanding of the layout of the workspace, the artifacts in the workspace, what controls people have (and don't have) over their environment, and how they use their environments to get work done.
Cultural models represent an understanding of the users' values and how they see themselves. It places the product in the context of the users' lives and the real-world environment in which they live. It includes both the formal organization of their experience—their other responsibilities, the competitive climate—and the informal—the emotions associated with the work, the work environment, the users' esthetic values and style, and so on.
| Note |
Frequency does not equate to importance. Just because something happens often doesn't mean that it's more important to the design of a system than something that happens rarely. For example, most people observed may keep paper notes of a certain transaction, and they may do it several times a day. Although this is a
|
In many cases, the "Aha!" moments come either during the actual observation or in the affinity diagram creation phase. The aha! may be sufficient to turn the conceptualization of a product around and provide enough information so that product-specific (as opposed to
It's never good to jump to conclusions, but even if time is
If you have more time and resources to look at the information you've collected, do so. The data can provide a rich and subtle understanding of the mental models and task flows people use in doing their work. Beyer and Holtzblatt spend a good deal of their book discussing just how to do this.
|
|
|