Section 10.6. Participant Definition and Recruiting

10.6. Participant Definition and Recruiting

All of the remaining user research methods, including surveys, focus groups, interviews, and ethnographic studies, require the selection of representative samples of users to participate in the research studies. With the possible exception of surveys, it's rarely possible to study every user of a web site.

The definition and prioritization of intended and actual audiences for the site is obviously a critical factor. As we discussed earlier, there are myriad ways of slicing and dicing these audiences. Just as you define a primary hierarchy for your web site, you also need to define a primary hierarchy for participant selection. This hierarchy should strike a balance between the traditional ways that an organization views its customers (e.g., home users, business users, value-added resellers) and the distinctions an information architect is interested in (e.g., people familiar with the old site, people unfamiliar with the old site).

For large projects, the information architect should consider working with a traditional market-research firm that has experience defining audience categories, developing profiles of participants within those categories, recruiting participants, and handling logistics like facilities, incentives, and note taking.

10.6.1. Surveys

Surveys are a broad-and-shallow research tool that provide an opportunity to gather input from a large number of people relatively quickly and inexpensively. Surveys can be conducted via email, web, telephone, mail, or in person, and can be used to gather qualitative or quantitative data.

When designing a survey, you'll need to limit the number of questions if you want a reasonable response rate. You may also need to guarantee anonymity and offer an incentive. Since there's little opportunity for follow-up questions or dialogue, surveys don't allow you to gather rich data about users' information-seeking behaviors. Instead, they are best used for identifying:

• Which content and tasks users find most valuable

• What frustrates users most about the current site

• What ideas users have for improvement

• The current level of user satisfaction

In addition to the inherent value of real users' opinions, the survey results will provide you with a powerful political tool. If 90 percent of users say that the employee directory is the most important and most frustrating intranet resource, that's a compelling argument for improving it.

10.6.2. Contextual Inquiry

Field study is an important component of research programs in a variety of disciplines, from animal behavior to anthropology. Environmental context is tightly interwoven with behavioryou can only learn so much about the bald eagle or the bottle-nosed dolphin by studying them in a lab. The same applies to people and their use of information technology. In fact, a growing number of anthropologists are being tapped by the business world to apply their ethnographic research methods to product design.

These methods of contextual inquiry can be useful to the information architect.^[§] For example, simply seeing the work spaces of users can be valuable in showing the spectrum of information resources they use on a daily basis (e.g., computer, phone, bulletin board, Post-it notes).

^[§] To learn more about contextual inquiry, we recommend reading Contextual Design by Hugh Beyer and Karen Holtzblatt (Morgan Kaufmann).

If possible, it's also valuable to watch people interact with a site during the normal course of business. If you're redesigning a mission-critical call-center application that users interact with all day long, spend a few hours watching them. On the other hand, if you're redesigning a typical business web site, this observational approach won't be practical given the sporadic nature of site use. Most users will visit only once every several weeks or months. In these cases, you'll need to rely on user testing, though you still may be able to run the tests in the user's natural habitat.

In some cases, it can be valuable to simply watch people work. Observing users performing normal daily tasksgoing to meetings, taking phone calls, and so oncan provide insight into how the intranet or web site might (or might not) help people be more productive. The difficult issue here (and, to some degree, with all the observation approaches) is that information architecture begins to bleed into knowledge management and business-process reengineering. In an ideal world, the roles and responsibilities of departments, teams, and individuals would all be designed in an integrated fashion. In the real world (and particularly in large organizations), most projects are limited by the scope, schedule, and budget of these different departments. The folks responsible for designing the information architecture rarely influence the way other departments do their work. For this reason, keep asking yourself throughout the research process whether you'll actually be able to act on the data. If you're going to get the job done, the answer better be yes.

10.6.3. Focus Groups

Focus groups are one of the most common and most abused tools for learning from users. When conducting focus groups, you gather groups of people who are actual or potential users of your site. In a typical focus-group session, you might ask a series of scripted questions about what users would like to see on the site, demonstrate a prototype or show the site itself, and then ask questions about the users' perception of the site and their recommendations for improvement.

Focus groups are great for generating ideas about possible content and function for the site. By getting several people from your target audiences together and facilitating a brainstorming session, you can quickly find yourself with a laundry list of suggestions. However, focus groups don't work as well for information architectures as they do for, say, consumer products. For example, people can tell you what they like, don't like, and wish for regarding their refrigerators, but most people don't have the understanding or language necessary to be articulate about information architectures.

Focus groups are also very poor vehicles for testing the usability of a site. A public demonstration does not come close to replicating the actual environment of a user who is navigating a web site. Consequently, the suggestions of people in focus groups often do not carry much weight. Sadly, focus groups are often used only to prove that a particular approach does or doesn't work, and they can easily be influenced in one direction or another through the skillful selection and phrasing of questions.