Preparation

Preparing a report is different from analyzing the data. The content of the report should be mostly done by the time the report writing begins (as described for surveys in Chapter 11). The report is the presentation of that information, a structured experience of the knowledge, and depends on the completed content. Thus, before the report and presentation writing begins, the audience and the data analysis process need to be understood.

Know Your Audience

An audience profile helps define what words to use, what level of detail is appropriate, and the order in which conclusions should be presented. This doesn't have to be a formal profiling process, but establishing several things about the audience really helps focus the writing of the report. Here are several questions to ask.

• Who is getting the report? The composition of the report's audience can change the emphasis of the content. CEOs will certainly need a shorter and more high-level report than the QA team. A large group of nontechnical people will need a different set of information from a small group of programmers. In certain cases, even the needs of specific individuals in the audience should be considered. If the vice president of production is driven by specifics and wants lists of quick fixes for her product, she's going to expect that. However, if usability testing shows that the product had systemic problems that needed more than just a list of repairs, the report should be prepared to satisfy both needs. For example, each quick fix can be introduced as a symptom of larger systemic problems, thus both the vice president's preference and the larger issues can be addressed.

• What are their goals? How are they going to use the information? Sometimes all that's required is an explanation of existing problems and a set of development targets. In other cases, the audience may need justification to argue for certain changes. Still others need an external perspective to evaluate how well they understand their users.

• What do they know? What's the level of technical sophistication in the group? This applies not just to the technology under consideration—though that should be taken into account, too—but the methods you are employing. If they've never been involved in user experience research, they may need more explanation of the reasoning behind your methods. If the presentation is to marketing research, the audience will likely not need to be told the difference between a focus group and an interview, but if they're engineers or designers, they may need an explanation.

• What do they need to know? Why are they getting this report? How is it going to make their jobs easier? The research project has a series of goals, which are based on a list of needs. Meeting those goals is the primary purpose of the report, but there may also be other information that should be communicated. During the process of conducting the research, the project's focus may have shifted, or a secondary class of information may have been revealed. These may need to be communicated. For example, when doing a contextual inquiry series for a kid's Web development site, it became apparent that the parents' level of technical sophistication wasn't much greater than the kids'. This insight was outside the goals of the original research (which was only to research the usage patterns and needs of kids), but it was very valuable to know.

• What are they expecting? Anticipating the audience's reactions to the information in a report helps you structure an effective presentation. The report should be written with an understanding of the audience's expectation of its contents. If the conclusions of the report go completely against the current belief of the development staff, then the report will likely need to contain a more thorough explanation and justification for the conclusions. Likewise, if many of the findings are in line with the staff's understanding of the issue, then the report may place greater emphasis on the aspects that differ from the common understanding.

If the restrictions on your audience are too constraining—you have to deliver information to nontechnical executives and the core programming staff, for example—it may be necessary to break up the audience. Presenting different aspects of the same information as different presentations may be preferable to everyone involved: the audience won't feel either lost or patronized, and the presenters will have to field fewer questions.

Know Your Process

There are limitations to even the most carefully conducted research. When considering how to present findings, keep these in mind. Acknowledging the limitations in the process helps the audience understand the results, believe in them, and act on the recommendations. The questions to ask about the process are the same that should be asked of any research.

• What are the data collection problems? The limitations and potential distortions in the data collection method should be acknowledged. Actual data collection problems should be mentioned in the report ("We were only able to speak to one of the two user audiences"), but potential problems with the method should be considered during its preparation ("We chose to focus on one user group because we believe that, given our limited resources, they will give us the best feedback"). If a member of the audience points out a potential flaw, it should have been anticipated and addressable.

• What are the limitations of the analysis? There are many ways to analyze a given set of data, and only a subset of them will have been used in your analysis. A videotape of a user test can be analyzed by watching it once and taking notes, or by doing a frame-by-frame analysis of every single utterance, keystroke, and mouse click. The strengths and limitations of the chosen analysis method should be known and acknowledged. So, for example, although tabulating survey results is fast and easy, it may not reveal as much as cross-tabulating the results of several of the variables, but that's a trickier procedure and requires more responses. Likewise, a focus group can tell you what people want, but not whether they need it or would be able to use it if it was given to them.

Understanding the biases in all facets of the research process helps when trying to explain it. The recruiting process, the participants' perspective, the research conditions, the analyst's experience— each can subtly skew the information collection process. Preparing for these biases and knowing about them helps minimize them and lets the truth of the situation shine through.