Chapter 13: Log Files and Customer Support

Overview

Much user experience research is geared toward understanding how people could experience your site or estimating how they would want to. But it's difficult to understand how they are experiencing it. Satisfaction surveys can get at what people feel is working and not working, but people's predictions and preferences are not good predictors of their behavior. Contextual inquiry can reveal issues in the way people use your site, but only one person at a time. Usability testing can uncover many likely problems, but suffers from the problem of projecting people's behavior in a laboratory environment versus real life. None of these actually tell you how people are currently using the site, what problems they're really having with it "in the wild." Knowing users' actual behavior closes the loop that started with the research and profiling you did before beginning the development of the site. Without it, you never know whether your designs really work.

Fortunately, there are two sources of information you probably already have that reveal your users' current experiences: customer support comments and Web server log files. Merging knowledge of people's behavior and opinions with your other understanding of your users and your company can reveal deep patterns of behavior and help you make their use of your product more efficient and profitable for everyone involved. Unfortunately, both of these sources feature large data sets that need analysis before they can be useful. While they're a gold mine of information about user behaviors and interests, gold usually doesn't lie around in nuggets. It requires energy to extract, but it's energy well spent.

The first place to start gathering information about users' experiences is their self-generated comments. Customer support comments are a direct line into the thoughts, interests, identities, and problems of your users. It's one area of research for which the researcher doesn't have to ask the questions, the answers come to you.

If customer feedback were objective and representative, there would be no need for any other kind of user research. People would tell you exactly what they want, what's wrong, and what's right. But customer feedback is not objective. The reasons that people contact customer support are many and varied, but they're overwhelmingly negative, and so it's impossible to get a balanced view of the experience your product provides just by looking at user comments. Nonetheless, customer feedback is an asset that needs little cultivation and should not be ignored. It provides insight into several key aspects of the user experience.

• It helps you see how people think about the product as a tool and to learn the language they use. This helps you understand their mental model of the task and the product.

• It reveals a lot about people's expectations and where those expectations are met and where they're not.

• It underscores the perceived "points of pain," which helps prioritize issues to determine which features to concentrate on.

• It gives you specific questions to ask in your research plan and directions for guiding future research.

In other words, these findings help to guide your research toward issues that are perceived as most important by your users, and the place to start understanding customer support comments is to understand the support process.