Collecting Comments

In automated systems, the comments should already be collected, which makes this part of the process easy. If it exists, a database of questions and answers is a good starting place. Such databases may have data going back a long time and over many versions. For a user support organization, this archival data is effectively free, so they keep it around just in case, but it's not all that useful when trying to figure out what's going on right now. Recent issues, from the last couple of weeks or months, are generally enough to provide a list of topics and features to focus on.

If there isn't an automated system, collecting the data may be more challenging. You can begin by interviewing the support staff about how questions are answered and gathering their "stock" answers to frequently asked questions. This gives you a top-level idea of what kinds of issues arise.

In addition, it's important to look at people's actual comments. Looking at all the comments, not just the ones that have been deemed important enough for replies, gives you a much fuller picture of the users' experiences. Gathering reams of feedback data is unnecessary. A randomly selected sample of 100–200 comments from across the previous month should provide a decent idea of people's problems and responses, though if the content is particularly diverse or if volume is particularly heavy, it may be necessary to gather more. When possible, preserve any additional information that's available, such as the section from which the comment was sent or if there's a clickstream associated with the session.