A Note on Questionnaires

There is no substitute for an interview.

• Do it first!

• Do it for every new class of problem!

• Do it for every new project!

We are often asked whether the team can substitute a questionnaire for this interviewing process. In some cases, the need expressed is perhaps a simple desire for efficiency ("I could do 100 questionnaires in the time it takes to do one interview"). In other cases, the need itself may come under suspicion ("Do I really have to talk to these people? Couldn't I just send them a letter?").

No matter what the motivation, the answer is no . There is no substitute for the personal contact, rapport building, and free-form interaction of the interview technique. We are confident that after one or two interviews, your worldview will change. Even more important, the vision for the solution will change along with it! Do the interview first. Do it for every new class of problem, and do it for every new project.

Although the questionnaire technique is often used and appears scientific because of the opportunity for statistical analysis of the quantitative results, the technique is not a substitute for interviewing. When it comes to requirements gathering, the questionnaire technique has some fundamental problems.

• Relevant questions cannot be decided in advance.

• The assumptions behind the questions bias the answers.

  Example:

  Did this class meet your expectations? Assumption: You had expectations, so this is a meaningful question.

• It is difficult to explore new domains ("What you really should be asking about is . . ."), and there is no interaction to explore domains that need to be explored.

• It is difficult to follow up on unclear user responses.

Indeed, some have concluded that the questionnaire technique suppresses almost everything good about requirements gathering. Therefore, we generally do not recommend it for this purpose.

Questionnaires can be used to validate assumptions and gather statistical preference data.

However, the questionnaire technique can be applied with good effect as a corroborating technique after the initial interviewing and analysis activity. For example, if the application has a large number of existing or potential users and if the goal is to provide statistical input about user or customer preferences among a limited set of choices, a questionnaire can be used effectively to gather a significant amount of focused data in a short period of time. In short, the questionnaire technique, like all elicitation techniques, is suited to a subset of the requirements challenges that an organization may face.