4.1 Requirements Definition

Designers and developers often think of requirements as a detailed list of the application's features and functionality. Although that is one goal of requirements definition, it is not the only one. You must understand and document other things in order to create a successful application. The goals of requirements definition include the following:

• Understanding the business goals and context

• Understanding the users

• Understanding the application

Once you understand these elements, you can define a set of metrics to measure application success and to provide guidance during design, development, and tuning.

4.1.1 Understanding the Business

A number of business issues must be understood:

• The business model: What is the primary business motivation for developing the system (e.g., to reduce costs, create new services)? What are the secondary motivations? Is the system replacing an existing system (e.g., touchtone)? If so, why?

• The corporate context: How does this product fit with other products the company offers? Are there multiple ways its customers can accomplish the same goals (e.g., on the Web as well as over the phone)? If so, in what way is this system supposed to complement the other systems (e.g., extend the organization's reach to mobile)? How does this application fit with all of the organization's other customer touchpoints?

• The corporate image: What image is the company trying to project? What are its brand attributes? How does the company portray itself in other media (e.g., Web, TV ads, touchtone systems)? How does this application figure into its overall branding efforts? Keep in mind that many companies are not aware of the branding and image-creating opportunity speech systems provide. You should be prepared to educate them. Make sure you have contact with those who are responsible for brand and image.

• The competitive environment: What products will this system compete with? What other methods do users have to accomplish the same goals?

• Schedule and rollout plans: How critical is time to market? (This may provide guidance on the size of the feature set.) How will customers be introduced to the system? Will they receive instructions in the mail? Will there be special incentives for them to start using the service? Will the service be rolled out in a series of phases with increasing functionality?

Answers to these questions can provide significant guidance during design and can help you make wise design trade-offs. For example, if the main goal of the business is to improve customer satisfaction, you might provide more information in the dialog explaining how to get to a live agent than if the business's main goal is to increase automation rates.

Once the business goals are understood, metrics can be defined that can be used to judge overall success. Such metrics can guide efforts during tuning and clarify priorities that will influence trade-offs during design.

Metrics may include task completion (how many callers successfully complete their task), user satisfaction (how happy callers are with their experience), and accuracy. Many other measures are possible, depending on application goals. For example, you might measure call durations, the number of dialog steps to complete tasks, or longitudinal results such as the proportion of customers who repeatedly call into the system over the following few months.

It is important to avoid the mistake of delaying the start of pilot testing on real customers pending the achievement of ambitious metrics. Analysis of pilot data is invaluable in achieving high performance.

The primary means of establishing the business requirements is through meetings with the appropriate people from the marketing group at the deploying company. However, you must supplement that with other sources of information. In the following sections, we describe techniques you can use to answer the business questions.

Evaluating Other Company Systems and Customer Touchpoints

Most companies touch their customers in a variety of ways. Some are one-way (not interactive) for example, television, radio, and newspaper advertising. Others may provide customers with means of accessing services, such as Web sites, touchtone systems, or live agents who provide information, transactions, or technical support.

Before meeting with company representatives, it will be worthwhile to familiarize yourself with the variety of the company's customer touchpoints. Explore its Web site. Call its touchtone system. Find an excuse to talk to a customer service representative. Pay attention not only to the functionality offered in each case but also to the look and feel. How does the organization reinforce its brand with each type of media? Additionally, look at its TV, radio, and newspaper ads. What are the image and message it is trying to communicate?

If you investigate these touchpoints before meeting with company personnel, you will be well equipped with questions and information that can make your meetings more productive and informative.

Meeting with Company Personnel

Meeting with marketing and other personnel at the deploying company will be a key means of understanding the business goals and context. To make sure that these meetings are productive and that you are getting to meet with the most appropriate people, send a list of questions and goals in advance. Make sure you meet not only with the people (probably in marketing) who are responsible for defining functionality but also with people involved in branding efforts. It can be useful to meet with someone responsible for training customer service representatives to understand the attitude and style the company tries to instill in its live agents.

Discuss all the issues mentioned in the earlier list. Additionally, ask for a brief description of the brand attributes. Then step through all the media used to reach customers, and find out how those brand attributes are manifested in each one.

Evaluating Competitive Systems

If other companies have deployed systems similar to the targeted application, you should evaluate them. At the very least, call them and exercise the systems thoroughly. It may even be worth conducting usability studies of competitive systems in order to improve on them and learn from what others have done. (Of course, time and money are always important factors in deciding the extent of such research.)

You should also evaluate other ways users can accomplish the same tasks. For example, if the target application is a fully automated directory assistance application, evaluate existing directory assistance applications that use live operators.

4.1.2 Understanding the Users

The population of expected callers to the system must be understood from two perspectives. First, you must understand caller profiles: Who will call the system? What are their characteristics and needs? Second, you must understand the usage profile: How will people use the system?

Caller profiles include the following elements:

• The set of target customers: Who are the target customers? Is a particular demographic group targeted? What is known about that group that is relevant to the application? Is the target customer set segmented? Are different classes of service planned for different segments?

• Dialect or jargon: Will these callers be from one industry and use industry-specific jargon? Are they from a specific dialect (geographic) region? Is the application focused on a particular age group or social group that has its own lingo?

• Level of technological sophistication: How comfortable are the callers with technology? Will most of the callers be Internet users? Will most of them have experience talking to automated systems?

• Caller's state of mind: Will this system be used in time-critical or mission-critical environments? Are callers seeking to manage their money? Looking for information? Looking for entertainment?

• Caller's self-image: Does this system target callers who have a particular self-image (e.g., tech-savvy, busy businessperson, on-the-go, relaxed, young and hip)?

• Caller's mental model: Part of the designer's job is to create an appropriate mental model in the mind of the caller. When callers initially interact with a system, they start with a mental model driven by past experience with similar systems and the ways they have performed the same or similar tasks in the past (including nonautomated ways). The goal, at this stage, is to understand the mental model callers bring with them when they first experience this system. That will help you craft a mental model that most effectively takes advantage of the caller's existing mind-set.

• Current user view of the brand: How do users currently view the company's brand and image?

Often, a business has multiple user populations. A tricky design challenge is to create an application that is suitable for all types of callers. Because of the cost involved in creating different applications for each user population, you usually must create one design that caters to multiple groups. If possible, try to gather statistics on how many of the callers will fall into each of the subgroups so that you have the necessary information to help you make decisions about design trade-offs.

Knowing who is going to use the system is only part of the story. You also must understand the when, where, why, and how. Usage profiles include the following elements:

• Primary versus secondary usage scenarios: What is the primary reason that people will use the system? Understanding this will help you optimize menus, error messages, help messages, and so on. How often will each feature will be used?

• One-time versus repeated use: Will this system be used repeatedly by the same callers, or should each caller be treated as if it is his or her first call?

• Level of attention: Will the caller's primary attention be elsewhere while using the system (e.g., is this a system typically used while driving)? If so, you must give careful consideration to cognitive load issues (see Chapter 9) and to features that accommodate sporadic lapses in attention.

• Channel and environment: Is the application focused primarily on mobile users? Will it usually be accessed by cell phone? Will it be accessed in noisy environments (e.g., hands-free in the car)?

• Voluntary versus involuntary use: Are the callers expecting to interact with a live agent and involuntarily talking to an automated system? Have they knowingly subscribed to an automated service?

• Integration with other systems: Will users of this system be expected to use other systems in concert with it, such as a Web interface to set preferences or call lists for a voice-activated dialer? Will users of this system sometimes do the same thing with other systems (e.g., use the Web when at their desktop, call this system when traveling)?

Clearly, discussions with the marketing department of the deploying company will be one important way to understand its users. However, it is important to also gather user information in other ways. The company may not see its users totally objectively, and it won't necessarily have all the answers to the kinds of questions and issues you must understand in order to optimize a VUI design. There are a number of approaches available. Some of these (e.g., observational studies) can be quick yet informative. Others (e.g., surveys and focus groups) can take more time but are warranted in certain cases.

Meeting with Company Personnel

As we suggested earlier, prepare for the meeting with company personnel by providing questions in advance, and make sure that the appropriate people will be there. The marketing group should be able to provide you with demographic profiles, information about industry-specific terminology (if, for example, it is a system for employees), and expectations of usage scenarios. Marketers also should be able to provide statistical data to predict the frequency of usage of the various features (e.g., from usage statistics of other systems or live agent calls or from statistics about the customer base). Many companies have accumulated a substantial amount of data about their customers, including their buying habits, lifestyles, and so on. All such data that they are willing to share can be of value to you.

Observational Studies

An observational study involves observing users performing the task that will be facilitated by the proposed application (Nielsen 1993; Preece et al. 2002). For example, if the application is for booking airline flights, an observational study might involve listening in on calls between travelers and travel agents. This approach can provide great insight into the mental model people bring to such tasks. You can observe their concerns, issues, and the ways they typically perform the task. For example, do travelers most often describe their trip based on departure time or arrival time? Do they prefer to get a long list of possible flights, or to consider a single flight and then ask questions about particular flight parameters (e.g., "Are there any later flights?")? Are their biggest concerns pricing, schedule, airline, type of airplane, or something else?

Observational studies create an early opportunity to get real data about motivated callers truly engaged in performing the task. As you will see, many of the approaches to testing during the design process use subjects who are assigned specific tasks to perform for example, "Call this system and book a flight from New York to Boston for next Tuesday." Although there are many things to learn from such studies, they do suffer from the lack of realism: The caller is not actually planning a flight. The next opportunity to get real data is not until the system is designed, built, and ready for pilot testing with real callers.

Interviewing Customer Service Representatives

Another approach is to interview customer service representatives in the example just mentioned, the travel agents. Service reps can often tell you a great deal about what customers ask about, what problems come up, typical confusions and misunderstandings, and so on. This is not a substitute for observing real calls. There is nothing as valuable as direct observation of real data, removing the filters that others put on it. However, it can be worthwhile to complement observational studies with interviews of customer service agents.

There are a number of approaches to gathering data directly from users, including focus groups, individual interviews, and surveys, covered in the next three sections. These three approaches have the advantage of direct user contact. However, they all depend on the ability of users to report accurately on their behaviors, reactions, and needs, and users are often imprecise or even wrong when reporting on their own behavior and needs. It is useful to combine one of these techniques with other approaches that directly observe or measure user behavior.

Focus Groups

A focus group is a moderated discussion with groups of four to ten participants (Nielsen 1993; McClelland and Brigham, 1990). Successful moderation of a focus group takes considerable skill. The moderator must keep the discussion on track, focusing on a predetermined set of questions and issues. At the same time, the moderator must keep the discussion flowing relatively freely, allowing for new ideas and directions to emerge. He or she must be skilled enough to identify opportunities for pursuing issues in greater detail and must be able to elicit input from all participants rather than let a few people dominate the session. It is a good idea to start with open-ended questions and then delve into greater detail. A skilled moderator crafts questions carefully to avoid biasing participants' answers. If possible, it is good to run sessions with several different groups.

One advantage of a focus group is that some participants will be encouraged to speak about their own usage patterns and preferences in new ways in response to what others in the group say. A disadvantage of a focus group is "groupthink": Some participants may be less than candid or may be influenced by what others say.

The most typical way to run a focus group is to gather in a room the participants, moderator, and possibly observers (e.g., designers and developers of the system). However, a focus group can also be run over the telephone. Despite the advantages of a live session (e.g., the ability to read body language), an over-the-telephone session is cheaper and makes it easier to achieve a representative and balanced sample of participants. Moderating a focus group over the phone takes special skill and works best with no more than four participants.

Individual Interviews

An alternative to focus groups is a series of individual interviews. The goals may be similar to those of a focus group. The advantage is that the interviewee is not influenced by opinions expressed by others. It may also be easier to delve into greater depth on particular issues that emerge.

Interviews can be conducted either over the telephone or live. In general, they are not much harder to run over the phone than live. Running them over the phone has the advantages of reduced expense and improved ease in reaching a geographically dispersed group of people.

Surveys

A survey consists of a set of questions typically sent through the mail or e-mail. The question format may be multiple choice, check boxes, or brief fill-ins. A survey can be an inexpensive and effective way to collect data from a large number of subjects, although issues are typically pursued in far less depth than with other approaches. The answers solicited must be very brief. There is no way to elicit elaboration on any issues, and no way to be sure that participants clearly understand all the questions.

4.1.3 Understanding the Application

The most obvious part of requirements definition is fleshing out all the application details. All tasks and features must be defined completely. Specific elements include the following:

• Tasks and subtasks: Describe in detail all the tasks and subtasks of the application. This includes a complete description of all information supplied by the caller to the system and all information supplied by the system to the caller.

• Task complexity: Analyze the complexity of each task from the point of view of the caller. How much does the caller need to learn in order to use the application? Will a large set of commands be necessary? How much will the caller need to learn just to start using the system? What is the caller already assumed to know? Answers to these questions will help you determine which approaches to use for teaching and instructing users.

• Recognition challenges: Identify those tasks that are likely to be challenging in terms of recognition accuracy. Examples of challenging recognition problems include spelling and alphabetics, very long lists of items (many thousands), long number strings (because the whole thing is wrong if a single digit is wrong), and so on. For those tasks with challenging recognition, look for sources of leverage that can help. For example, if a long alphanumeric account number must be recognized, find out whether there is a special structure to the number (such as a checksum digit) or whether it is feasible to test the N-best recognition results against a database of existing account numbers. Chapter 13 covers these and other approaches to optimizing accuracy.

• Application environment: The environment in which the application operates includes hardware and software platforms, computer telephony integrations, and databases. It may also include other software systems such as fax-back or credit card verification. The interfaces to these systems may place constraints on the VUI design. For example, if the application must access a database in order to return information to the caller, find out the expected database latency (delay in returning data). If latencies are likely to be long, the application must handle it. For example, you might change the order of tasks so that the system can do useful work while waiting for a database response, or you might play a latency sound or message. Furthermore, you must understand all the failure modes for each of the systems so that you can design appropriate dialogs for each one.

• Other user interface technologies: What technologies, in addition to speech understanding and prompt playback, will be part of the user interface? Will the system use speaker verification, text-to-speech synthesis, and the like? How will these impact the design? For example, if speaker verification is used, will this system be responsible for gathering voiceprints? Will it be combined with other means of user verification such as PINs?

Evaluating Other Company Systems

If the system will replace or complement an existing system, it is a good idea to evaluate the existing system to help define all tasks. For example, speech systems are often implemented to replace touchtone systems. Analysis of the touchtone system can provide a detailed definition of all tasks and features. Keep in mind that the goal of this analysis is not to duplicate the user interface (an effective speech interface is typically very different from a touchtone interface with the same functionality) but rather to understand all the tasks and features.

Meeting with Company Personnel

Marketing personnel should be able to provide you with detailed information on the system's functionality. In addition, meet with developers and integrators, or look at the appropriate documentation, to get a thorough definition of all the behaviors and failure modes of systems with which the application integrates.