3.1 Methodological Principles

Five key principles underlie much of the methodology. An understanding of these principles will help you more effectively apply the methodology to projects with real-world constraints. Furthermore, the principles will give you a basis for extending the methodology as new technologies create new opportunities and challenges.

The five key principles are as follows:

1. End-user input: Inform design decisions with end-user input.

2. Integrated business and user needs: Find solutions that combine business goals and user goals.

3. Thorough early work: Avoid expensive downstream changes by focusing on thorough work in the early definition and design stages.

4. Conversational design: Move the design experience close to the user experience so that the designer can experience design elements in their appropriate conversational context.

5. Context: Make all design decisions with appropriate consideration of context.

Let's discuss each principle in detail.

3.1.1 User Input

The designer of a user interface is, in some ways, the worst person to evaluate its usability. Designers know what they intended when crafting the interface. In the case of VUIs, designers know exactly the intended message behind every prompt and system action and tend not to see any ambiguities that may exist. They know how to use the interface, however difficult it may be for others to learn.

This is a problem in the design of all user interfaces. With VUIs, the medium of communication with the user is spoken language. Language functions in context, and effective communication depends on the user's life experience and world knowledge as well as innate and learned linguistic skills. As a result, VUI designers are at great risk that they will not even notice the confusions and complexities that must be faced by users of the interfaces they design

The only solution is to gather input from users to validate and refine design decisions. This idea is the key concept underlying the user-centered design paradigm, which has become the cornerstone of modern user interface design (see Norman and Draper 1986, Rubin 1994, and Cohen 2001). The type of user input you need varies as you progress through the phases of design. For example, early in the process you may focus on understanding users' mental model when performing tasks of the type you intend to enable. In the midst of detailed design, you may run user tests of specific elements to see how easy they are to use.

With each phase of the methodology we present, we discuss the ways of gathering user input appropriate to that phase. As you move from phase to phase, your questions will change, and the techniques for collecting user input will change.

3.1.2 Integrated Business and User Needs

The opening story in Chapter 1, about the first test subject for the Schwab system, showed an extreme example of a system that met a user need without meeting business goals. Delighted though the subject was in finding a patient, conversational partner, no brokerage business was conducted! Although designers must be advocates for end-user needs, they must always work with a clear understanding of the business context.

Many steps in our methodology, especially in the early phases, are focused on achieving that understanding. Many dimensions of the business must be clarified, including, for example, understanding how the application will fit with all the business's other means of touching its customers, its branding and image needs, and so on. Many design decisions are influenced by the business context. For example, business goals influence design trade-offs, branding needs influence creation of a system persona or voice, and so on. The job of the designer is to meet the needs of the business and its users simultaneously.

3.1.3 Thorough Early Work

In the software industry, many years of experience have taught developers and project managers that the most efficient way to deploy successful software is to follow a process that includes thorough analysis of requirements and high-level functional design before developers dive into the details of specific software modules. Skipping the early steps has led to many software disasters (McConnell 1996).

Thorough early work is equally important for VUI design, for many of the same reasons. If you have a comprehensive understanding of the application from both the user and the business point of view before you dive into the design details, you greatly reduce the risk of making poor design decisions. It is far more expensive to make design changes later in the design process or, worse yet, after implementation is complete. A detailed understanding, up-front, of end users' needs will help you make effective design choices. Early user testing can provide early design guidance and avoid the risk of extensive changes late in the process.

Additionally, making high-level design decisions up-front facilitates consistency throughout the design, which is one of the keys to usability. For example, if you choose the persona or character of the application early, you can use it to guide the crafting of prompts. In this way, you will create a consistent character that is reinforced throughout the application.

The methodology we describe has a significant focus on the early stages (see Part II, Definition Phase). Our experience has proven that thorough early work greatly reduces the risk of poor usability and the need for reworking designs.

3.1.4 Conversational Design

Imagine painting a picture without ever seeing it. For example, suppose you are using software that lets you type commands such as "Place a three-inch red square in the upper-left corner" but never displays the image you are creating. Even an experienced painter would not have a good result using this approach. Whether you are creating a work of art, a consumer product, or a user interface, to create an effective design you need to envision the end product as you shape the elements.

Often, VUI designers get buried in the details of the mechanics of specific dialog states and lose sight of the larger context of their design decisions (the conversations).^[1] The elements of the dialog state (e.g., initial prompt, prompt to play in case of recognition reject, prompt to play in case of timeout) are often crafted one after another, without regard to the actual conversational flow in which they would occur. The result is language that sounds stilted, unnatural, and less comprehensible than the designer expects.

^[1] A dialog state is roughly defined as a single interchange between the caller and the system (i.e., a prompt followed by a response), along with all the special handling that supports the interchange. Examples of special handling include what the system says to get the caller back on track if there is a problem such as a recognition reject, a timeout, a caller request for help, and so on. Chapter 8 has a more detailed definition.

Chapter 8 describes a number of design approaches that help you consider design elements in their conversational context, thus bringing your experience closer to that of the user. The result should be more naturally flowing and more comprehensible dialog. We all have tremendous skill and lots of practice engaging in conversation, even if we are not consciously aware of the underlying structural elements involved. Given that much "knowledge" of conversational structure is unconscious, techniques that help you imagine conversations as you design will allow you to bring to bear more of your unconscious knowledge of conversation as you craft prompts and make other design decisions.

3.1.5 Context

All design decisions must be considered in context. The relevant context ranges from high-level considerations, such as business goals and user needs, to low-level details such as the role of a particular prompt in moving the conversation forward.

Let's look at the contextual considerations involved in a single design decision. Consider a travel application that is in the pilot stage and is experiencing lots of recognition rejects in the GetDate dialog state, where the system asks the caller the date of travel. Imagine that the pilot data show lots of unexpected fillers (e.g., "Um, well I need to get there Tuesday for a Wednesday meeting") as well as unexpected formats for specifying the date (e.g., "I wanna leave a week from Tuesday"). We decide to use the following new prompt when a recognition reject occurs in the GetDate dialog state: "Sorry, I didn't get that. Please say the date of travel. For example, you could say 'March tenth' or 'June sixth.'" The contextual considerations of this prompt include the following:

• Application need context: We know at that point that there is a recognition reject, so we need to reprompt for the date.

• User context: The data show that the users are using fillers and date formats that the grammar cannot handle. Although we can add more variations to the grammar, we see enough variation that we decide it would hard to achieve high grammar coverage, so we decide to provide more guidance to the user.

• Language-use context: We know from experience and published work that callers tend to pattern their utterances after the prompt, especially if examples are provided.

• Discourse context: Let's consider the role of this new prompt in the conversation. The discourse goal has moved from information gathering (soliciting the trip details such as origin and destination) to a need to resolicit information. Not only is this a new role in the discourse, but also it is an unexpected shift. The caller probably expects to continue his or her travel plan, but the system needs to backtrack. We want to signal the change to the caller, both to avoid confusion and to encourage help. "Sorry, I didn't get that" signals the caller that we are changing mode and explains why.

• Persona (branding) context: Assuming that the company image calls for a friendly, helpful persona, we want to signal the new discourse goal and be very clear about how the user is to respond. If we were seeking to create a different persona, we might have chosen different words. For example, a more formal persona might have started with "I'm sorry, I didn't understand."