10.1 Multimodal Interaction for Ubiquitous Access

XForms' separation of the model from the view makes it ideally suited to become a key building block for enabling ubiquitous access to Web applications and Web services. We focused on putting a human face to such Web services using XForms in Chapter 9; this chapter focuses on ensuring that the human facing interfaces we create are accessible from a variety of interaction modalities and access devices.

The focus in this chapter will be on accessing such information services with devices that vary in their capability; we will turn this around in the chapter on accessibility (see Chapter 11) where we will focus on ensuring that these same applications can be delivered to users with a wide range of capabilities. Notice that multimodal access to information services and ensuring the accessibility of such services to all users are, in fact, two sides of the same coin. The XForms design leverages this fact in solving the issues of multimodal interaction and accessibility by depending on the same set of features inherent in the XForms design.

10.1.1 Multimodal Access

With the coming of age of speech technology, spoken interaction can be integrated as a first-class modality for creating multimodal user interaction. Such multimodal interaction is especially relevant in the context of mobile devices where traditional user interface peripherals may not be available or appropriate. Rich multimodal interfaces that integrate new user interaction modalities will need to be based on a set of user interface metaphors and design principles that leverage the presence of these new forms of interaction. These metaphors will manifest themselves within application and user interface frameworks where they play the role of integrating user input arriving from different modalities and in synthesizing output to multiple media.

The next section will enumerate a set of user interface principles that are essential for good multimodal interaction. The remaining sections of this chapter will demonstrate how the various components in the XForms architecture aid in realizing multimodal interfaces that satisfy these principles.