Chapter 14: Notifications

Overview

Todor Fay

Sometimes, an application would really like to know when a specific sound or musical event occurs. This is useful for many reasons. For example, if the audio system can report when a Segment is about to finish, the application can select and queue the appropriate Segment to follow. There are plenty of creative paths to follow once you have a robust, easy-to-use system that "notifies" the application when specific audio events occur. Examples of things you can do with a good notification system include:

• Placing signals in a Segment to indicate when specific sounds occur to trigger visual elements

• Synchronizing animation to the musical beat so characters walk and move with rhythm

Think about the corollary between film and games. In the film editing process, sounds often synchronize to visual happenings, and sometimes visual elements cut to the sound or music. You might think of an MTV video for the extreme example, but even action sequences where the music is not center stage can be subtly improved by letting sound drive some of the timing. Rhythm and pacing are far more critical in the audible domain than visual. Think about speech. It works very well to animate lips to words, but it is invariably clumsy the other way around.

So far, with game sound, it's been pretty much a one-way street; sounds are triggered by visual events. But, with the proper tools and techniques, you can change all that and make more compelling environments.

DirectX Audio includes a rich set of options for synchronizing visuals and actions to the sound track. Depending on the notification requirements, DirectX Audio provides different approaches to accomplish synchronization.

• Performance notifications: The Performance Engine provides a standard system for handling notifications. Standard notifications range from Segment start and stop to groove level change.

• Lyrics: A Lyric Track in a Segment provides an easy way to provide notifications that are defined as text names. The beauty of this approach is any text word can be placed in a Segment timeline and then delivered to the application at the specific point in time.

• Script track: A Script Track calls script routines at specific points in the Segment playback.

In this chapter, we talk about all three and add features to Jones to visualize both performance notifications and lyrics.