Chapter 18: A DirectMusic Case Study for Interactive Music on the Web

Download CD Content

Ciaran Walsh

Overview

Background

Web content today consists of sophisticated graphics, animation, and interactive visual content. By contrast, web sites use sound only sporadically, often as the occasional button sound or introductory flourish. Web sites use music even less frequently, and this music is often repetitive and of poor sound quality. The difference that sound and music can make in a web presentation is enormous, however — just as it is for other audiovisual media, such as film or television. Music has the power to set a mood appropriate to the content and enhance the overall impact of a web site. Appropriately, for an interactive medium like the Internet, interactive music can enhance a web site further still. It can contribute to the usability of the site, assist navigation, and add emphasis to key content.

Bandwidth is the main cause for both the lack of quality and use of audio content on the web. Standard uncompressed sound files (i.e., .wav and .aiff) are large and therefore take a long time to download over slow connections. Compression reduces the size of sound files but deducts from the quality of the file. Poor sound quality is inconsistent with the high production values expected of quality web content. While short sound effects are usable, short, repeating music loops become tedious to the listener very quickly.

The General MIDI (GM) standard is one solution to the bandwidth problem, playing MIDI files using standard instrument mappings to ensure that the sequence performs using the correct instruments. The benefit of using General MIDI is that small MIDI files are the only sound-related file required for downloads. The trouble with General MIDI on the other hand is that the instruments are limited to a defined set of 128, mostly variants of standards, such as piano, guitar, and basic orchestral instruments, and tend to be poor quality. General MIDI instruments can vary dramatically in timbre and quality from one sound card to another, despite the intended standardization. Any digital musician knows, for example, that piano sounds on two different brands of sound cards can be drastically different from one another. This difference can wreak havoc on the balance and feel of a piece of music.

Streaming music is one alternative to General MIDI synthesis. Streaming consists of downloading sound files bit by bit as they play, allowing playback to start relatively quickly, thus decreasing the perceived loading time of the site. Fast streaming requires a significant amount of compression, even with broadband access becoming more commonplace. Also, streaming ties up valuable bandwidth for the duration of the music file. This can slow down the loading of other pages or cause the music to stop as other pages use bandwidth to load.

Looking beyond the bandwidth problem, it is worth considering that web content is interactive by definition. Standard music delivery methods are designed to play a sound or MIDI file from start to finish and do not allow for user interaction. As web content is inherently interactive, linear music (regardless of how it is delivered) can never truly do justice to the unpredictable nature of the user experience. For this, we need a system that is not only bandwidth efficient and provides high sound quality but is also capable of responding to user actions flexibly and in real time.

DirectMusic is ideal for delivering music over the web. Its use of the DLS standard for synthesized instruments makes it bandwidth efficient. For instance, a drumbeat using a DLS instrument may consist of a kick drum sample, snare drum sample, and two hi-hat samples. All of these samples together are likely to amount to less than one measure of the whole beat but allow for endless permutations and fills, while treatments such as panning and filtering further add to the possibilities. The same amount of sample data required for a single prerecorded drum pattern as part of a stream is therefore able to produce countless variations. A sound designer can manipulate and reuse samples in new instruments without adding additional material to the download. Furthermore, the sound designer can compress the samples on a case-by-case basis, allowing for the best compromise between size and quality. In addition, since the user downloads the actual DLS instrument as web content, the music sounds consistent across the full range of playback hardware. The ability to use real-time effects such as reverb, chorus, delay, and distortion further enhances the potential to make the most out of a few samples, as well as adding depth to the final mix.

DirectMusic scripting adds the vital ingredient of interactivity. DirectMusic content creators feed button clicks, mouse movements, and other triggers into the script and use them to shape the music in accordance with the user's interactions with the site. It is also possible to give the user direct control over aspects of the music. This control can range from something as simple as a volume control to much more abstract control, say, over subtle harmonic changes; the possibilities are limitless.

DirectMusic Styles and patterns allow for a great deal of control over variation, making it perfect for background music that needs to remain interesting over a long period of time (for instance, while the user is reading the contents of the site). Within a pattern, you can create up to 32 variations of each pattern part and control the playback of these in a variety of ways. Parts, and even individual notes, have parameters that can apply a degree of randomness to playback. It is possible, with these features, to create self-varying musical accompaniments guaranteed not to repeat in a predictable way.

DirectMusic's efficiency, variability, and interactivity make it uniquely suited to use on the web. When I first investigated the possibility of using it in this way, however, there was no available means of communicating user interactions to a DirectMusic script. Therefore, it became necessary to create an ActiveX control to do this.

The Wide Sounds DMX Control

Wide Sounds identified the potential of DirectMusic for music on the web. We created the DMX control, an ActiveX control that accepts various control messages and converts them into routine or set variable calls and then passes these calls to DirectMusic. The DMX control can also extract information from DirectMusic and pass it back to the caller. The data for the control is stored in a .cab file on the host's web site and is downloaded and extracted when the control loads. Thus, a web page can use the DMX control as a generic interface to DirectMusic's capabilities.

Figure 18-1: The DMX control in action.

Prior to the development of the DMX control, using HTML+TIME (Timed Interactive Multimedia Extensions — extensions to HTML adding timing and media synchronization support) was the only way to play DirectMusic content from a web page. This approach did not provide the vitally important scripting functionality. In any case, Internet Explorer 6 and later no longer supports HTML+TIME.

Creating the DMX control enabled us to have much greater interactive control over the playback of DirectMusic content on the web than was previously possible. By allowing a web designer to call routines and set variables in a DirectMusic script from user interactions, a vast amount of interactive functionality becomes available.

Functionality

The potential uses of scripted music for a web site are almost limitless, but here are a few of the key techniques and components that I find useful when designing interactive music for a web site:

• Play Segment: Play a specific piece of music for a content area, or start a new piece when clicking through to a different part of the site.

• Transitions: Script changes to the primary Segment in such a way that they provide a seamless transition in order to maintain musical coherence and flow.

• Motifs: Experiment with using motifs over the background music.

• Chord tracks: Map out the harmony of a Style, allowing you to fit secondary Segments such as motifs harmonically over the background music.

• ChordMaps: Vary Styles harmonically using the branching functionality of ChordMaps. Anything from simple key changes to random chord progressions are possible.

• Mix effects: Use specially designed secondary Segments to manipulate a wide range of mix parameters. You can adjust the volume of different instruments, real-time effect values such as delay feedback, DLS instrument filter cut-off, and many more useful parameters for livening up a mix by using the continuous controllers, the Pattern Tracks, or Parameter Control Tracks.

• Randomness: Use script routines to assign a random number to a variable. This enables interactions to have an unpredictable effect. You can control the probability of outcomes to keep the randomness within reasonable limits. You can also give variations within a Style a degree of randomness, further avoiding repetition.

• Timers: Use timers to lock and unlock features during transitions, control queues of events to avoid clashes, or force variation after periods of inactivity. You can create timers by placing script routine calls at a certain point in a Script Track within a secondary Segment. Timers are a very useful under-the-hood element, especially in complex projects.

• Automatic variation: Embed routine calls in Script Tracks utilizing any of the above-mentioned features. This is a useful way of avoiding too much repetition.

Consider these techniques as a "toolbox" of features with which to enhance the content of a web site.