Chapter 17: A DirectSound Case Study for Halo

Marty O'Donnell and Jay Weinland

Introduction

Audio the Bungie Way

The evolution of the audio in Halo began with the first two installments of the Myth series of games developed by Bungie in 1997 and 1998. A lot of time was spent laying the groundwork for the audio engine that we developed further during the production of Oni and utilized in Halo. Functionality was developed to allow for multiple permutations, nonrepetitive loops, detail sounds, dynamic dialog, and impacts reactive to velocity and distance, as well as cascading sound effects. While Halo was rewritten from the ground up for the Xbox (after Bungie was acquired by Microsoft), it is important to note that many of the high-level sound design concepts and their implementation into the Halo audio engine are the result of years of development and refinement with Jason Jones and the Bungie programming team, in particular Halo's audio programmer Matt Segur.

We share a philosophy here at Bungie that we like to call "Audio the Bungie Way," which details fundamental audio design for any of our games. At the core of this philosophy is the understanding that repetitive audio is not only annoying but also unrealistic. This is why we allow for so many permutations in every sound call, why music does not play constantly throughout a gaming session, and why we insist on the many customizations to our audio engines.

This core idea means that we spend a huge amount of our efforts in the implementation stage. There are many games released every year that have audio assets of the highest quality, but due to lackluster implementation, they fall short of having the maximum impact. We believe firmly that half of what makes a game sound good is in the implementation. This process begins with the technical design and continues through pre-production, production, and post-production. The process requires the full effort and support of the production and programming teams.

History of Halo Audio

In July of 1999, our cinematic director, Joseph Staten, approached Totalaudio (at that time Marty O'Donnell and Mike Salvatori) about writing a soundtrack for a live game demo of Halo at the upcoming MacWorld Expo to be shown by Bungie's Jason Jones during Steve Jobs' keynote address. Bungie had a scripted demo running in real time through OpenGL on the Macintosh but with no audio code. The music composed for the demo was played in sync by hitting Play on a CD player when the demo began.

Marty describes his approach to the music of Halo and establishing a mood and feel for this ancient, mysterious ring artifact found by humans 500 years hence in some unexplored corner of the galaxy:

I felt that I could evoke an ancient and mysterious feeling by starting with some Gregorian monk-style chanting over a bed of strange ambient sounds and then give the action sequences that followed an epic and important feel by getting orchestral strings from the Chicago Symphony to play over a somewhat rock 'n roll rhythm section. I added an improvised Qawwali chant voice over the top to help reinforce the "alien" nature of the environment. Whether these decisions were the right ones or not doesn't matter. I had two days to write and produce this piece and there simply was no time to ponder or experiment, which is sometimes a good thing.

Since this was also a venue that would feature a big screen, a large auditorium, and a gigantic stereo sound system, I wanted to not only capture the mood but also hook the audience. Anything that sounded like "game music" was going to be a disappointment. Plus, the track needed to be interesting enough in its own right so that the audience wouldn't notice that they weren't hearing any sound effects. It seemed to work out pretty well.

For E3 2000, Bungie had ambitious plans. The team would show the game being played and then follow it up with a ten-minute trailer that hinted at the storyline of Halo and showed the technological prowess of the Halo team. The hands-on demo had merely a rudimentary sound engine, but the trailer was produced in full 5.1 surround sound and played from a DVD in an enclosed theater at E3. Halo was one of the hits of the show, but at that point it was not known that discussions had been ongoing for Microsoft to buy Bungie and bring Halo to the Xbox. Given that the audio capabilities of the Xbox would surpass any previous console/computer platform, this was an exciting turning point in the audio production of Halo, not only because the audio could be carefully directed toward one set of audio hardware, but also because it greatly expanded the scope of what could be accomplished.

Coming to the Xbox

All of the work that had been done on Halo leading up to E3 was cast aside when Bungie arrived in Redmond, Washington, in July of 2000. The team decided to rewrite all parts of the game to maximize the power of the Xbox. This certainly included the audio engine. The audio capabilities were such that we would have been foolish to not take advantage of them. We focused the audio engine on the features that we most wanted to take advantage of, which included 256 voices, real-time 5.1 Dolby Digital encoding, 3D positioning, DSP effects such as reverb, use of the hard drive for audio streaming, and the Xbox ADPCM compression through which the audio chip can play back without any CPU hit.

Before we break down the various areas of Halo's audio as they relate to DirectSound and the Xbox, here's a word on our terminology. The most basic building block of our audio engine is a soundfile, which in our case is an AIFF file, stereo or mono, and 22.05 kHz or 44.1 kHz depending on usage. All soundfiles are grouped as permutations into a soundtag (after ADPCM compression). A soundtag is a file that contains not only the ADPCM audio data but also all relevant information about the playback of the enclosed soundfile permutations such as variable pitch/volume, skip fractions (percentage chance each permutation will play), and soundclass (e.g., "weapon fire" or "unit footstep"). The audio engine does not recognize soundfiles, only soundtags. Soundtags are attached to other types of tags in the Halo engine, such as animation, effect, or particle tags.

The most complex building block in the Halo audio engine is the soundlooping tag. Every element in a soundlooping tag is a soundtag, which in turn contains a soundfile. These tags contain instructions for how to loop the soundfiles including assignments, beginnings and endings (which we call the "in" and "out" respectively), multiple tracks, and detail sounds. These three types of files are all we need to permeate a Bungie game with audio.

Although there are many more complex examples, if we want to assign a gunshot to a weapon, we merely open the weapon tag, go to the slot for the firing action (where we also see the smoke and flame particle effects attached), and attach our soundtag (which has multiple soundfile permutations of a gun firing with randomized pitch/ gain), and we are done. Another example would be creating sounds for a character, such as the Hunter running. We make sound effects based on an animation (AVI or MOV file) and then attach a soundtag directly in the animation tag for the Hunter (which contains the dozens of animations the Hunter might use) in the slot for the appropriate animation.

Lastly, here's a word on soundclasses; we can assign each soundtag to a soundclass that groups similar types of soundfiles, such as a weapon firing, music, unit dialog, or any of a dozen others at our disposal. In essence, our soundclasses function in the same way that sub-masters function on a mixing console. This is an important divisor because it allows us some control over each area of the audio in some key areas.

The soundclass attributes (delineated in the audio code) set such things as the maximum number of soundfiles from that soundclass, which can play simultaneously, whether the soundclass is of an importance where it cannot be late (such as reloading a weapon which needs to be synced to animation playback), or if it could be delayed by a few CPU cycles (such as unit dialog), and also what rolloff values should be used for that soundclass. This is important for us because every soundclass needs to have a different rolloff. For instance, footsteps needed to rolloff rather quickly. Distance rolloff was considered as well. Marine dialog should be heard from a short distance away, while explosions such as grenades should be heard from a great distance. The delay factor was important to our audio programmer, as he was the one juggling our 12,300 soundfiles to play back as necessary within 3MB of RAM. With constant loading and purging of audio RAM, it's crucial for the audio engine to know whether it needs to play a sound immediately or if a 30-to 100-milli-second delay is okay.