Music for Video

Music for Video

Before we look at the thornier issues of exact synchronization as it's used in dialog and sound effects, let's start with the rougher sync that's used in creating music for film and video. While dialog that's even a frame or two out of sync is immediately noticeable, music often benefits from having different internal timing than what's happening onscreen. Just ask anyone who's played Pink Floyd's Dark Side of the Moon to the movie The Wizard of Oz : sometimes accidental synchronicity is more interesting than intentional synchronicity. Making music work in terms of timing involves balancing intentional synchronization (usually within a few frames ), and more flexible, unintentional synchronization. For those moments when you do want control over how music is fitting to picture, digital video and audio software provides a number of tools for aligning the two.

Spotting and Hit Points

The music swells as the heroine leans in for a kiss. A metallic scrape of piano strings screeches as the murderer's hand slides across the knife . Duh-duh: A string motive tells you that Jaws is lurking underwater. As comedians say, "it's all in the timing." Although there are many expressive elements to film scoring other than just lining up cues, timing is an essential ingredient.

A hit point is an event in the video, a cue, at which you want some musical event to coincide with what's on screen. Even before any sound is added, just deciding what will be a hit point is an art in itself. Choose too few hits, and music will just sit over top of the film as background, lifeless and failing to interact with the image. Choose too many, and the music becomes distracting.

Spotting is the act of watching a video, usually many times over, and finding points that make effective hit points. A master film composer can slip in subtle hit points so they trigger an emotional reaction without you even noticing the coincidence . That's why a lot of energy and finesse goes into the spotting process.

Before computers, film composers like Bernard Herrmann and Alfred Newman would spot using pencil and paper. Now, you can use markers to keep track of hit point locations ( Figure 12.6 ). Most DAWs have a marker function, so while watching a digital video of the material you want to score, you can add a marker each time you come across a desired hit point. (It's a good idea to memorize the keyboard shortcut for adding a marker.)

Figure 12.6. Spotting with markers involves selecting points at which you'd like something to occur in the music in a DAW like Logic Pro. With the crucial points marked in real time (SMPTE time), you can refer to these landmarks later to adjust sound and music to fit.

Adjusting Tempo

Once you've marked the hit points, your second challenge is to find a way to make music fit them. Music is usually measured in beats and bars, not seconds and frames, so if you're creating regularly metered music, some thought needs to go into tempo and meter. Usually, you'll assemble most or all of your hit points first, marking their timecode locations, and then adjust meter and tempo to see how the music's beats and bars correspond to the video's seconds and frames. (You may need to use a "lock markers" feature to make sure you can remap tempi while leaving your time markers in place, since it's the music that's changing, not the video.)

Finding a musical flow that fits the video depends on what's happening onscreen, and on how the video is edited. Expert video editors often build a musical rhythm into their edits, either intentionally (by watching timecode) or intuitively. Choosing certain tempi can help images onscreen fall naturally into the musical rhythm. Multiples of 60 work especially well. For instance, at 120 bpm (beats per minute), two beats will fall every second; that's every 12 frames at 24 fps, or every 15 frames at 30 fps.

For a simple score, you might opt for a 4/4 meter at 60 or 120 bpm and then leave everything else to the music. You won't want regular meters and static tempi all the time, though. Some applications, like MOTU Digital Performer and Steinberg Cubase SX, include tools for automatically calculating initial tempo and subsequent tempo changes so that even gradual changes can be shaped to fit hit points ( Figure 12.7 ). If your DAW can't do this, you can try adjusting tempo envelopes/automation, or simply make changes and compare the timecode values in your Markers window to the resulting musical values.

Figure 12.7. MOTU Digital Performer can automatically calculate different tempi, displaying how they'll fit with the markers you've selected. At some tempi, beats will come close to certain hit points, whereas others are farther off; they're displayed in descending order of accuracy.

You won't necessarily spot on every project. The phrase "cut picture to music" refers to editors who cut the film to the music's rhythmic events rather than the other way around. Some directors will even find existing music that naturally fits the rhythm of editing, without substantial modification to either. Digital audio technology makes these techniques easier than ever: you can quickly try a what-if scenario and see and hear what happens. You can also cut segments of recorded audio without micromanaging tempi and meters, for a relaxed relationship between sound and picture.

Timing and Notation

Traditionally, working with notation software has meant spotting first, then moving to the notation software to do the actual scoring. A composer would set up markers in a DAW like Digital Performer, then either manually copy them into a program like Finale or export tempo and meter changes as an SMF file. Some composers still spot on pencil and paper, watching the video on a computer or TV display.

See spot hit: Digital video is now so precise that some composers and editors find spotting has gotten too accurate. Instead of carefully spotting to the exact frame where a hit occurs, they play the video at normal speed and intuitively hit the add marker keystroke where they feel the hit should go. Being slightly off can make the hits seem more organic.

Once video is integrated with notation software, however, it's possible to spot timecodes within the notation application, which is much more flexible ( Figure 12.8 ). Using Sibelius 4 (www.sibelius.com), you can import video, play back video in sync with the score, mark hit points, and export an audio rendering of your score with the video file. The quality of such a rendering is limited, but it's very convenient for running the music past a director before you hire musicians to play an expensive session.

Figure 12.8. Sibelius's Hit Points display lets you spot video from within the scoring application. Timecode and bar/beat/tick are displayed side by side, so you can see how each cue will line up in musical time.

Whichever way you choose to work, you'll want to print timecode indications above the score, as shown in Figure 12.8, with annotated markers for significant hit points. That way, you'll know which points you most urgently need to hit, and you'll have a reference for lining up the score with the video when you record a performance with live musicians.

Performing with Live Musicians

Working entirely in the hermetically sealed realm of digital audio and video, resolving timing issues is reasonably easy. Working with live musicians, though, means mixing accurate digital technology with decidedly organic technology (namely, human beings). That doesn't make computers any less handy. You can either designate a conductor to keep players in time with the music, have ensemble members watch a display, or give players headphones and play a click track as they record their parts . (A click track is a generated metronome that corresponds to the tempo of the score, including any meter or tempo changes that might occur.)

Many DAWs include oversized or scalable timecode displays ( Figure 12.9 ). Even in high-end professional settings, many conductors simply project these displays so that they can watch the timecode and the score at the same time. Alternatively, with real-time display of scores improving and simultaneous video playback possible in DAWs and programs like Sibelius, they can simply open a video window, open a timecode display, and hit Play, watching the score from a screen. To replicate this Hollywood setup on a budget, you'll want a projector or a large television or computer display so you can see what you're doing while you record.

Figure 12.9. An oversized timecode display like Logic Pro's is useful for dragging to a second monitor or projector so you can watch the counter while live musicians play the score.

Exporting Audio for Video

Once you have your mix, you'll want to export your audio in a form that makes it easy to drop it into a video editing application. We'll look at synchronization issues in the second half of this chapter, but suffice it to say, whether you're doing the video editing and postproduction or someone else is, exporting the audio in a suitable file format is essential.

Most of the time, you'll want to export as follows :

1. AIFF or WAV file

2. 48 kHz, 16-bit audio

3. The file should contain a "2-beep," a short beep two seconds before the beginning of the actual audio; see the sidebar, "The 2-beep," p. 511.