Synchronization, Tools, and Formats
| Building on what you have already learned about producing digital audio, you'll be able to utilize a variety of tasks specific to producing sound for video:
All of these tasks involve processes we've looked at, from recording, editing, and mixing to scoring notation and using virtual instruments. In addition, soundtrack work requires coming to terms with a new issue: synchronization. Matching Sound and ImageSynchronization in video extends both to literal sync of image and sounds and to the looser synchronization of image and music. You'll obviously need dialog, sound effects, and other sounds to synchronize exactly with what's onscreen so the sound matches the action. Music is more loosely synchronized based on where you want it aesthetically. Music doesn't always need to line up exactly with picture, but it may need to start or end at certain times (like the beginning of the opening credits or a change of scene) and to line up with specific events onscreen for maximum impact (like a swell of music when the villain in a horror flick appears). These specific events are called cues or hit points . Because you're matching up two different elements, audio and visual, it helps to have a reference as you're working. Usually, this is in the form of a video file, which gives you both a reference to the timing of the video you're scoring and the literal visual content of the movie itself. The video will usually run in a small window in your DAW or audio editing software. It will automatically play back in synchrony with the audio. In addition, you may see a visual indication of the video time in the form of a burn-in , in which video timecode is overlaid on one corner of the picture. It's also possible to view the video in a VCR (if you still have one) while scoring the music, but this requires expensive synchronization hardware and entails other headaches , so almost all composers who work with film and video have moved to computer-based video playback. You're already accustomed to working with two kinds of time in digital audio: clock time (3 minutes, 47 seconds) and musical time (4/4 time, 90 bpm, bar 17, beat 3). With audio alone, you rarely have to convert between the two: if two tracks are lined up at 120 bpm, it doesn't really matter when events occur in seconds. Working with video and music, though, you'll need to be able to make those conversions. You'll also need to resynchronize media that were recorded separately, such as audio that was recorded on a separate recorder from the video. Video Capabilities in Audio SoftwareWorking with video makes new demands on your audio software. Here are some of the features you'll want to look for when using music software during the process of producing sound for video:
Most often, the tool that fulfills these requirements is a DAW. Pro Tools, SONAR, Cubase, Logic, Digital Performer, Sony Acid, and Mackie Tracktion all have integrated video features, as do tools specializing in video scoring like Apple Soundtrack Pro and Adobe Audition. Using these tools, you can record dialog and sound effects, and edit and clean up an existing soundtrack from the video. If you're creating your own original music soundtrack for video, you have several choices:
Managing Video FilesThere are two reasons you might need to work with video files when doing sound scoring. The first, obvious reason is that you'll need to look at the video for reference. The second is that you might need to embed your own audio in the video, either to ensure that timing is synced in a final project or to provide a mock-up for a director or collaborator of how video and sound will mesh. WorkflowsWhen it comes to working on video, workflow, which is a fancy word for the order in which you complete a series of tasks on a project, becomes a vital consideration. You might edit all your audio first, then bring it into a video editor. You might take a preedited video and create music that fits it, or (as would be the case in a music video), you might create the music first and bring that into a video editing application, making the music the reference while you assemble a video. Your workflow becomes more significant if you're collaborating with other people in the phase video producers call postproduction , or simply "post." (Postproduction is, literally, everything that happens after the video has been shot.) A standard video production workflow is to shoot first, then edit the video, then edit sound. In this case, a video editor will cut the picture, then send all the audio at once for editing by a sound editor, who will return the edited sound to the video editor for the last stage in the video mastering process, which is referred to as the final cut. Even in professional work, this process isn't totally fixed: film composers, for instance, often work from rough cuts of the film and wind up making revisions to the music as the film changes. If the video editor trims two and a half seconds from a scene, the composer is expected to edit the existing music so that it still fits the shorter scene. Even if you're just one person, you'll probably need a way of moving video and audio content between your audio editing application and your video editing application. If you're working with other people, you'll need a way of sending projects back and forth. This could be as simple as receiving a low-res copy of the video to use as reference, or as complex as sending full multitrack audio for each scene as it's edited. File formats and encodingDigital video files are necessary both for exchanging video files with other collaborators and for keeping a reference for video scores and soundtracks. You'll find a variety of typical formats, including Apple QuickTime (.MOV) files, Microsoft Windows Media files (usually labeled .AVI), and MPEG files (.MPG or another .MPx extension). Just because a file appears to have a supported file extension doesn't necessarily mean your music software will be able to read it. MPEG files are available in different flavors like MPEG-4 and MPEG-2, and QuickTime and Windows Media files may require the current version of the player software (usually a free download). Furthermore, QuickTime and Windows Media formats are often compressed using different codecs (short for coder -decoder). Fortunately, downloading the latest QuickTime or Windows Media software will let you handle many of these formats.
The preferred file format for audio production for video remains Apple's QuickTime, which supports a variety of video compression codecs and content types. Despite the Apple brand name , QuickTime is often preferred even by PC users, because so many video and audio applications on Windows and Mac alike are able to open and save QuickTime .MOV files. As a result, this format is popular for sharing videos . To work with QuickTime files, you'll probably want to invest the $29.95 needed to upgrade to QuickTime Pro, which unlocks a slew of conversion options for audio and video on both Windows and Mac ( Figure 12.3 ). QuickTime Pro is adept at handling both Apple formats like .MOV and AAC and standard formats like MPEG-4. Using QuickTime Pro, you can also capture video and audio and create surround sound. If you're on a Mac, you can automate video conversion tasks using Mac OS X's Automator and AppleScript. If you're a Final Cut Pro user , you'll have additional encoding options using the Compressor application included with Soundtrack Pro, Final Cut Pro, and Final Cut Studio. Figure 12.3. Apple's QuickTime Pro serves as an inexpensive Swiss Army knife for handling video on both Windows and Mac, with extensive import/export options and basic editing.
Windows Media videos (.AVI files) are also typical in video and audio work. (Windows Media files can be opened easily on the Mac using Windows Media Player for Mac, but not all audio and video applications support it.) Unlike Apple's tools, most of Microsoft's tools are free. You can handle many conversion tasks in the Windows Media Player application and, for heavy-duty batch conversion and encoding, can download the free Encoder tools. The Mac encoding tools for Windows Media are more limited than the Windows tools, as you might expect, but can still be worth a download. If you're using video only for reference, you won't need to use a full-resolution file. Reducing the quality of the file can conserve hard disk space and CPU bandwidth, so that you can devote more resources to your audio work instead. You'll want to maintain the original frame rate (the number of frames displayed per second), but you can use a lower display resolution (like 320x240 instead of 640x480 or HD) and compress the video. Integrated toolsIf all you need to do is import video footage, edit the soundtrack or add a score, and give it to your video editor, opening a small QuickTime file or sending it to a collaborator may be all you need to do. But what if you need to edit audio tracks while working on a video project? A major selling point of the leading video and audio suites is their ability to select audio in a project, switch over to an audio application to edit it, and modify the original video project without manually exporting and importing. Since a single manufacturer (like Apple) sells a suite of products (including Final Cut Studio), it's in the manufacturer's best interest to make it easy to integrate the included applications.
For instance, using Final Cut Studio, you can select audio in Final Cut Pro, DVD Studio Pro, or Motion, and by right-clicking on audio (Ctrl-clicking on a one-button Mac mouse), send it to Soundtrack Pro for either destructive or nondestructive editing ( Figure 12.4 ). You'll see the associated video in Soundtrack Pro while editing, and when you save, the audio will automatically update in the original project. Similarly, in Sony's Vegas Pro for Windows, you can use ACID files and loops directly from the interface, and by right-clicking on audio in the timeline, can choose Edit Source Project to make changes in your original ACID audio project. Figure 12.4. Via an integrated suite, you can edit audio in place, nondestructively, with the associated audio tool. Here in Apple's Final Cut Studio, you can send audio from an application like Final Cut Pro to Soundtrack Pro for editing. These features are especially useful if your video project and audio project are both on the same machine, since they facilitate switching between video and audio applications. However, by exporting and importing video and audio files, you can use these features to exchange files remotely as well, in case your video editor is in Melbourne, Australia, and you're editing audio in Madrid. The projects may include not only basic audio and video information, but also editable multitrack sound complete with plug-ins, edits, and automation.
Interchange formatsWorking with integrated software suites works well for many purposes, but that doesn't mean you have to limit yourself to working with applications that share a brand name. File exchange formats like OMF, AAF, MFX, and Final Cut Pro/XML all allow easy import/export of multichannel audio projects. At the very least, these formats include multichannel audio information as separate, editable channels; also, some formats (AAF, MFX, XML) include additional information like track automation. (See Chapter 10, "Put It Together: Arrange, Mix, and Master," for more details on these file formats.) Support for formats like AAF is growing not only in DAWs but in other media programs like Adobe After Effects and Premiere. OMF is popular in postproduction because the format, developed by Avid, is compatible with Avid video systems, Pro Tools audio systems, and many DAWs and video programs. A typical workflow might include exporting multitrack sound from Pro Tools as an OMF and opening it on an Avid or other video system, or vice versa. Other formats include application-specific information. Apple's Final Cut Pro/XML format, for instance, lets you move audio between Logic Pro and Final Cut Pro. You could assemble a multitrack audio soundtrack in Logic, and then export it to Final Cut with mixing settings intact and editable. Time, Frames, and SMPTEIn order to properly measure time in video, for synchronization and cueing, you need a standard way of calculating locations in time. It'd be pretty useless to figure out that you wanted an orchestral hit to be heard at a certain point in the action if you couldn't describe exactly where that point was, and you can't synchronize two media files if you don't know where the correct starting point is. The standard means of expressing time in film and video is SMPTE timecode , a set of industry standards for designating each frame of the video. SMPTE timecode (usually pronounced "simp-tee") is used both to synchronize equipment, by providing a standard frame of time reference, and to measure relative location in video and film. Technically, it's called SMPTE/EBU, named for the Society of Motion Picture and Television Engineers (www.smpte.org) and European Broadcasting Union (EBU/UER, www.ebu.ch), the standards organizations that developed it. Unless you're connecting multiple pieces of pro video gear, one thing you won't have to worry about is how SMPTE is used for synchronization. Instead, you'll probably first see SMPTE timecode as a simple series of numbers in this format: Hours : Minutes : Seconds : Frames (xx : xx : xx : xx) For example, 2 hours, 7 minutes, 23 seconds, 14 frames would be written 02:07:23:14. The nuts and bolts of how this works when it's encoded in media like film and digital video (DV) tape are a great deal more complicated, but for most tasks you only need to understand how to read the timecode as it's displayed in your DAW or video tool ( Figure 12.5 ). (Technically, your DV tape or QuickTime file uses its own means of encoding timecode digitally, not literal SMPTE timecode signal as is used on analog tape and film. But the formatting of minutes, seconds, and frames is displayed to the end user the same way.) Figure 12.5. Transport controls of programs like Cubase SX can display timecode in various frame rates, alongside musical measurements of time. There's one twist: what a frame means depends on the frame rate . TV, digital video, and film all provide the illusion of motion by displaying a series of successive still images called frames. They do so at a constant rate, but what that rate is depends on the medium and even the country you're in. Table 12.1 contains a comparison of commonly found frame rates. Table 12.1. Common Frame Rates (frames per second, fps)
You'll need to make sure your audio software is set to display timecode at the correct frame rate so you can properly locate each frame. These frame rates are mostly straightforward, except for the 29.97 fps rate used by default for NTSC in certain parts of the world. Because of a quirk of engineering, when American broadcasters tried to create a new color broadcast format that was compatible with black-and-white receivers, they had to use the oddly numbered frame rate (29.97 is an approximation ; the rate is actually an even stranger 30 x 1000/1001 fps, or 29.97003 fps). The 30 drop frame rate is the same as 30 fps, but " drops " frames at regular intervals so that it doesn't fall out of sync with the 29.97003 fps rate. (If you're in Europe or most of the rest of the world except the United States or Japan, you don't have to worry about drop-frame, because it's not used by the popular PAL format.) Bizarre frame rates and regional differences aside, what's important is that all your audio and video be set to the same frame rate. You can synchronize media that uses different frame rates, but it can cause a range of problems. If you're involved with a project in which you're mixing film and video, for instance, you may want to communicate with the postproduction house or processing facility that you're dealing with. At the very least, make sure you know in advance what your target medium is, and ensure that all your recordings match up. |
