Understanding DES Concepts

Understanding DES Concepts

Before diving headlong into the programming for DES, we need to cover some of the concepts key to understanding how DES is organized. The basic unit of media manipulated by DES is known as the clip. A clip can be either an audio or a video segment, and it can be any length. A short clip could be a single frame of video footage, just one-thirtieth of a second, while a longer one could stretch for many minutes. In addition, there s no preferred format for either video or audio clips. DirectShow converts all clips to a common format that is, if it can locate transform filters to handle the conversion from clip format to its common format. Clips are sometimes called source objects because they are the ultimate source material for any editing project.

The editing process consists of placing clips together into a timeline. The timeline is the key feature of a DES application. You can visualize a timeline as a line that extends horizontally from left to right, and as you move from left to right, you progress chronologically. Clips are arranged along the timeline with DES commands; you can set the start and stop time for any clip, relative to the timeline, placing each clip precisely where you want it.

A minimum of two independent timelines are available for your use: one works with video clips, and the second is for audio clips. Many video clips have their own audio soundtracks, and keeping the timelines separate allows you to score a video clip with a new audio soundtrack. Although the timelines are separate, they keep time together. A point on the video timeline will be rendered into the final movie at the same time as the corresponding point on the audio timeline.

Each timeline is composed of any number of tracks. You might be familiar with this term from audio recording. Most modern albums are recorded onto 48 separate tracks, each of which can be recorded independently. When recording is complete, these tracks are mixed together by the album s producer into a finished product. That s the way tracks work within DES; each timeline has as many tracks as required to accommodate the clips that are to be mixed into a movie. You can have multiple clips on a single track, but clips can t overlap on the same track. In other words, if you want to fade from one video clip into another (using a transition, which we ll discuss in a moment), you place the clips on two different tracks in timeline positions that would cause the end of one clip to overlap the beginning of the next. You can do the same thing with audio tracks and fade from one sound to another, or you can mix audio tracks together. A collection of tracks can be gathered into an object known as a composition. The composition is a convenient way to group tracks together and manipulate them as a unit. Finally the entire collection of compositions composes the timeline. Again, video and audio keep separate timelines, so they have separate collections of compositions, which are composed of separate tracks, each containing clips. Figure 8-1 shows the timeline model.

Figure 8-1. The timeline model, which gathers objects into tracks and tracks into compositions

Clips can be reused; just because you ve put a clip on one track doesn t imply that you can t use it elsewhere. In fact, you can reuse clips as often as you like throughout a track, composition, or timeline. You can get some very interesting effects this way, including repetitive sounds, video patterns, and so on.

The final elements in the DES model are transitions. A transition is an effect such as a fade from one video clip to another that is added to a track and is applied to the rendered movie. Although transitions reside within a single track, they can affect other tracks. For example, you could use a fade effect in one video track to fade from the current clip within that track to a clip within another track, which means that transitions can and often must reference tracks outside themselves. Beyond transitions, which define how one track ends while another begins, there are track effects, which allow you to change the appearance of a source clip (for example, transform color to black and white) when it s rendered into a movie. A rich set of both transitions and effects is built into DES, for both audio and video. Nearly anything you might want is already available to you in DirectShow.

Source clips, transitions, tracks, compositions, and timelines are the basic elements manipulated by DES. Let s continue our exploration of DES with a simple console-based program that simply displays a list of the available audio and video effects and transitions. You can also create your own custom effects and transitions using the Microsoft DirectX Media 6.0 SDK, which can be downloaded at http://www.microsoft.com/downloads/release.asp?ReleaseID=45808.

Listing DES Effects

This short program, DESList, allows you to list all the video effects available on your computer:

// Enumerate all of the effects. HRESULT EnumerateEffects(CLSID searchType) { // Once again, code stolen from the DX9 SDK. // Create the System Device Enumerator. ICreateDevEnum *pSysDevEnum = NULL; HRESULT hr = CoCreateInstance(CLSID_SystemDeviceEnum, NULL, CLSCTX_INPROC_SERVER, IID_ICreateDevEnum, (void **)&pSysDevEnum); if (FAILED(hr)) { return hr; } // Obtain a class enumerator for the effect category. IEnumMoniker *pEnumCat = NULL; hr = pSysDevEnum->CreateClassEnumerator(searchType, &pEnumCat, 0); if (hr == S_OK) { // Enumerate the monikers. IMoniker *pMoniker = NULL; ULONG cFetched; while (pEnumCat->Next(1, &pMoniker, &cFetched) == S_OK) { // Bind the first moniker to an object. IPropertyBag *pPropBag; hr = pMoniker->BindToStorage(0, 0, IID_IPropertyBag, (void **)&pPropBag); if (SUCCEEDED(hr)) { // To retrieve the filter's friendly name, do the following: VARIANT varName; VariantInit(&varName); hr = pPropBag->Read(L"FriendlyName", &varName, 0); if (SUCCEEDED(hr)) { wprintf(L"Effect: %s\n", varName.bstrVal); } VariantClear(&varName); pPropBag->Release(); } pMoniker->Release(); } pEnumCat->Release(); } pSysDevEnum->Release(); return hr; } // A very simple program to list DES effects using DirectShow. // int main(int argc, char* argv[]) { // Initialize the COM library. HRESULT hr = CoInitialize(NULL); if (FAILED(hr)) { // We'll send our error messages to the console. printf("ERROR - Could not initialize COM library"); return hr; } // OK, so now we want to build the filter graph // using an AudioCapture filter. // But there are several to choose from, // so we need to enumerate them, then pick one. hr = EnumerateEffects(CLSID_VideoEffects1Category); hr = EnumerateEffects(CLSID_VideoEffects2Category); CoUninitialize(); return 0; }

This program bears a resemblance to DSAudioCap, which it s based on. The basic function is very straightforward: after COM is initialized, two calls are made to the EnumerateEffects function. Each time the function is called, a different CLSID is passed, indicating which set of effects should be enumerated. CLSID_VideoEffects1Category enumerates all the effects that can be applied to a single source clip, while CLSID_VideoEffects2Category enumerates all the transitions that can be used when going from one source clip to another.

EnumerateEffects is a direct adaptation of the DSAudioRender function Enumerate AudioCaptureFilters; it uses the system device enumerator to create a list of monikers corresponding to the CLSID. Then that list is walked through, and the FriendlyName for each entry is printed to the console. If your system is configured in a manner resembling my own, here s what you should see on that list (shown in two columns to save space):

Effect: Fade	Effect: Iris
Effect: BasicImage	Effect: RadialWipe
Effect: Chroma	Effect: Fade
Effect: Matrix	Effect: ZigZag
Effect: Pixelate	Effect: RandomBars
Effect: ICMFilter	Effect: Spiral
Effect: Scale	Effect: Pixelate
Effect: Blur	Effect: Wheel
Effect: Glow	Effect: Inset
Effect: Alpha	Effect: Compositor
Effect: DropShadow	Effect: Blinds
Effect: Wave	Effect: Wipe
Effect: Additive Surface	Effect: CheckerBoard
Effect: Shadow	Effect: GradientWipe
Effect: Emboss	Effect: Slide
Effect: Engrave	Effect: Barn
Effect: Light	Effect: Stretch
	Effect: RandomDissolve

The entries up to Light on this list correspond to the effects available to DES, while the next entries, beginning with Iris, are the transitions DES can use. (Although you might think that we could add two more calls to Enumerate Effects with CLSIDs of CLSID_AudioEffects1Category and CLSID_Audio Effects2Category, these calls return empty lists. Audio effects have to be addressed individually by their CLSIDs.) Now that we know the effects and transitions available to use, we can put them to work in a simple DirectShow application that uses DES to create an automated editing program.