Filter Graphs

Filter Graphs

The DirectShow filter graph organizes a group of filters into a functional unit. When connected, the filters present a path for a stream from source filters, through any transform filters, to renderer filters. However, it isn t enough to connect the filters; the filter graph has to tell the filters when to start their operation, when to stop, and when to pause. In addition, the filters need to be synchronized because they re all dealing with media samples that must be kept in sync. (Imagine the frustration if the audio and video kept going out of sync in a movie.)

For this reason, the filter graph generates a software-based clock that is available to all the filters in the filter graph. This clock is used to maintain synchronization and allows filters to keep their stream data in order as it passes from filter to filter. Available to the programmer, the filter graph clock has increments of 100 nanoseconds. (The accuracy of the clock on your system might be less precise than 100 nanoseconds because accuracy is often determined by the sound card or chip set on your system.)

When the programmer issues one of the three basic DirectShow commands run, stop, or pause the filter graph sends the messages to each filter in the filter graph. Every DirectShow filter must be able to process these messages. For example, sending a run message to a source filter controlling a webcam will initiate a stream of data coming into the filter graph from that filter, while sending a stop command will halt that stream. The pause command behaves superficially like the stop command, but the stream data isn t cleared out like it would be if the filter graph had received a stop command. Instead, the stream is frozen in place until the filter graph receives either a run or stop command. If the run command is issued, filter graph execution continues with the stream data already present in the filter graph when the pause command was issued.

The Life Cycle of a Sample

To gain a more complete understanding of DirectShow, let s follow a sample of video data gathered from a DV camcorder as it passes through the a filter graph on its way to the display. Figure 1-1 illustrates the path.

Figure 1-1. Path of video data gathered from a camcorder

The video sample is generated in a source filter in this case, a capture source filter because the filter is capturing from a live device, a camera, rather than reading from a disk. Video camcorders in the United States, Canada, and Japan (and some other regions and countries) generate 30 frames of video data per second; it s reasonably safe to assume that one DirectShow sample will be equivalent to one frame of video captured by the camcorder. In this case, the source capture filter is a WDM object that speaks directly to the computer s hardware (which, in turn, is communicating directly with the camcorder, probably over a FireWire interface). The capture source filter issues commands to the driver for the camcorder, and in return, the filter receives data. Each packet of data is equivalent to one frame of video.

In this example, the capture source filter s primary role is to convert the packets of data arriving from the camcorder into series of samples. When considered as a whole, these discrete samples compose a stream of video data formatted for DirectShow. Once the camcorder data has been converted into a stream, it can be presented on the source filter s output pin so that it can be passed along to a downstream filter. Additonally, the capture source filter provides a timestamp for each sample in the stream so that DirectShow can maintain sample processing in the correct order; that is, samples captured first will be processed first.

The next filter in the graph is a transform filter, a DV video decoder. This filter takes the raw video data presented by the source capture filter and proc esses it in two significant ways. First it will likely convert the color model of the video stream from the YUV model common in camcorders and other digital imaging devices to the RGB model commonly used on computer displays. Many different color models are used by imaging and display devices, and you ll find that many of the DirectShow transform filters convert from one color model to another to perform their tasks.

Next the DV video decoder might convert the interlaced video fields presented by the camcorder into non-interlaced video images. Using interlacing, a single frame of video imagery is actually broken down into two fields. The first field contains all the odd-numbered lines of video information, while the second field contains all the even-numbered lines. The two fields must be combined that is, de-interlaced to produce the non-interlaced (also known as progressive scan) image used on nearly all computer displays. These two transformations, first of the color model and then de-interlacing, have created a video stream that can now be rendered on the display. After processing, these samples are presented at the output pin of the DV video decoder, which maintains the per-sample timestamp, ensuring that the images stay correctly synchronized as they move from filter to filter.

Finally the stream arrives at its destination, the video renderer. The renderer filter accepts a properly formatted video stream from the DV video decoder and draws it onto the display. As each sample comes into the renderer filter, it is displayed within the DirectShow output window. Samples will be displayed in the correct order, from first to last, because the video renderer filter examines the timestamp of each sample to make sure that the samples are played sequentially. Now that the sample has reached a renderer filter, DirectShow is done with it, and the sample is discarded once it has been drawn on the display. The buffer that the filter allocated to store the sample is returned to a pool of free buffers, ready to receive another sample.

This flow of samples continues until the filter graph stops or pauses its execution. If the filter graph is stopped, all of its samples are discarded; if paused, the samples are held within their respective filters until the filter graph returns to the running state.