Chapter 9: Using the Video Mixing Renderer

Chapter 9

Using the Video Mixing Renderer

Although most of the examples we ve used in this book have a video component, we ve employed only the Video Renderer filter for video output. This filter has proved entirely adequate thus far, but the Video Renderer is designed to render a single stream of video to a single rectangle on the screen. Microsoft DirectShow is capable of dealing with multiple video streams, all flowing through a filter graph in real time, with a range of transform filters capable of producing real-time video effects. With DirectShow and the latest generation of video hardware, we have enormously powerful video processing capabilities. DirectShow provides a video rendering filter to match those capabilities.

The Video Mixing Renderer (VMR) filter was first introduced with Microsoft Windows XP; today, as part of Microsoft DirectX 9, its capabilities are available across the entire range of Microsoft operating systems. In this chapter, we ll explore the capabilities of VMR 9, the Video Mixing Renderer released as part of DirectX 9. (VMR 7, which was part of the Windows XP release, is still available, but it will run only on Windows XP based systems.) The VMR filter does exactly what the name says: it takes multiple video sources and allows you to mix them together in real time to produce spectacular results. The video mixer isn t a recent idea; hardware-based video mixers have been around for many years, costing anywhere from a few thousand to a few hundred thousand dollars. DirectShow offers a software-only solution for video mixing, which means you can have many of the same features of a TV studio right in your own computer, with the appropriate DirectShow application.

The basic function of the VMR is simple: it takes one or several video streams and mixes them into a single video stream, which is then rendered to the display. The VMR presents a series of interfaces to give the application programmer a very fine level of control over the display of each stream coming into the VMR. Not only can you control the mix of each video stream by controlling the transparency, you can make streams fade in and out of each other but you can also control the location of a stream in the output.

A video stream doesn t need to be spread across the entire output window; it could occupy a small corner, a thumbnail view that might be reminiscent of the picture-in-picture feature offered on high-end television sets. You can also use the VMR to mix static images into the video stream and produce an overlay of the video image, or you can take the VMR output and send it to a 3D model (rendered using Microsoft Direct3D, another component of DirectX) and paint the surface of a 3D model with a video image. With another VMR feature, reminiscent of the analog video, you can interactively change the hue, saturation, brightness, and contrast values of all video streams on a per-stream basis.

The VMR is very efficient, within the limits of the video card and processor you re using. That said, the VMR provides so much power that a DirectShow application mixing and blending a multitude of simultaneous video streams could harness nearly 100 percent of the CPU on all but the fastest computers. The features available to the VMR programmer are so comprehensive that we can do little more than cover its basic features and operations in this book. By itself, the VMR possesses more power and capability than any other single DirectShow filter. After you ve read this chapter and studied the example code, you might be interested in reading the extensive documentation on the VMR in the DirectX documentation for C++, which covers the design and operation of the VMR in greater detail.