Fixing Typical Problems

As you must know by now, because I've seized every opportunity thus far to drill it into your head, the number one solution to most matte problems is to break the image being matted into multiple sections rather than trying to get one matte in one pass. You can divide the matte as follows :

Core and edge matte (hard and soft matte)
Holdout masks for particular areas of the frame (useful if lighting varies greatly within the frame, or if one area contains a particularly challenging element, such as hair or motion blur)
Temporal split (if light conditions change as the shot progresses)

All other tricks fall short if you're not willing to take the trouble to do this. And now for some more bad news.

On Set

No matter how advanced and well paid you may be, your time is likely to be far cheaper than that of a full crew on set. That meansshocking, I knowyou'll have the opportunity to fix things in post that should have been handled differently on set. You could call it job security (but that sounds so cynical ); however you think of it, it's bound to be part of your job.

Close-Up: The Right Color

The digital age lets shooters play fast and loose with what they consider a keyable background. You may be asked (or try on your own) to pull mattes from a blue sky, from a blue swimming pool, or from other monochrome backgrounds.

If you have the opportunity to create your own keyable background, however, there is one brand to look for when purchasing your paint, fabric backdrop, or adhesive tape: Rosco. Rosco's Digicomp products are of a color designed specifically to allow for the most effective keying, with a purity and intensity of saturation unavailable at your local paint store. They are available in three colorsyou guessed itblue, green, and red.

How different must the background color be from the foreground? The answer is "not very." Red is worth avoiding simply because human skin tones contain a lot of it, but a girl in a light blue dress or a soldier in a dress blue uniform often keys just fine from a blue screen (although spill suppression will require special attention).

If you're fortunate enough to be supervising the effects shoot (I recommend it for the craft services alone), you can do all sorts of things to ensure that the footage will key successfully later on.

A hard cyclorama, or cyc (pronounced like "psych"), painted a uniform key color is far preferable to a temp cloth background. If you can't rent a stage that has one, the next best thing might be to invest in a roll of floor covering and paint it, to get the smooth transition from floor to wall, as in Figure 6.30 ( assuming the floor is in shot). Regarding the floor, don't let anyone walk across it in street shoes, which will quickly contaminate it with very visible dust.

Figure 6.30. On a set with no hard cyclorama, you can create the effect of onethe curve where the wall meets the floorusing blue-screen cloth instead. It doesn't behave as well (note the hotspot on the curve), but it will certainly do in a pinch and is much preferable to removing the seam caused by the corner between the wall and floor. (Image courtesy Tim Fink Events and Media.)

Assuming you begin with a correct-colored, footprint-free background with as few seams and other variations as possible, the most important concerns are to light it correctly, balancing the foreground and background lighting.

This job is, of course, best left to a professional, and any kind of recommendations for a physical lighting setup are beyond the scope of this book. But, hey, you're going to spend more time examining this footage than anyone else, so here are a few things you can keep in mind as the on-set effects supervisor:

Ideally, the light levels on the foreground and background should match. A spot light meter tells you if they do.
Diffuse lights are great for the background (often a set of large 1 K, 2 K, or 5 K lights with a silk sock covering them), but fluorescent lights will do in a pinch. With fluorescents you just need more instruments to light the same space. In correspondence about the previous edition of this book, VFX Supervisor Scott Squires noted that Kino Flo lights are a popular option ( Figure 6.31 ).

Figure 6.31. Diffuse white lighting that causes no hotspots in the background is ideal.
It is helpful to keep a good deal of space between the foreground and background. Ten feet is ideal.
Avoid dark unwanted shadows like Indiana Jones avoids snakes , but by all means light for shadows if it looks like you can get some clean ones on the floor. Note that this only works if the terrain in the final shot is as flat as the floor of the studio.
If your scene is set outdoors, definitely consider shooting outside (almost certainly using portable backgrounds such as solid color cloths and carpets). You may require fill lights to avoid hard shadows on the background, but remember that daylight is a difficultalmost impossiblelight quality to re-create on a stage, especially for a shot any larger than a close-up.
Record on the highest-quality, least-compressed component video format you can afford. Even if the rest of your project will be shot on HDV, it might be worth shooting the effects a less compressed format and converting them later if you can.

If the setup permits , bring along a laptop with After Effects on it and with some representation of the scene into which the footage you're taking is going to be keyed. This can be enormously helpful not only to you but to the gaffer and director of photography, to give them an idea of where to focus their efforts. You will have to work with the camera operator to get a live signal (and figure out a PCI transfer setup if you're working with HD), and a film camera may require the addition of a video tap, but it is even possible to test shots on set.

Finally, once the lighting has been finalized and before action is called on the shot, ask the camera operator to shoot a few frames of clean platethe background with no foreground characters or objects to be keyed out later. There are all sorts of ways to make use of this, and it's easy to forget; if you do, try to get it at the end of the setup, or the end of the day.

Notes

Many computer graphics artists will also shoot environment reference of a set, either using a camera capable of taking ultra -wide angle photos (using a circular fisheye lens), or by aiming the camera at a reflective silver ball (known in garden shops as a gazing ball). Not only does this provide reference for the lighting setup, but it can be used to recreate HDRI lighting in 3D software.

Matte Problems

It's an ugly fact of life that there will be times that you have to cheat and manipulate the matte that Keylight (or another method) produces. Make sure you're at least using the right tool for the job, with the lightest possible touch.

For whatever reason, Minimax is sometimes employed by artists who would enjoy greater success with either of the tools in the Matte category: Matte Choker or Simple Choker. Minimax is old school: It works in whole pixel increments to choke and spread pixel values. It has some interesting usesbut not for subtle matte edits. It is a quick way to spread or choke pixel data, particularly without alpha channel information.

In many cases, Simple Choker is the tool to use. It allows you to choke or spread alpha channel data (via a positive or negative number, respectively) at the sub-pixel level (use decimal values). That's all it does. It's really no different from its counterpart in Keylight, but you can use it anywhere .

Tip

A useful third-party alternative to Minimax is Erodilation from ObviousFX (www.obviousfx.com). It can help do heavy choking (eroding) and hole-filling (dilating) in situations where the Simple Choker, well, chokes, and its controls are simple and intuitive (choose Erode or Dilate from the Operation menu and the channeltypically Alpha).

Matte Choker adds tools to introduce extra softness to the choke. Its default settings are an effective starting point for many mattes. Unfortunately, softness controls tend merely to add error to the process and can be the symptom of a bad key. It's also simply not possible to push Matte Choker as far as Simple Choker. Don't be afraid to " settle " for Simple Choker: It's no monkey wrench.

Edge Selection

If you have the Red Giant Software Key Correct Pro plug-ins (or their predecessor, Composite Wizard), Edge Finder can create a grayscale map outlining only the edges of the alpha channel, which can be applied as a luma matte to an adjustment layer. That adjustment layer's effectsblurs, color correction, or a holdoutapply only to the edge.

If you lack these third-party filters, no worries; with a few extra steps you can make your own. The simplest method is to

Apply Shift Channels. Set Take Alpha From to Full On and all three color channels to Alpha.
Apply Find Edges (often mistaken for a useless psychedelic effect because, as with Photoshop, it appears in the Stylize menu). Check the Invert box ( Figure 6.32 ).

Figure 6.32. A typical use for an edge matte such as this is to blur the edge slightly using this selection, in which case you want to blur the background and foreground together via an adjustment layer. These effects settings are on your disc as edgeMatte.ffx. Beware that the bottom edge is included in the matte; don't let this bite you if your frame has no padding (non-visible area at the edge of frame).

Minimax can help choke or spread this edge matte. The default setting under Operation in this effect is Maximum, which spreads the white edge pixels by the amount specified in the Radius setting. Minimum chokes the edge in the same manner. If the result appears a little crude, an additional Fast Blur will soften it ( Figure 6.33 ).

Figure 6.33. Need thicker, softer edges? A quick Minimax set to the default Maximum allows you to specify a Radius by which the white area will grow. If the result looks a little chunky , a quick Fast Blur will soften it back. This is very useful, for example, when there is motion blur on the object that needs to be controlled in the edges.
Apply the result via a luma matte to an adjustment layer. You should not need to precompose before doing so.

Now what? Apply Fast Blur to soften the blend area between the foreground and background, yet another way around a chewy matte. Or add a Levels adjustment to the composited edge to darken or brighten it. A de-saturation operation (using Hue/Saturation) has a similar effect to using a gray edge replacement color in Keylight.

Matte Holes

With a good key, there should be no noise or holes in the foreground; a heavily choked core matte will close them.

However, if the background on the set had fundamental problemsuneven lighting, seams, dirt on the flooryou may discover noise and holes in the matte background. A software solution to close these without affecting edges exists, but only as a third-party tool: Alpha Cleaner, which is also a part of the Key Correct set (formerly Miracle Alpha Cleaner in Composite Wizard).

Because it is an automated solution, it will occasionally fill holes that should remain unfilled, such as the little triangular gap that can open up under outstretched arms ( Figure 6.34 ). Thus matte holes present a situation in which you may have to rotoscope. It's usually not as bad as you think to work this way; the painstaking part of keying is defining the edge. Holes can often be fixed using crude masks or by tracking in paint strokes (each covered in the next two chapters, respectively).

Figure 6.34. An automated solution, such as Alpha Cleaner (part of the Key Correct Pro pack), fills holes in the foreground matte but will also often close holes that you want to keep, such as this tiny gap under the arm. For that reason, veteran effects compositors tend to avoid these solutions, although they present a tempting alternative to rotoscoping .

Matte Fringing

Visible fringing around the edge of a feathered matte is often the result of an alpha having been applied via a track matte. For example, it could occur if, in attempting to process color spill outside of Keylight, you applied the Keylight matte as a track matte.

Don't panic and choke all that detail out of your edge! There is a better solution, but it is hidden away in the Channel menu. Remove Color Matting (sometimes vainly sought as "Unmultiply") is designed specifically to suck color out of edges that are behaving as if they should be premultiplied , not straight (if the distinction eludes you, you are encouraged to review Chapter 3).

Premultiplied images are typically matted against black, leaving a dark fringe around the edges. Remove Color Matting uses black as the default Background Color, and this is the only user -adjustable parameter in this effect. You can instead use the eyedropper to sample a background color. If the fringing is bright colored, premultiplication a white or gray background may have been premultiplied.

Tip

The DVD includes the Unmult plug-in from Red Giant Software. John Knoll developed this plug-in specifically for use with Knoll Light Factory, which generates flares that have no alpha channel. Unmult differs from Remove Color Matting in that it always assumes the background is black, and it adds transparency to pixels based on how much black they contain.

Color Spill

Color spill need not be a big deal. If you're not happy with the spill suppression in Keylight (or another keyer), you can apply the matte as an alpha track matte to the source footage and pursue alternative options.

Sometimes the Spill Suppressor tool that is included with After Effects will do the trick. It uses a simple channel multiplication formula to pull the background color out of the foreground pixels. All you need to do is select a sample from the background color (you can even copy your original choice of screen color from Keylight, but it really only matters whether you choose blue or green) and leave the setting at 100%.

If even Spill Suppressor has undesirable side effects, then it may be advisable to open up Hue/Saturation and target the specific hue that is causing you problems, either desaturating or shifting it.

This may sound positively Neanderthal in its lack of subtlety, but it works a surprising amount of the time, provided you can identify the range of color that needs to be removed. Open Hue/Saturation, and select the channel in Channel Control that needs attentionsay, Blues.

Look closely at Channel Range. You might not have noticed the little controls beneath the upper color spectrum ( Figure 6.35 ). These allow you to set the color range as precisely as you like. The inner hash marks control the range affected by the controls below; the region between those and the outer, triangular markers is the feather area.

Figure 6.35. The little hash marks and triangles under the upper gradient of the Channel Range control specify a region to desaturate ; the amount of desaturation is reflected in the lower gradient. It's quick and dirty but in many cases effective for knocking a particular color out of your scene.

In most cases you'll select a channel, refine the selection, and lower Saturation, alter the Hue, or both, to reduce spill. To understand clearly which pixels are affected, crank Saturation up to 100. The pixels in the affected range are easy to spot; if not, you have chosen a range that is unused by any pixels.