Color Matching

You've examined the tools; now it's time to move on to the bread and butter of compositing: matching foreground and background images.

Color Finesse

The Color Finesse tool set is elegant and full featured, but I'm considering it out of scope for this book for a couple of reasons.

First, its core features are based on the same principles as the tools I've outlined: histograms, such as those in Levels, Curves, and Hue/Saturation adjustments (see Figure 5.21). You have no business working in Color Finesse, therefore, if you don't understand how to get a similar result from the core tools.

Figure 5.21. Do the controls in Color Finesse seem familiar? They certainly should. Although snazzier looking, they are fundamentally based on the tools covered in depth thus far in the chapter.

Second, I don't like the fact that Color Finesse runs in its own user interface; for example, you have to open a second application to do color correction. I find this cumbersome for use in the effects pipeline and even less stable than sticking with the core tools.

For some situations, however, you will definitely want to check out Color Finesse on your own, and look through its 60-page manual. I love the fact that it lets you isolate secondary color ranges and adjust only them. When I need a quick color fix, this method is much faster than taking the trouble to create a mask for a layer so I can adjust just some subset of the full image. I also think that it could be the tool you want if your only job is conforming the color of an entire edited piece, a job that often goes to dedicated colorists and Inferno operators.

Just make sure you've mastered the core tools first.

Melding images together and eliminating all clues that they came from separate sources is as much science as art. Up to this point we've focused on optimizing source footage, which is a more relative and artistic practice. Once the background has been properly graded, however, matching the foreground using the same tools is much more straightforward. The process obeys such strict rules that you can do it with no artist's eye at all. Assuming the background has already been color-graded, you even can satisfactorily complete a shot on a monitor that is nowhere near correctly calibrated.

How is that possible?

As with so many things in visual effects work, the answer is really a question of correctly breaking down the problem. In this case, the job of matching one image to another obeys rules that can be observed channel by channel, independent of the final, full-color result.

Of course, effective compositing is not simply a question of making colors match; in many cases that is only the first step. You must also obey rules you will understand from having done the kind of careful observing of nature described in the previous chapter. And even if your colors are correctly matched, if you haven't interpreted your edges properly (Chapter 3) or pulled a good matte (Chapter 6), or if such essential elements as lighting (Chapter 12), the camera view (Chapter 9), or motion (Chapter 8) are mismatched, your composite will not succeed.

These same basic techniques will work for other situations in which your job is to match footage preciselyfor example, color correcting a sequence to match a hero shot, a process also known as color timing.

The Fundamental Technique

Integrating a foreground element into the color space of a background scene breaks down into three steps:

1.	Match brightness and contrast as if the images were grayscale, using Levels. When matching the black and white points, pay attention to atmospheric conditions. Matching the midtones, however, may not always be necessary or even possible.
2.	Using Levels, match the brightness and contrast of individual color channels as needed.
3.	Note any fundamental mismatch problems that remain and ascertain whether they are related to color matching or some other issue, such as the edges, matte, lighting, camera view, and so on.

This approach, although not complicated or even particularly sexy, can take you to places your naked eye doesn't readily want you to go when looking at color. Yet, when you see the results, you realize that nature beats logic every time.

Start with a simple example: inserting a 3D element lit for daylight into a daylight scene. As you can see in Figure 5.22, the two elements are close enough in color range that a lazy or hurried compositor might be tempted to move on without adjusting the foreground.

Figure 5.22. An unadjusted foreground is slapped over the background.

With only a few minutes of effort, you can do a world better. Make sure the Info palette is somewhere that you can see it, and for now, choose Percent (0-100) in that palette's wing menu to have your values line up with the ones discussed here. You are free to choose any of the color modes in this menu; I advocate this one only for the purpose of standardizing the discussion.

This particular scene is a good beginner-level example of this technique because it is full of elements that would be monochromatic under white light. The background is dominated by concrete, which is generally flat, colorless gray, and the foreground element is an aircraft with a silver body that is lit from the top with white light with dark shadows underneath.

Begin by looking for suitable black and white points to use as references in the background and foreground. In this case, the shadow areas under the archways in the background, and underneath the wing of the foreground plane, are just what's needed for black pointsthey are not the very darkest elements in the scene, but they contain a similar mixture of reflected light and shadow cast onto similar surfaces, and you can expect them to fairly nearly match. For highlights, you happily have the top of the bus shelter to use for a background white point, and the top silver areas of the plane's tail in the foreground are lit brightly enough to contain pure white pixels at this point.

Figure 5.23 shows the targeted shadow and highlight regions and their corresponding readings in the Info palette. The shadow levels in the foreground are lower (darker) than those in the background, while the background shadows have slightly more red in them, giving the background a warmth that the unadjusted foreground lacks. The top of the plane and the top of the bus shelter both contain levels at 100%, or pure white; the blue readings on the bus shelter, which are a few percentage points lower, give it a more yellow appearance.

Figure 5.23. The target highlight and shadow areas are outlined in yellow; levels corresponding to each highlight are displayed in the adjacent Info palette.

To correct for these mismatches, apply Levels to the foreground and move the Output Black slider up to about 7.5%. This raises the level of the blackest black in the image, lowering the contrast, something we didn't expect to do when optimizing images earlier in the chapter.

Having aligned the contrast levels, it's time to correct for the differences in color. Remember the red levels in the background shadows are higher than blue or green, which is a clue that you should now switch the Composition window to the red channel (click on the red marker at the bottom of the window or use the Alt+1/Option+1 shortcut). A thin red line around the outside of the display reminds you that you are looking only at red levels, and you can zoom in to an area that shows both of the regions you're comparing (Figure 5.24).

Figure 5.24. Match the red channel only.

Now you can see clearly that the black levels in the red channel are still too low in the foreground, so raise them to match. Switch the Channel pop-up in Levels to Red, and raise Red Output Black slightly to about 3.5%. You can move your cursor from foreground to background and look at the Info palette to check whether you have it right, but the great thing about this method is that your eye usually gets variations in luminance correct when looking at a grayscale image.

Now for the whites. Because the background highlights have slightly less blue in them, switch to the blue channel (clicking the blue marker at the bottom of the Composition window or use Alt+3/Option+3). Pull back slightly to where you can see the top of the bus shelter and the back of the plane. Switching Levels to the blue channel, lower the Blue Output White setting a few percentage points to match the lower blue reading in the background. Back in RGB mode (Alt+3/Option+3 toggles back from blue to RGB), the highlights on the plane take on a more sunlit, yellow quality. It's subtle, but it seems right.

The human eye is more sensitive to green than red and blue. Often, when you look at a shot channel by channel, you will see the strongest brightness and contrast in the green channel. For that reason, a sensible approach to matching color may be to get the overall match in the ballpark so that the green channels match perfectly, and then adjust the other two channels to make green work. That way, you run less risk of misadjusting the overall brightness and contrast of your footage.

What about the midtones? In this case, they're taking care of themselves because both the foreground and background are reasonably well balanced and your corrections are mild.

Figure 5.25 shows the result, with the same regions targeted previously, but with the levels corrected. To add an extra bit of realism, I also turned on motion blur, without yet bothering to precisely match it (something you will learn more about in Chapter 8, "Effective Motion Tracking"). You see that the plane is now more acceptably part of the scene. Work on this composite isn't done either; besides matching the blur, you could add some sun glints on the plane as it passes, similar to those on the taxi. On the other hand, you can tell that you don't need to bother adding a pilot to the cockpit; the blur on the plane is too much to even notice that the pilot is missing.

Figure 5.25. This is a better match; motion blur helps sell the color adjustment as well.

All kinds of studies show how taking a break once in a while helps your concentration and lets you stay healthy by stretching your body, but in this case there is a more specific use for turning away from your monitor for a minute or two. When you come back, even to a labored shot, you regain an immediate impression that can save you a lot of noodling. The process outlined in this section is designed to remove many variables to color correction, but it's still a question of whether the final shot looks right or not.

Depth Cueing

The first example was fairly elementary with lots of good reference in the background, corresponding levels in the foreground, and no variables in the environmentno particulate matter in the air, for example. Now let's complicate matters.

One of the most common phenomena noticeable in the world at large is what happens to objects at a great distance (provided they are here on our planet, anyhow). Particulate matter in the airnot just pollution and fog, but even normal water condensation in otherwise pristine climatescauses objects in the distance to lose contrast and to take on some of the atmosphere color.

Figure 5.26 shows this phenomenon in full effect. Examine the airplane landing in the background, and the land around it. The darkest black levels are a medium gray, while the rooftops visible in the foreground at the bottom of the frame are as dark as any of the shadow areas in the previous shot.

Figure 5.26. The unadjusted background plate is not pretty, but it is hazy and realistic.

Now say you want to insert that same airplane into this shotnot as far away as the plane landing in the distance, but not as close as those rooftops. Drop the plane in and scale it down to about 20%, and you'll see that in spite of its size, the depth cueingthe indications to your eye of its scaleare all wrong because of its contrast levels (Figure 5.27).

Figure 5.27. At this point, we appear to have a toy airplane from another shot.

You want the black levels somewhere between those of the plane in the distance and the rooftops in the foreground. You have foreground black level reference (the rooftops) but no white levels. From the black levels in the foreground, you can see that there is not much of a color shift occurring in the shot overall; the blacks are fairly monochromatic, with RGB values within a percentage or two of one another, all around 20%.

In the background, however, everything starts turning cyan. Black and white levels in the background have nearly 10% less red than green or blue.

When Reference Is Missing Something

There will be times when your background is missing the clues that simplify this work. In particular, your background may not have anything in it that would be pure white under normal lighting conditions. In extreme cases, you may have to guess what the color of an object in your background would be if you took a close-up shot of it under normal lighting conditions. Things may also be happening in your background, such as a cloud passing overhead, that should not be occurring in the foreground.

In such cases, you have to do your best knowing the basic rules about how light behaves (outlined here and in Section III).

There is one situation where you will have to use scientific fact instead of reference, unless your background was shot from a spaceship. In outer space, outside of the Earth's atmosphere, there is very little particulate matter (other than the occasional cloud of flying debris) and no atmosphere, so items in the distance tend to retain their full contrast.

On other planets, however, the same rules as Earth's will apply, assuming the planet in question has some kind of atmosphere.

To place the plane in the mid-ground, you're going to take the levels toward, but not quite as far as, those of the background. First, working in RGB, raise Output Black about 30% and lower Output White around 85% to bring the blacks above 40% and the whites into the 80% range. The Info palette reveals that the corresponding levels in the far background are in the mid-50% range for blacks and the mid-70% area for highlights, so you've approached those levels without going quite that far.

Now if you toggle among the red, green, and blue channels in the Composition window, you'll notice that the red channel is slightly darker overall than the other two, accounting for the overall cyan quality of the image. Here is a clear case for the use of gammaspecifically, lowering the Red Gamma value to about 0.85, darkening the red midtones and taking the overall layer more toward cyan.

Just for fun, a second plane is added in the distance, as if it was chasing the plane that is landing, its levels matched to the source plane exactly. Figure 5.28 shows both planes added to the background footage.

Figure 5.28. Here distance cueing has been corrected.

Gamma Slamming

Maybe you've seen an old movie on televisionmy favorite example is Return of the Jedi (before the re-release)in which you see black levels that you really shouldn't be seeing. Jedi was made prior to the digital age, and some of the optical composites worked fine on film, but when they went to video, subtleties in the black levels that weren't previously evident suddenly became glaringly obvious. For example, some ugly garbage mattes around the inside of the Emperor's hood looked black on film, but look astonishingly bad on video.

Don't let this happen to you! Now that you know how to match levels, put them to the test by slamming the gamma of the image. To do this, you need to make a couple of adjustment layers. I usually call one slam up and the other slam down, as in the examples. Be sure that both of these are guide layers so that they have no possibility of showing up in your final render.

To slam up, apply Curves with the gamma raised significantly (Figure 5.29). This exposes any areas of the image that might have been too dark to distinguish on your monitor; if the blacks still match with the gamma slammed up, you're in good shape.

Figure 5.29. Slamming gamma is like shining a bright light on your scene. Your black and midtone levels should still match when seen this way.

Similarly, and somewhat less crucial, you can slam down by lowering the gamma and bringing the highlights more into the midrange (Figure 5.30). All you're doing with these slams is stretching values that may be difficult for you to distinguish into a range that is easy for you to see.

Figure 5.30. If in doubt about the highlights in your footage, you can also slam the gamma downward.

This method is useful anywhere that there is a danger of subtle discrepancies of contrast; you can use it to examine a color key, as you'll learn in the next chapter, or a more extreme change of scene lighting.

Extra Credit

This chapter has covered some of the basics for adjusting and matching footage. Obviously there are exceptional situations, some of which occur all of the time: changes in lighting during the shot, backlighting, interactive light and shadow. There are even cases in which you can, to some degree, relight a shot in After Effects, introducing light direction, exchanging day for night, and so on. These topics and more are covered in depth in Chapter 12.