Sky Replacement

Sky replacement is one of the cheapest and easiest improvements you can make to a shot. It opens up all kinds of possibilities for shooting faster and more cheaply: You don't have to wait for particular conditions, and you can swap in not only a different sky but also a skyline from a different location.

The sky, after all, is usually part of the story, often a subliminal one but occasionally a starring element. In Vanilla Sky, for example, the surreal-looking sky was the first clue we got that maybe Tom Cruise's character was not in the real world. An interior with a window could be anywhere, but show a Manhattan skyline outside the window and New Yorkers will try to place the exact neighborhood and city block of that location, along with the time of day, time of year, weather, outside temperature, and so on, possibly without ever really paying conscious attention to it.

Why spend extra time and money to get the exact distant background conditions you want, if you can get the shot cheaper and more quickly by taking it elsewhere? Why spend tens or hundreds of thousands of dollars for that view apartment on Central Park East and try to get the whole scene done at golden hour (the beautiful time of the sunset, an "hour" that typically lasts about 20 minutes and doesn't show up on an overcast day)? Instead, you could take it in your friend's apartment, shoot all day, and add the sunset view in post. True, sometimes there is no replacement for a real environment. Days of Heaven, for example, is as close to being a film about golden hour as you're ever likely to see, and it took a couple of years to shoot. Often, the real story is elsewhere, and the sky is a subliminal (even if beautiful) backdrop that must serve that story (Figures 13.6a through d).

Figure 13.6a through d. For an independent film with no budget set in San Francisco, the director had the clever idea of shooting it in a building lobby across the bay in lower-rent Oakland (13.6a), pulling a matte from the blue sky (13.6b), and match moving a still shot of the San Francisco skyline (from street level, 13.6c) for a result that anyone familiar with that infamous pyramid-shaped building would assume was taken in downtown San Francisco. (Images courtesy of The Orphanage.)

The Sky Is Not (Quite) a Blue Screen

Only on the clearest, bluest days does the sky approach any kind of ideal as a keyable background. The color tends to wash out toward the horizon, cloudless skies are not always so easy to come by, and even clear blue skies are not as saturated with color as they might sometimes seem.

Still, some combination of a color keyer, such as Keylight, and a hi-con luminance matte pass or a garbage matte, as needed, can key out the existing infinite depth in your shot. Chapter 6, "Color Keying," focuses on strategies for employing these, and Chapter 7, "Rotoscoping and Paint," describes the alternative, rotoscoping, when keys and garbage mattes aren't doing the trick.

In any case, the first step of sky replacement is making the existing "sky" (which may include other items at infinite distance, such as buildings and clouds) transparent by developing a matte for it. As you do this, place the replacement sky in the background; a sky matte typically does not have to be as exacting as a blue-screen key because the replacement sky often bears a resemblance to the source (Figure 13.7).

Figure 13.7. This rather poor blue-screen matte is acceptable for this type of shot because the color range and contrast of the target background are not so different from the source. The holes in the matte are the result of reflected sky color in the highlights of the foreground; this looks acceptable in the final shot.

Infinite Depth

If your shot is locked off, your job is already complete with the creation of the matte. If, however, there is camera movement in the shot, you might think that you need a 3D track to properly composite in a new sky element.

Typically, that's overkill. Instead, you should consider these options:

• If you are matching motion from the original shot and anything in the source sky can be tracked, by all means track the source.

• If only your foreground is trackable and you're following the suggestions in Chapter 8, "Effective Motion Tracking," for applying a track to a 3D camera, move the replacement sky to the far background. (This may require a Z Position value in the four or five digits, depending on camera settings.) Scale it up to compensate for the distance. In this case you must essentially eyeball the depth.

• If the shot contains a push or zoom (Chapter 9, "Virtual Cinematography," describes the difference), these may be more easily re-created using a 3D camera (although you should look at Chapter 8 for tips on getting away with a 2D track). If you're adding them from scratch, place the sky in the far background as with the last tip and add your camera animation. If the push or zoom is in your source shot, you may need camera data from third-party 3D tracking software (see Chapter 8).

The basic phenomenon you're trying to re-create is that scenery at infinite distance moves less than objects in the foreground, the basic parallax effect. This effect is less pronounced with a long, telephoto lens, and more pronounced in a wide angle. For the match in Figure 13.6, a still shot (no perspective) was skewed to match the correct angle and tracked in 2D; the lens angle was long enough and the shot brief enough that they got away with it.