Compositing: Science and Nature

What exactly is happening in a simple A over B composite? You're just laying one image over another, right? Nothing to itwas obvious from the first time you used Photoshop, right?

For most people, the basic compositing operation is a completely intuitive process that is rarely questioned, but to deal with the crucial part of an A over B compositethe edge detail of layer Ait helps to know what is going on, not only in the world of software, but in the real world as well. The two worlds do not operate the same, but it is the job of the digital world to re-create, as faithfully as possible, what is happening in the natural world.

Bitmap Alpha

A bitmap selection channel is one in which each pixel is either fully opaque or fully transparent. This is the type of selection you get if you use, say, the Magic Wand tool in Photoshop. You can feature or blur the resulting edge, but the default selection has no semi-transparent pixels.

This type of selection has its placesay, isolating pixels of one particular color range to change thembut it is not how nature works. A bitmap cannot describe a curve or angle smoothly, and even a straight line with no edge transparency looks unnatural in an image whose goal is realism (Figure 3.5).

Figure 3.5. Using 400% magnification, you can see clearly how ugly a curved or angled shape is if it can be described by only bitmap pixels, pixels that are either fully off or fully on.

Feathered Alpha

But although it's easy enough to see that a bitmap edge does not occur in nature, it's hard to imagine that a feathered alpha does occurhard objects don't have semi-transparent edges, do they? Of course not. Look at the edge of this page, or anything in the foreground of your field of vision. Do you see semi-transparent edge areas? No.

Not exactly, anyhow. It so happens that semi-transparent edge pixels are the best approximation we have digitally for nature (Figure 3.6), because they solve two problems in translating the world of objects to the world of pixels:

• They come closer to describing organic curves.

• They approximate the physics of light as it bends around objects.

Figure 3.6. Ahh, that's better. With a proper alpha channel and a one-pixel feather adjustment, even at 400%, the edge does a much better job of describing a smooth, soft curve.

Say what? The first point is easy enough to see and appreciate, but the second one is a doozy, an observation that can be traced back a century to Einstein's annus mirabilus, in which he demonstrated how mass and light are interrelated. Light is bentslightlyby any amount of mass that it passes. The greater the mass, the higher the amount of bending, right up to the extreme of black holes from which light cannot escape. With most objects, your eye sees a slight, subtle effect.

Geek Alert: The Compositing Formula

When you layer a raster image with semi-transparent alpha over an opaque background image, here's what happens: The foreground pixel values are multiplied by the percentage of transparency, which, if not fully opaque, reduces their value. The background pixels are multiplied by the percentage of opacity (the inverse of opacity), and the two values are added together to produce the composite. Expressed as a formula, it looks like

(Fg * A) + ((1-A)*Bg) = Comp

With real RGB pixel data of R: 185, G: 144, B: 207 in the foreground and R: 80, G: 94, B: 47 in the background, calculating one edge pixel only might look like

[(185, 144, 207) x .6] + [.4 x (80, 94, 47)] = (143, 124, 143)

The result is a weighted blend between the brightness of the foreground and the darker background.

Other effects compositing programs, such as Shake, do not take this operation quite as much for granted the way that After Effects and Photoshop do. You can't simply drag one image over another in a layer stack, you must apply an Over function to create this interaction. Is there a difference? Not until you add to the discussion the operations that go along with an Over, in particular premultiplication, which is detailed later in this chapter.

You can study this effect close-up in a digital photo with no compositing whatsoever (Figures 3.7a, b, and c). In the digital image, you can see areas at the edge of objects that become a wash of color combining the foreground and background. This is caused by the influence of the mass of that object itself on the light (color) coming from behind it.

Figure 3.7a, b, and c. Consider a simple digital photo that is not composited (3.7a). When you look closely, you can see that there is a natural softness around the contour of the head despite that it is in sharp focus (3.7b). The effect is even more evident at 400%, if you compare the sharpness (albeit pixilated) of the detail in the fur to the softness of the edge (3.7c). (Images courtesy of Matt Silverman.)

Opacity

But we're not done with the surprises connected to basic image combination; the way that Opacity edits work in After Effects often takes people by surprise as well, although they can work with it for years without coming face to face with what is counterintuitive about it.

Say you have two identical layers, no alpha/transparency information for either layer. Set each layer to 50% Opacity, and the result should look exactly like either one of the layers at 100%, right?

Wrong (Figure 3.8)! If you've ever heard of Zeno's Paradox, that's something like how opacity is calculated in After Effects, but for good reason.

Figure 3.8. Two white solid layers sit one on the other, each at 50%, yet they do not add up to 100% (full) opacity. They add up to 75%, hence the 25% visibility of the checkerboard background.

Zeno was evidently violating the bounds of common sense with his observation. But a lead developer on the After Effects team once described the program's opacity calculations similarly: Imagine you have a light which is 1. You now have half the light showing through (0.5 * 1 = 0.5). Put a 50% transparent object in front of that. Half of the light shows through (0.5 * 0.5 = 0.25), and so on.

Hence, and invoking Zeno's Paradox, After Effects is mimicking how transparency behaves in the real world, which you can try yourself using sheets of vellum or other semi-transparent paper. (This is extra credit, of course!)