Optimizing Tracking via 3D

So you get the idea how to set and apply a tracker. Is that all there is to know about tracking in After Effects? Not by a long shotyou've only laid the groundwork for the good stuff. Say you want to track an arbitrary number of overlaid elements into a scene, matching the movement of a camera. You may think that you would have to track each layer individually.

Wrong. There is a way to set up tracking so that adding in tracked elements, including motion blur, rotation, and even scaling of elements, requires no extra steps. You drop them into the scene as easily as you might have added them when the shot was taken, and they look just as natural. That's right, you can come very close to the look of 3D tracking on many shots without a 3D tracker.

The key is to stabilize the background layer (it will not appear stabilized in the final shot, although it could, or you can smooth out a rough camera motion) and then parent a camera to that stabilization. You now have the motion captured in the movement of a camera, just as it was when shot, so that any elements you add to the scene just pick up on that motion. It's really cool.

The 3D Camera: The One-Stop Solution

Suppose you need to add an arbitrary number (more than one, maybe more as you go along) of foreground layers to a motion-tracked background plate: CG objects, color corrections, effects with hold-out masks, you name it. Applying track data to each of those layers individually would be a time-consuming headache.

Skip it. Don't apply your tracker at all. Instead, create whatever type of track your shot needs: Position only or with the addition of Rotation or even Scale. Now try this:

1.	In the Tracker Controls, change the Track Type from the Transform (the default) to Stabilize. (This method is not meant for corner pinning.)
2.	Apply Stabilize to the track that generated it. You actually have no other choice; try choosing a different layer than the source track layer by clicking Edit Target, and you'll find the other layers grayed out.
3.	Your layer has now been stabilized and will probably slip and slide around in the frame to maintain stability (Figure 8.20). No, this is not what you're after, but don't worry. It's only one step from looking normal again. Figure 8.20. By the end of the shot, the motion stabilizer has managed to keep the world's most obnoxious stretch Humvee perfectly in place, but there's that gutter problem, and anyhow, you wanted that bit of camera shake and wobble associated with a handheld shot here. What to do?
4.	Return to the first frame of the track (quite possibly frame 0 of your comp). Turn on its 3D switch. Add a 3D camera (context-click in an empty area of the Timeline window) and parent it to the tracked background layer (Figure 8.21). Figure 8.21. All is now looking normal again at the end of the shot with the addition of a 3D camera, parented to the stabilized background layer, which had already been switched to 3D so that the camera would pick up its motion. If you watch this shot, you cannot see that anything has changed with it.

Everything now seems back to normal, but any new item that you add to the scene joins the motion track as soon as you switch it to be a 3D layer. Any layer that you don't want to join the track, such as an adjustment layer, can remain 2D (Figure 8.22).

Figure 8.22. It's very hard to show how impressive the result is in a still figure, but believe me, those orange markers now hold their position on that roadway throughout the shot. A look at the perspective view (inset) shows that all layers are essentially 2D, but only the camera and background actually animate.

The Z-depth value of any layers you add will typically remain at 0, so the pylons in Figure 8.22a were scaled and repositioned rather than moved in 3D space. This is not because it wouldn't work to reposition them in 3D space, but simply because it's more work to get the amount of Z depth correct so that the object actually sits where it is supposed to; you have no frame of reference for where it should be in Z space (Figure 8.23).

Figure 8.23. Instead of faking perspective in 2D, you have the option of actually repositioning layers in 3D space, which yields a shot that looks a lot like 8.22a. The only problem is that now you have to get the View Angle and relative distance of the camera and objects just right or they will start to slip and slide in the shot. It's typically not worth the trouble.

Relocating added layers in Z space might be worth the trouble in extreme cases, such as when you're adding in something that is nowhere near co-planar with the shot. On The Day After Tomorrow, The Orphanage had several shots of a helicopter pilot inside a moving cockpit, with a blue screen outside the window that was replaced with a distant horizon. By moving it back several thousand pixels on the Z axis to approximate the relative depth, we were able to get parallax for free. Of course, we didn't have to worry about anything looking like it was sitting right on the ground plane, as in the Humvee example.

When considering the camera lens setting, keep in mind that a 50 mm camera lens introduces no lens distortion to your scene; a set of 3D layers with a Z Position of 0 will look the same as they would as 2D layers. Generally, that's a good thing. It is conceivable, of course, that your scene was shot with a wider (or longer) lens, and that there is some lens distortion in your scene. If this distortion is substantial, and you're relocating items in Z depth, you may want to try to match it. See Chapter 9, "Virtual Cinematography" for details.

Fake 3D Tracking

As I was writing, I decided to see what would happen if I tried to use the methods I described in the previous section to approximate an actual 3D camera on a shot with a zoom-intracking Position, Rotation, and Scale. What would happen, I wondered? Would it work, or would the fabric of space and time itself be rent asunder and the universe as we know it come to a screeching halt?

Happily, the result was largely the former. Figure 8.24 shows my tracked shot in which the camera zooms in on the infamous Hummer. If I switch the Track Type from Transform to Stabilize (I seem to always forget to do that before I start, and luckily the After Effects tracker has improved to the point where you can change your mind), I can apply this to the source plate layer and get a resulting end frame like the one in Figure 8.25. Not good. But again, it's just an interim step on the way to tracking glory.

Figure 8.24. It's not hard to get a good track of the zoom in this handheld shot. Most of the movement is straight in (Scale), but there is also some wobble (Position and Rotation).

Figure 8.25. Obviously, stabilizing a zoom shot results in a tiny frame, albeit one that is absolutely identical to the first frame of the shot, if it were cropped like this.

The drill from here is just as before: Add a 3D camera, parent it to this plate layer, then enable the plate layer's 3D toggle. The scaled-down shot pops back to 100% size (make sure the camera covers the whole shot if you set it on the end frame). As before, the magic begins when you drop objects into the scene, but in this case, you definitely want to enable their 3D toggles as well (if there's more than one of them) to get the effect of parallax as you zoom in (Figure 8.26).

Figure 8.26. The result at the beginning, middle, and end of the shot. The pylons are fairly well locked to the pavement, but the quality of the result mostly comes down to how good the track was in the first place, because any slight bump ripples out to all of the objects.

Now before you get too excited, let me remind you that this works only in certain cases and that true 3D trackers generate true 3D data (including understanding the difference between a camera zoom and a push, which this one decidedly does not). Still, if you pull off a track that looks this three-dimensional using only the tools supplied with After Effects, you will amaze your friends, indeed (at least the ones nerdy enough to know the difference).

Smoothing a Moving Camera

But wait, there's more.

What about stabilizing a shot in which the camera is meant to be moving? There's no track type for this in the tracker controls and simply choosing Stabilize (the closest option) locks the shot in place. This will not do with a pan, tilt, or other camera move (including a handheld shot), and there's that problem with it introducing gutters around the image. Is there any way to reduce some of the jitter or rotation in a shot without removing the camera motion altogether?

Of course! Now that you've re-created the physical camera as an After Effects camera, you apply your smooth to the data that animates that camera, and barring extreme jolts that cause enough motion blur to blow the gag, it just works.

Figure 8.27 features a tilt-down shot taken with an old Arriflex camera that seems to have had a rather serious pin registration problem, causing lots of unwanted gate chatter. The process would be the same if the shakiness was the result of a handheld camera move or a stiff tripod head. As you see from the timeline in the figure, the now familiar steps are in place. There is only one problem: The camera has no data to smooth; its motion is derived only from being parented to the stabilized layer, and you don't want to apply a smooth to the source layer. The smoothing effect will come from the difference between that layer's motion and that of the camera.

Figure 8.27. It's difficult to show what a pin registration problem on a film camera looks like in a printed figure, but look at the motion on this simple tilt-down from the sky. In particular, notice the jumps in attach point animation. You can tell something is up. The motion is stuttery and jerky, although the pan was smooth.

Therefore, duplicate the tracked layer and turn off its visibility. Rename it "stabilizer," or something like that (I called mine, "Ken Stabler," recalling my early childhood as a Raiders fan), and just to be on the safe side, make it a guide or adjustment layer as well, so that there's no possibility of it rendering should its visibility get checked back on. Figure 8.27 shows this preparation on a sample shot.

There are a couple of ways to apply smoothing to the motion data of the camera:

Use of The Smoother (Figure 8.28)
Figure 8.28. I don't really like The Smoother, a relic from the pre-expressions days of After Effects. You get only one shot using a single Tolerance adjustment, and thereafter your original keyframes are destroyed, replaced by a hodgepodge of new ones. The good news is that the result is smoother than the original.
Use a smooth expression (Figure 8.29)
Figure 8.29. Ah, that's better. The source and result motion can be easily compared and adjusted after the fact; you just have to be comfortable changing those little numbers in between the parentheses (known as arguments). If you're not, read on, and check out Chapter 10 for good measure.

Either way, once you've smoothed the duplicated camera motion, don't forget to enable the 3D toggle for the source plate layer.

My advice is to take the second of the two smoothing options, and to help get you on board with that decision, I'm going to show exactly how it's done, a few chapters ahead of actually detailing how to create expressions. The main problem with The Smoother is that it's a one shot deal: Once you've applied it, you can undo it, but if later on you decide you need to refine itgood luck. The Smoother actually changes your keyframe data. Furthermore, it offers only one adjustable setting, for Tolerance; it's up to you to guess how that setting will influence your track.

Expressions are completely nondestructive and thus much more in the true spirit of After Effects.

To apply a smooth expression

Alt/Option-click on the stopwatch for the null to which the camera is parented.

With the default expression (anchorPoint) still highlighted, go to the Expressions menu (Figure 8.30) and under Property choose smooth(width = .2, samples = 5, t = time).

Figure 8.30. No need to typenot much, anyhowbecause the expression you need is waiting for you in this menu, and the default arguments are fine in this case.

In many cases, this default setting will do the trick. If it doesn't, try changing the width value from the default of 0.2, which is the width of time sampled (raising it will use a larger sampled area but take longer), or increase samples from 5 (increase the number of samples within the specified time) for more smoothing. The final argument, t = time, is not applicable unless you need to offset the smoothing function in time for some reason. If the idea of expressions and arguments seems like gobbledygook to you, take a look at Chapter 10.

The same basic trick will work to destabilize, rather than smooth, a camera motion. Because this expression doesn't simply work with the default parameters, I've left discussion of it to Chapter 10.

What about rotation? If your shot has unwanted rotation, and you've managed to track it, you may wish to eliminate it altogether by setting a static rotation value to the camera null or to use a high smoothing value on it.

And what about compositing other elements into this now smoother scene? You can do it, but you have to pre-compose all of this and retrack the result, as if the smoothed shot was the one you started with. Next, follow the instructions on compositing objects into a tracked shot. Use the same track point that worked for you before with the same settings, and it should be a piece of cakehonest!