Hack21.Objects Move, Lighting Shouldn t

Hack 21. Objects Move, Lighting Shouldn't

Moving shadows make us see moving objects rather than assume moving light sources.

Shadows get processed early when trying to make sense of objects, and they're one of the first things our visual system uses when trying to work out shape. [Hack #20] further showed that our visual system makes the hardwired assumption that light comes from above. Another way shadows are used is to infer movement, and with this, our visual system makes the further assumption that a moving shadow is the result of a moving object, rather than being due to a moving light source. In theory, of course, the movement of a shadow could be due to either cause, but we've evolved to ignore one of those possibilitiesrapidly moving objects are much more likely than rapidly moving lights, not to mention more dangerous.

2.10.1. In Action

Observe how your brain uses shadows to construct the 3D model of a scene. Watch the ball-in-a-box movie at:

http://gandalf.psych.umn.edu/~kersten/kersten-lab/images/ball-in-a-box.mov (small version)
http://gandalf.psych.umn.edu/~kersten/kersten-lab/demos/BallInaBox.mov (large version, 4 MB)

If you're currently without Internet access, see Figure 2-12 for movie stills.

The movie is a simple piece of animation involving a ball moving back and forth twice across a 3D box. Both times, the ball moves diagonally across the floor plane. The first time, it appears to move along the floor of the box with a drop shadow directly beneath and touching the bottom of the ball. The second time the ball appears to move horizontally and float up off the floor, the shadow following along on the floor. The ball actually takes the same path both times; it's just the path of the shadow that changes (from diagonal along with the ball to horizontal). And it's that change that alters your perception of the ball's movement. (Figure 2-12 shows stills of the first (left) and second (right) times the ball crosses the box.)

Figure 2-12. Stills from the "ball-in-a-box" movie

Now watch the more complex "zigzagging ball" movie (http://www.kyb.tue.mpg.de/links/demo.html; Figure 2-13 shows a still from the movie), again of a ball in motion inside a 3D box.

Figure 2-13. A still from the "zigzagging ball" movie¹

This time, while the ball is moving in a straight line from one corner of the box to the other (the proof is in the diagonal line it follows), the shadow is darting about all over the place. This time, there is even strong evidence that it's the light sourceand thus the shadowthat's moving: the shading and colors on the box change continuously and in a way that is consistent with a moving light source rather than a zigzagging ball (which doesn't produce any shading or color changes!). Yet still you see a zigzagging ball.

2.10.2. How It Works

Your brain constructs an internal 3D model of a scene as soon as you look at one, with the influence of shadows on the construction being incredibly strong. You can see this in action in the first movie: your internal model of the scene changes dramatically based solely on the position and motion of a shadow.

I feel bad saying "internal model." Given that most of the information about a scene is already in the universe, accessible if you move your head, why bother storing it inside your skull too? We probably store internally only what we need to, when ambiguities have been involved. Visual data inside the head isn't a photograph, but a structured model existing in tandem with extelligence, information that we can treat as intelligence but isn't kept internally.

T.S.

The second movie shows a couple more of the assumptions (of which there are many) the brain makes in shadow processing. One assumption is that darker coloring means shadow. Another is that light usually comes from overhead (these assumptions are so natural we don't even notice they've been made). Both of these come into play when two-dimensional shapesordinary picturesappear to take on depth with the addition of judicious shading [Hack #20] .

Based on these assumptions, the brain prefers to believe that the light source is keeping still and the moving object is jumping around, rather than that the light source is moving. And this despite all the cues to the contrary: the lighting pattern on the floor and walls, the sides of the box being lit up in tandem with the shifting shadowthese should be more than enough proof. Still, the shadow of the ball is all that the brain takes into account. In its quest to produce a 3D understanding of a scene as fast as possible, the brain doesn't bother to assimilate information from across the whole visual field. It simplifies things markedly by just assuming the light source stays still.

It's the speed of shadow processing you have to thank for this illusion. Conscious knowledge is slower to arise than the hackish-but-speedy early perception and remains influenced by it, despite your best efforts to see it any other way.

2.10.3. End Note

Zigzagging ball animation thanks to D. Kersten (University of Minnesota, U.S.) and I. Bülthoff (Max-Planck-Institut für biologische Kybernetik, Germany)

2.10.4. See Also

The Kersten Lab (http://gandalf.psych.umn.edu/~kersten/kersten-lab) researches vision, action, and the computational principles behind how we turn vision into an understanding of the world. As well as publications on the subject, their site houses demos exploring what information we can extract from what we see and the assumptions made. One demo of theirs, Illusory Motion from Shadows (http://gandalf.psych.umn.edu/~kersten/kersten-lab/images/kersten-shadow-cine.mov), demonstrates how the assumption that light sources are stationary can be exploited to provide another powerful illusion of motion.
Kersten, D., Knill, D., Mamassian, P., & Buelthoff, I. (1996). Illusory motion from shadows. Nature, 379(6560), 31.