Hack24.Create Illusionary Depth with Sunglasses

Hack 24. Create Illusionary Depth with Sunglasses

We can use a little-known illusion called the Pulfrich Effect to hack the brain's computation of motion, depth, and brightnessall it takes is a pair of shades and a pendulum.

This is a journey into the code the visual system uses to work out how far away things are and how fast they are moving. Both of the variablesdepth and velocitycan be calculated by comparing measurements of object position over time. Rather than have separate neural modules to figure out each variable, performing the same fundamental processing, the brain combines the two pieces of work and uses some of the same cells in calculating both measures. Because depth and motion are jointly encoded in these cells, it's possible (under the right circumstances) to convert changes in one into changes in another. An example is the Pulfrich Effect, in which a moving pendulum and some sunglasses create an illusion of the pendulum swinging in ellipses rather than in straight lines. It works because the sunglasses create an erroneous velocity perception, which gets converted into a depth change by the time it reaches your perception. It's what we'll be trying out here.

2.13.1. In Action

Make a pendulum out of a piece of string and something heavy to use as a weight, like a bunch of keys. You'll also need a pair of sunglasses or any shaded material.

Ask a friend to swing the pendulum in front of you in a perpendicular plane, and make sure it's going exactly in a straight line, left to right. Now, cover one of your eyes with the shades (this is easiest if you have old shades and can poke one of the lenses out). Keep both eyes open! You'll see that the pendulum now seems to be swinging back and forth as well as side to side, so that it appears to move in an ellipse. The two of you will look something like Figure 2-21.

Figure 2-21. Matt and Tom use sunglasses and a pendulum made out of a bootlace to test the Pulfrich Effect

Show your friend swinging the pendulum how you see the ellipse, and ask her to swing the pendulum in the opposite manner to counteract the illusion. Now the pendulum appears to swing in a straight line, and the thing that seems odd is not the distance from you, but the velocity of the pendulum. Because it really is swinging in an elliptical pattern, it covers perceived distance at an inconsistent rate. This makes it seem as if the pendulum is making weird accelerations and decelerations.

2.13.2. How It Works

The classic explanation for the Pulfrich is this: the shading slows down the processing of the image of the object in one eye (lower brightness means the neurons are less stimulated and pass on the signal at a slower rate [Hack #11] ); in effect, the image reaches one eye at a delay compared to when it reaches the other eye. Because the object is moving, this means the position of the image on the retina is slightly shifted. The difference in image perception between the two retinas is used by the visual system to compute depth [Hack #22] . The slight displacement of the image on the retina of the shaded eye is interpreted as an indication of depth, as in Figure 2-22.

Figure 2-22. The geometry of the Pulfrich Effect: although the pendulum is, in reality, at point 1, the delay in processing makes it appear to be at point 2 to the shaded eye. When the eyes are taken together, the pendulum therefore appears to be at point 3, at a different length.

This explanation puts the confounding of depth and motion on the geometry of the situationthe point of confusion lies in the world, not in the brain.

Taking recordings of the responses of individual brain cells, Akiyuki Anzai and colleagues have shown that this isn't the whole story. The confounding of motion and depth goes deeper than a mathematical ambiguity that arises from computing real-world interpretations from the visual images on the retinas.

It seems that most of the neurons in the primary visual cortex are sensitive to motion and depth in combination. These neurons are optimally responsive to some combination of motion and depth; what makes up that optimum combination can be varying amounts of motion and depth. This means you when you see something and judge its distance your brain always also makes a judgment about its velocity, and vice versa. From the first point in your primary visual cortex where information from the two eyes is combined (i.e., very early in visual processing), motion and depth are coupled. You don't get a sense of one without getting a sense of the other.

This may result from the use of motion parallax to detect depth [Hack #22] . Moving your head is one of the basic ways of telling how far away something is (you can see spitting cobras using motion parallax by shifting their heads from side to side to work out how far to spit). It works even if you have the use only of one eye.

The joint encoding theory explains why you can get Pulfrich-like effects in situations with less obvious geometry. If you watch television snow with one eye shaded, you will see two sheets of dots, one in front of the other and one moving to the left and one moving to the right. The reasons for this are complex but rest on the way our eyes try and match dots in the images for both eyes and use this matching to calculate depth (stereoscopic vision). Adding a shade to the image in one eye creates a bias so that instead of perceiving all the dots at a single average depth we see two sets of skewed averages, and because depth and motion are jointly encoded, these two planes move as well (in opposite directions).

2.13.3. In Real Life

The Pulfrich Effect can be used to create 3D effects for television, as long as people are willing to watch with one eye shaded. It's hard to do since the motion of the image/camera has to be smooth to create a consistent illusion of depth, but it has been done.1

2.13.4. End Note

1. Descriptions of some TV shows that have included applications of the Pulfrich Effect (http://www.combsmusic.com/RosesReview.html).

2.13.5. See Also

• Anzai, A., Ohzawa, I., & Freeman, R. D. (2001). Joint-encoding of motion and depth by visual cortical neurons: Neural basis of the Pulfrich Effect. Nature Neuroscience, 4, 513-518.

• The Psychology Department at Southern Illinois University Carbondale's Pulfrich Effect page (http://www.siu.edu/~pulfrich) has many links for further information.