Hack36.Feel the Presence and Loss of Attention

Hack 36. Feel the Presence and Loss of Attention

Following seemingly identical objects around with your eyes isn't an easy job. Concentrating, it's possible, and the brain can even track objects when they momentarily pass behind things and disappear, but only in certain circumstances.

The problem with attention as a mechanism is that we use it continuouslyit's an intrinsic part of perceptionand consequently it's very hard to spot what it actually does or what giving attention to something actually feels like.

This hack has a go at showing you what allocating attention actually feels like, by getting you to voluntarily give attention to some fairly generic objectsin this case, you'll be tracking small, colored shapes as they move around. And you'll be able to feel what happens to these shapes when you take attention away. These are humble beginningsattention allocation to moving shapesbut we use these mechanisms for following any thing as it moves around: tennis balls, dogs, ants, and cursors.

3.4.1. In Action

Watch the sequence of multiple object tracking (MOT) demonstrations at Dr. Zenon Pylyshyn's Visual Attention Lab (http://ruccs.rutgers.edu/finstlab/demos.htm).¹ Multiple object tracking is a class of experiment based around trying to keep track of many objects (small circles in the first demonstration) simultaneously, as they jiggle about. It tests the limits of your attention and specialized tracking skills.

Just in case you're not online at the moment, Figures Figure 3-4 through Figure 3-6 provide screenshots of the experiments for your convenience.

Figure 3-4. You have to track four of these circles as they move around the screen

Start with the General MOT experiment (http://ruccs.rutgers.edu/finstlab/mot.mov; QuickTime; Figure 3-4). In this demo, you're required to track four of the eight circles as they move around; you're told which four as they flash briefly at the beginning of the movie.

The point of this demonstration is simply to point out that you can indeed attend to more than one object at a time. It's not a trivial matter to follow all four circles around simultaneously, but you'll find that you can gaze at the center of the screen and track your four chosen circles fairly easily, without even having to stare directly at each in turn.

In the Occluder task (http://ruccs.rutgers.edu/finstlab/mot-occ-occlusion.mov; QuickTime; Figure 3-5), the circles have been replaced by identical white squares, and now they disappear occasionally behind bars placed over the field of movement.

Figure 3-5. Tracking the moving shapes becomes harder when they periodically move behind the black bars (outlined in white)

Aside from the newly introduced bars, the experiment is the same as the General MOT experiment; four of the eight squares flash at the beginning, your task being to track those four for the duration of the movie. This certainly isn't as easy as the general MOT experiment and may take a couple of attempts, but you should be able to complete the task.

The Virtual Occluder MOT movie (http://ruccs.rutgers.edu/finstlab/mot-occ-virtocc.mov; QuickTime; Figure 3-6), on the other hand, requires serious concentration. Now instead of sliding behind visible bars, the white squares momentarily vanish. The occludersthe bars that occlude, or hide, the squares behind themare now the same color as the background and have become invisible. Tracking the four white squares that flash at the beginning for the duration of the whole experiment is now a real challenge. You have to give the task your full concentration, and any distraction will cause you to confuse one of your chosen white squares with one of the other, apparently identical, distracter objects. It doesn't help that they all keep disappearing and reappearingthe two smaller white rectangles in Figure 3-6 are just reappearing from behind one of the invisible virtual occluders.

Figure 3-6. Tracking gets harder still: the slimmer white shapes are squares half-hidden behind invisible obstructing bars

While it is still possible to do a reasonable job of tracking the four targets, we've now reached the limits of your attention. But there's one more movie to go: the Implode/Explode task (http://ruccs.rutgers.edu/finstlab/mot-occ-implosion.mov; QuickTime). The single difference between this experiment and the previous (Virtual Occluder) is that here the squares shrink down to a dot when they encounter one of the invisible black bars instead of slipping behind it. On the other side of the bar, they grow back from a dot into a square instead of appearing from underneath the bar's edge.

Give it a whirl. It's not just difficult this time, it's more or less impossible. You can't complete this multiple object tracking task at all.

3.4.2. How It Works

The ability to perform multiple object tracking (MOT) is the skill we get by giving attention to objects: without attention, you can't keep track of which object is which (let alone more than one at a time). Attention is both a mechanism the brain uses for giving some objects more processing time and what you feel is an extra layer on your visual perception. Despite all eight circles in the General MOT experiment having the same visual appearance, you perceive four of them as somehow different, just because they flashed at the beginning. That's the feeling of attention feeding into your visual perception.

If you had followed around only a single circle in that first movie, you would have very easily been able to distinguish it from all of the others. But you wouldn't have been able to distinguish the other seven from one another. That's what attention does.

Although in this case you've applied attention voluntarily to certain objects, it's actually a semiautomatic process. Attention can be captured [Hack #37] by sudden movements, for example. And it doesn't always feel as obvious as "I am now able to distinguish these objects"you momentarily give your attention to every single car that passes you when you're waiting to cross the road, but not in the same way as you give attention to these multiple object tracking demonstrationsit's not a concentrated kind of attention, just an awareness that you've seen it.

Attention is something that can be allocated piecemeal. You can choose to notice certain colors, for example, or look out for particular movements. Conversely, you can choose to suppress attention for those features (that's what negative priming [Hack #42] is all about). In this case, you're choosing to allocate attention to collections of features that appear to move round together: blue-ness, circle-ness, move-at-a-certain-speed-ness. We tend to refer to bundles of features as objects. (It's possible that attention has a role to play in bundling the features together.)

We deal with objects in our attention in a special way, setting up object files that can persist through time. The brain automatically says, "This is an object I need to remember" and sets up a file (a kind of invisible sticky tag on the object) to do it. Imagine if you used your finger to point at something as it moved about, like running your finger along a row in a table of figures. It helps you remember which row, out of all the similar-looking rows, was the one you were following. Object files are like finger pointing, but using attention instead. That's how you know that the circle at the end of the task is the same as the one you identified at the beginning: your brain set up a file about itan index to the bundle of features that are appearing in your visual perceptionand kept that for as long as your attention was on that object.

Given this, the brain must have some automatic processes to reclaim attention as soon as it's no longer needed. One way of doing this would be to close the object file as soon as the object disappears. That would often be a hindrance howeverimagine if you, as an early human living on the African savannah, lost track of a predator every time it went behind a bush.

That's what the Occluder demonstration shows. Object files are kept open when it looks as if the object being tracked is slipping behind some other object in the visual field, in this case black bars. The Virtual Occluder demo is hard because you're maxing out your attentional resources tracking four objects (four or five objects is about the maximum we have room for) and relying entirely on your automatic processes to imagine where the objects are even when they're hidden and reattach the object file when the squares reappear from behind the invisible bars. But it's still possible because this is the kind of situation visual perception needs to deal with: an animal darting around in a forest will keep disappearing behind branches or greenery, and the dense foliage in the foreground has the same pattern as everything else and is basically invisible against the background.

Here's the trick your object files system is using to know not to shut down the file: As the squares disappear behind the barsthat is, as they are occludedthey disappear line by line. They vanish from one edge. That's the cue your brain uses to know occlusion is occurring.

The final demo, the Imploder/Exploder, is a bit of a cheat. It's supposed to be impossible. Tracking that many objects is deliberately hard so it fills your voluntary attentional capacity and forces you to rely on your automatic brain functions. That's a way of getting the automatic functions to reveal themselves.

In this case, the demonstration disrupts the occlusion cue. Shrinking down to a dot, the squares instead present the cue that they're moving away into the distance. Thinking the object has disappeared from the vicinity and is no longer important, your brain immediately swings in and closes the object file, reclaiming the attention for use elsewhere. When the square reappears from a dot a moment later, it's as if it's coming back from a long way away. But the object file has already been closed, it's as if it's a different white square entirely.

It's cues as small as how objects disappear behind other objects that your brain uses, even in cartoons such as these movies (which don't have shadows, or perspective, or even 3D depthanything that we'd usually say makes a scene feel real or physical), to figure out what to track and what to give attention to.

3.4.3. End Note

1. Scholl, B. J., & Pylyshyn, Z. W. (1999). Tracking multiple items through occlusion: clues to visual objecthood. Cognitive Psychology, 38, 259-290. Reprint available on the Visual Attention Lab's publications page (http://ruccs.rutgers.edu/finstlab/ZPPublications.htm).