Pointing Devices

Direct manipulation of objects on a 2D screen is made possible through the use of a pointing device. Clearly, the best way to point to something is with your fingers. They're always handy; you probably have several of these convenient pointing devices nearby right now. The only real drawback they have is that their ends are too blunt for precisely pointing at high-resolution screens, and high-resolution screens also can't, on their own, recognize being pointed at. Because of this limitation, other pointing devices have taken the place of fingers, and each substitute has its own strengths and weaknesses. The mouse is the most omnipresent today, but it's not clear that the mouse will reign supreme forever.

Light Pens and CRTs

The first computer pointing device, the light pen, was a very logical extension of the finger. You held the light pen in your hand and pointed it at the screen like a pen. It was the perfect tool for direct manipulation, except for the tragic fact that it was completely unusable with computers.

When we use a stylus or any other writing device, we exercise fine motor control of our hand muscles to manipulate the tip of the stylus with our fingers. To do this reliably we have to have something to rest the heel of our hand on; otherwise, our movements are cast adrift. No matter how precise our finger motions are, they drift unless we provide our hand with a firm foundation.

Computers use big, clunky cathode ray tubes or flimsy, delicate liquid crystal display panels as their display screens. These typically face us vertically rather than lying flat on our desks like books and papers. As easy as it is to use a stylus on a sheet of paper on a firm horizontal surface, it is terribly difficult to make precise movements with that same stylus on a vertical surface with your arm and hand in the air, unsupported. Using a pen on vertical display squanders the fine motor control of our fingers and forces us to rely on the gross motor control of the muscles in our arms. These muscles are well suited for moving much greater distances, but they cannot give us the precision we expect for accurate pointing.

It is also extremely difficult to draw on a vertical surface while resting the ball of your hand on it—try it on your wall. Your wrist just won't bend backwards far enough. Sign painters, who must paint on the vertical surfaces of walls, doors, and windows, frequently use a tool called a mahlstick—a half-meter–long wooden dowel with a padded end. The artist rests the padded end on the wall and holds the other end in her free hand. Then she rests the heel of her drawing hand on the center of the stick. The mahlstick enables her to change the relative incidence of the painting surface from pure vertical to one that is better suited to keeping her drawing hand under control.

Unfortunately, a mahlstick is impractical for computer users, so we invented other tools, like the mouse.

Mice and indirect manipulation

As you roll the mouse around on your desktop, you see a visual symbol, the cursor, move around on the video screen in the same way. Move the mouse left and the cursor moves left; move the mouse up and the cursor moves up. As you first use the mouse, you immediately get the sensation that the mouse and cursor are connected, a sensation that is extremely easy to learn and equally hard to forget. This is good, because perceiving how the mouse works by inspection is nearly impossible. There is a scene in the movie Star Trek IV: The Voyage Home, where a character from the 24^th century comes to 20^th century Earth and tries to work a computer. He picks up the mouse, holds it to his mouth, and speaks into it. This scene is funny because of its underlying truth: The mouse has no visual affordance that it is a pointing device until someone shows us how the movements of the cursor are related to its movements. At that point, understanding is instantaneous. All idioms must be learned. Good idioms need only be learned once, and the mouse is certainly a good idiom in that regard.

The motion of the mouse to the cursor is not usually one-to-one, however. Instead, the motion is proportional. On most PCs, the cursor crosses an entire 30-centimeter screen in about 4 centimeters of mouse movement. With the heel of your hand resting firmly on the tabletop, your fingers can move the mouse with great accuracy. The fine motor control of the muscles in your hand enables you to precisely place the cursor, even with a 1:8 movement ratio. Those users who have a difficult time mastering the mouse usually don't place the heel of their palm firmly on their desk.

Although we use the term direct manipulation when we talk about pointing and moving things with the mouse, we are actually manipulating these things indirectly. A light pen points directly to the screen and can more properly be called a direct-manipulation tool because we actually point to the object. With the mouse, however, we are only manipulating a mouse on the desk, not the object on the screen.

With a thin-bodied stylus, we can get very precise control of the point, but with the palm-sized mouse, the muscles in our fingertips don't come into play the way they can with a pen. This is why we cannot enter handwriting practically with a mouse. Although we utilize fine motor control with a mouse, it is nothing like the extremely detailed control we exercise with the tip of a pen. With our hand wrapped around the much larger mouse, we can easily move the cursor to a particular place, but we cannot effectively define shapes or make the continuous self-relative movements that are required either for cursive or block printing. Thus the mouse is great for pointing at things on the screen, but miserable for entering graphical data. The stylus is fine for both tasks, assuming a horizontal surface for data entry.

Other pointing devices

The mouse is a clever tool that allows us to point to things on a vertical screen without entangling ourselves with the drawbacks of pointing or drawing on a vertical surface. In all other cases, it is worse than a pen. The fact that you can enter cursive handwriting with a pen and that you cannot do so with a mouse should be clue enough that the pen can be more precisely manipulated than the mouse. It is only when the writing surface goes vertical that the mouse emerges as the better tool.

There are many other pointing devices that have competed with the mouse for years but have not been universally accepted. These include trackballs, digitizing tablets, and touchpads.

TRACKBALLS

Trackballs have been around almost as long as microcomputers, but besides their use in games and early laptops, they have never really caught on. There's a good reason for this—trackballs are simply more awkward to use than mice. The biggest issue is placement of buttons. It is arguable that trackballs provide a degree of precision equal to or greater than that of mice, as far as moving the cursor goes. A trackball doesn't need to slide and is, therefore, more compact in terms of footprint on a desk. However, the buttons on the trackball, by necessity, need to be off to the side where they are awkward to click. Perhaps, if the button (for a Mac anyway) could be placed under the ball so that pressing gently on the ball itself activates a click (similar in concept to the no-button Apple mouse), the trackball might gain greater acceptance.

DIGITIZING TABLETS

Digitizing tablets have also been around for years, and although they have a dedicated following among artists and graphic designers, most people find them almost impossible to use to navigate a computer desktop. Part of the problem might be that people have been too well trained in using the mouse. The mouse is a relative pointing device—picking it up and putting it down elsewhere doesn't alter the position of the cursor on the screen; only motion of the mouse on a surface affects cursor location. Tablets are generally absolute pointing devices—each location on the tablet maps directly to an analogous location on the screen. So if you pick up the pen from the top-left corner and put it down in the bottom-right, the cursor will immediately jump from the top-left to the bottom-right of the screen. The really big problem with the digitizer tablet is that the display and digitizing area are separate. For some reason, this really causes ease of use problems for most people. Add to that the high price of these tablets, and you can see why they've never gone mainstream.

TOUCHPADS

Touchpads (Apple calls them trackpads, with some good reason) are, to date, the most successful pointing device after the mouse, largely because of their ubiquitous placement on laptop computers, which are quickly becoming more popular than desktop systems. Touchpads are fascinating because they combine behaviors of mouse, trackball, and digitizing pad. They use a tiny digitizing surface like a tablet, but like a trackball or mouse, you use your fingers, not a specialized pen. Like a mouse, they are relative pointing devices—movement of the cursor depends only on you moving your finger on the pad surface, not on your finger's position on the surface. Like a trackball, their buttons are placed, by necessity, in an awkward position where it is hard to access them except with the thumb. More recent touchpad drivers let you tap or double-tap the pad to click and double-click. This works well as long as you don't accidentally slide your finger while tapping, which moves the cursor. Of course, if you need to right-click (in Windows), you're stuck with the awkward hardware button. Although they are not perfect, touchpads have one great advantage to which they owe their success: You don't need to bring along a mouse when you travel with your laptop.

THE TRACKPOINT

IBM and Toshiba laptops, as well as a few other brands, make use of a somewhat bizarre pointing device called a TrackPoint, which consists of a pencil-eraser–sized nub stuck in the middle of its keyboard, nestled between the G, H, and B keys. Nudging this eraser-head with your finger moves the cursor in the direction of the nudge. Although some people swear by these devices, many users find them difficult to control, and they suffer from the same button placement problem as touchpads.

The return of the pen?

About 10 years ago, the first affordable products that made use of a technology that merged LCD displays and digitizing tablets—touchscreens—appeared. One of the earliest and best known of these was, undoubtedly, the Apple Newton. This device allowed users to use either a stylus or their fingers to directly manipulate objects on the screen, to write on the screen directly, and even to translate handwriting to computer-readable text (though the original Newton became notorious for its handwriting recognition problems). The design was such that you could hold the Newton in your hand like a notepad, thus providing the much-needed horizontal orientation of the display. This, in turn, allowed users to execute the precision movements that a pen is capable of. The device was large enough to support the hand properly, and this was perhaps part of its downfall—many people found it too large to stick in a pocket, thus critically weakening its purported function as an electronic notepad.

Palm Computing quickly followed Apple's lead. They created a similar, smaller device that did not recognize handwriting, but instead recognized a simplified alphabet called Graffiti. Oddly, you could not enter graffiti characters on the screen itself without third-party software; instead you needed to enter them in a digitizing area below the screen—perhaps the weakest part of the Palm interaction, and one that Palm licensees like Sony have since fixed. You could, however, use a stylus or your finger to directly manipulate objects on its small, square, digitized display. The most important success factor for Palm was that its device was small and simple. It was and still is a runaway success, and the Palm OS has now even found its way into some cellular phones.

By the time you read this book, Microsoft will have introduced its Tablet PC platform. Microsoft had released a version of Windows for pen-based computers years ago, but the hardware technology was still too slow, big, and heavy for it to catch on. This time around, the Tablet PCs will be light and fast, with high-resolution displays and powerful handwriting recognition. But most important, they will be in a form-factor that is comfortable for the pen-wielding human hand. Will the pen finally be in a position to challenge the mouse? Only time will tell.

For the time being, however, we must still contend with the mouse. Let's explore its interaction model further.