Capturing Input from the Pen

Capturing Input from the Pen

Before we cover how to write applications that receive pen input from the tablet digitizer, let s take a look at the software that enables us to harness the power of the pen.

NOTE
This section can be considered optional reading because one doesn t really need to know the internals of tablet input to get inking in an application. You can therefore skip ahead to the section titled Platform SDK Support for Tablet Input if you re not interested in how things work, though we do hope you stick around here because we think it s interesting stuff.

The Tablet PC s tablet input subsystem refers to the logical pieces of software that transfer and transform the data generated from a tablet digitizer into the input an application can make use of. Exactly what kind of input is it that an application can make use of? Chapter 2 gives us a good idea of the experience we want to provide for a Tablet PC user, so let s see whether we can come up with some requirements here that might help us better understand the functions of the input subsystem.

Requirement #1 Mouse Emulation

As you know by now, all tablet-unaware Microsoft Windows XP compatible applications are fully supported under Windows XP Tablet PC Edition the pen behaves like a mouse in these cases. It also happens that mouse input is valuable to tablet-aware applications standard Windows behaviors, controls, and the like are already driven by the mouse, so it s convenient to be able to leverage this functionality for the pen. And of course, it s likely that a physical mouse device will be used to drive the application because many Tablet PCs will have integrated mouse hardware such as a touch pad or mini-trackball.

NOTE
We predict that someday all mouse input handling in Windows will also become tablet input enabled. This will pave the way for pen-specific behaviors in all areas of the Windows user interface.

Because most applications rely heavily on mouse input, our first requirement of the input subsystem is that it be able to map pen input to mouse input a process we ll refer to as mouse emulation. Both left-mouse and right-mouse buttons should be supported for compatibility.

Requirement #2 Digital Ink

Tablet-enabled applications often accept user input in the form of digital ink, which can be added to a document or recognized into a command. Capturing ink from the pen is one of the most important aspects of realizing an electronic paper paradigm. As such, digital ink must mirror physical ink as closely as possible to provide the most natural and unobtrusive end-user experience. From a user s perspective, ink should just work on a Tablet PC and be practically indistinguishable in behavior from physical ink. We can therefore impose the following sub-requirements on a system that s used to capture digital ink:

• Performance

  Ink should appear to be flowing directly out of the tip of the pen in real time and never lagging behind. This requires that the time between sampling the position of the pen and rendering ink on screen be imperceptible to an average user.

• Accuracy

  Ink should follow the exact path of the pen as the pen moves. This requires that the frequency and resolution at which data is captured should result in the digital ink appearing to be smooth in shape to an average user. And as you ll see later, using data captured at higher frequency and resolution also helps improve handwriting and gesture recognition results.

• Robust data capture

  Ink should reflect as much of the physical handling that the pen is subjected to as possible. This requires that not only the pen position be sampled, but that support for sampling of pen tip pressure, the angle between the pen and tablet surface, the rotation of the pen body, and the like should also be provided.

In the initial version of the Tablet PC Platform, normal pressure is the only property besides X and Y position that the rendering of ink can reflect. However, future versions of the Tablet PC Platform will likely take more properties into account.

It is entirely possible for you to custom draw ink if you wish to have more properties taken into account, as we ll learn in Chapter 5. In the majority of cases, we think x, y, and pressure yield incredible results.

Requirement #3 Pen-Based Actions

Pen-specific actions like press-and-hold, using the top-of-pen eraser, and pressing pen buttons can become powerful means to streamline the user model of a tablet-aware application. In addition, employing pen input for means other than digital ink is useful because pen-specific properties can enhance existing user interface behaviors. For example, pen rotation could be used to rotate a selection while it s being dragged, or pen pressure could be used to determine how large an erasing area should be used for an eraser tool. Distinguishing between the user tapping the pen and the user dragging it is therefore important functionality.

NOTE
Pen-specific actions should not be confused with ink-based gestures such as scratchout, up arrow, and curlique . Pen actions which are formally known as system gestures don t use ink and don t define any application behavior. They are on a par with mouse actions such as click and drag.

Although the tablet hardware s device driver can easily report top-of-pen use (called pen inversion) and pen button presses, detecting press-and-hold and associated actions such as tap and drag is a nontrivial algorithm that arguably falls outside the driver s scope; this algorithm is best centralized so that any tablet device can use it.

Summing Up the Requirements

We have now specified that the input system must be able to transform raw pen movement into mouse input (supporting left-mouse and right-mouse button emulation), provide realistic digital ink, and detect higher-level pen-based actions (press-and-hold, pen inversion, pen button presses, and tap versus drag). The Tablet PC Platform s tablet input subsystem (which we ll start referring to as the TIS for brevity) provides all this and more.