Overview of the Managed API | Building Tablet PC Applications (Pro-Developer)

Overview of the Managed API

You ve now been introduced to the various components that constitute the Tablet PC Platform SDK. You ve also successfully set up the SDK in your development environment. It s high time for some code! The following code listing shows the Tablet Input API, Ink Data Management API, and Ink Recognition API interacting together in one application. The bulk of the code sets up the necessary UI very little code is required to demonstrate the application s simple use of the managed API.

The application creates a window with an area that permits inking. When the Recognize button is pressed, ink is recognized into text at the bottom of the dialog. The next three chapters describe in detail the parts of the managed API used in this application. The code below is available on the companion CD as HelloManagedAPI. If you try the code directly from the installed samples, simply open its solution file (HelloManagedAPI.sln), compile, and run. Alternatively, if you d like to get a better feel for the code by entering it directly, create a new project and add the following code as its own C# file into the project. Then you ll be able to compile and run the code you ve entered.

HelloManagedAPI.cs

//////////////////////////////////////////////////////////////////// // HelloManagedAPI.cs // // (c) 2002 Microsoft Press, by Philip Su // // This program demonstrates basic usage of the managed API //////////////////////////////////////////////////////////////////// using System; using System.Drawing; using System.Windows.Forms; using Microsoft.Ink; public class HelloManagedAPI : Form { private Panel m_pnlInput; // The inking surface private Button m_btnReco; // Recognize button private Label m_labelReco; // Recognition text private InkCollector m_inkCollector; // Ink collector object static void Main() { Application.Run(new HelloManagedAPI()); } public HelloManagedAPI() { SuspendLayout(); // Create and place all of our controls m_pnlInput = new Panel(); m_pnlInput.BorderStyle = BorderStyle.Fixed3D; m_pnlInput.Location = new Point(8, 8); m_pnlInput.Size = new Size(352, 192); m_btnReco = new Button(); m_btnReco.Location = new Point(8, 204); m_btnReco.Size = new Size(100, 24); m_btnReco.Text = "Recognize"; m_btnReco.Click += new EventHandler(BtnRecoClick); m_labelReco = new Label(); m_labelReco.Location = new Point(120, 204); m_labelReco.Size = new Size(230, 24); m_labelReco.Text = "(press Recognize to interpret ink)"; // Configure the form itself AutoScaleBaseSize = new Size(5, 13); ClientSize = new Size(368, 236); Controls.AddRange(new Control[] { m_pnlInput, m_btnReco, m_labelReco }); FormBorderStyle = FormBorderStyle.FixedDialog; MaximizeBox = false; Text = "HelloManagedAPI"; ResumeLayout(false); // Create a new InkCollector using m_pnlInput // for the collection area and turn on ink // collection. m_inkCollector = new InkCollector(m_pnlInput.Handle); m_inkCollector.Enabled = true; } // Handle the tap of the Recognize button private void BtnRecoClick(object sender, EventArgs e) { Strokes strokes = m_inkCollector.Ink.Strokes; // When no recognizers are installed, such as on a // non-Tablet PC computer, recognition attempts will // throw an exception. try { // Recognize the strokes from the ink collector's // ink object by using the ToString() method m_labelReco.Text = "Recognized text: \"" + strokes.ToString() + "\""; } catch (Exception) { m_labelReco.Text = "No recognizers installed!"; } } }

The HelloManagedAPI class derives from Form to provide the application s main UI. In its construction, we create and size a few controls. More importantly, at the end of the constructor we create and enable an InkCollector object, which is one of the objects that capture digital ink in the managed API. The only other thing we do is respond to a tap on the Recognize button by converting all the strokes of the collected ink into a string. Don t worry if you don t quite understand all that s going on yet HelloManagedAPI is here only to introduce you to some managed API fundamentals. You ll get a chance to explore the managed API in much greater depth in coming chapters.

All That You Can t Leave Behind

Let s step back a second to take a high-level view of the managed API and its role alongside the rest of the SDK. In this section, we ll begin by discussing the design goals of the API as well as the features that are (and are not) meant to be addressed by it. Then we present the chief components and objects of the managed API and the interrelationships among them. Finally we dive into a brief survey of the distinctions between the core managed API and the Ink controls, which will help you decide which to use based on your needs.

The managed API is the essential component of the Tablet PC Platform SDK. All other components of the SDK are either variations or simplifications of the managed API and its capabilities. For instance, the COM automation API is simply a variation of the managed API, rigged to be automation compatible so that it can be used from both native C++ and Visual Basic 6. Similarly, the Ink controls are subsets of the managed API, packaged as controls to make them even easier to use.

You will see the influence of the managed API pervading all other components of the SDK because it is the fundamental way by which Tablet PC technologies are exposed. Various methods and properties will sound familiar even when using the Ink controls or the COM automation API because they have been shaped by the managed API design.

A solid understanding of the managed API is thus indispensable when using the SDK. This section presents the foundations of the managed API that will be useful to all Tablet PC developers, whether you end up using the Ink controls, the COM automation API, or the managed API itself.

Design Goals of the API

The managed API was designed with several key goals in mind. In designing any API, there are always trade-offs to be made that favor some imagined usage scenarios over others. As with all first-version APIs, only time (and candid feedback from alert readers such as you!) will tell how closely the imagined usage scenarios match real needs.

That said, several of the design goals behind the managed API follow. Keep in mind that these are merely goals, and that the jury is still out on whether they have indeed been achieved!

Ease of use
One of the key motivators behind the SDK being grounded in the managed API (as opposed to, say, the COM automation API) is its ease of use. There are certainly more developers that understand C++/COM and Visual Basic/automation than there are Microsoft .NET developers (at least at the time of writing this book!). However, Microsoft as a whole has bet very heavily on .NET as the future of Windows development. In similar style, the Tablet PC Platform SDK features the managed API as its core technology. Both C# and Visual Basic .NET are easier to use than traditional COM, as almost any developer who has tried both will attest.
Flexibility
It is impossible, particularly in designing the first version of an API, to foresee all the uses to which developers will put an API. Recognizing this, the API has been developed in a way that permits a bit of flexibility in what can be done with it. There are numerous ways in which the API enables you to control its assortment of features. However, it is likely that you will find certain aspects of the API not flexible enough, especially if your usage scenarios differ greatly from those that the API s designers imagined. This book will endeavor to provide you with knowledge of how to work around some of these limitations.
Performance
As always, there are many difficult trade-offs that can be made in the arena of performance. Designers of the managed API definitely sought to optimize some scenarios in favor of others. Perhaps the most important performance goal that drove the API s development is that digital ink should be as close to physical ink as possible in both its capture and its appearance. API performance characteristics and their implications are discussed in detail in Chapter 9.

The Prime Directive

The goal of the SDK in general, and of the managed API in particular, can be summarized in two words: digital ink. Enabling digital ink and its many scenarios is the primary driver behind the managed API s design. One of the key differentiators between the Tablet PC and the tablet-like devices that came before it is that the Tablet PC experience is focused around the intelligent use of digital ink. Other devices and their respective SDKs have largely aimed at supporting pens as a means of text entry. However, the Tablet PC is all about digital ink. When someone mentions the Tablet PC Platform SDK or the managed API, you should think digital ink.

What s Covered by the Managed API

There are many aspects of Tablet PCs and the Tablet PC way of life that could have been addressed by the managed API. The managed API distills from all these possibilities three main categories of Tablet PC features, all related to digital ink. In logical order, these are

Capture
Digital ink must be captured before it can be stored or put to other use. Features related to the capture and concurrent display of ink are conceptually bundled together as the Tablet Input API, which works with hardware drivers to collect digital ink and gestures. It dynamically renders ink as it s drawn by the user, supports basic access to pen buttons and pen tips, and provides notification of pen events.
Manipulation
Once digital ink has been captured, you may wish to do a variety of things with it. Common ink manipulation features supporting the structure, appearance, and storage of ink are grouped in the Ink Data Management API. This API handles ink strokes and their properties (for example, width, shape, and color), and loads, saves, and renders ink.
Interpretation
Digital ink also has an important semantic significance. When a user writes on the Tablet PC, it is often useful to know what was written so that the underlying intended text or gesture can be used as well. Algorithms that interpret digital ink are contained in the Ink Recognition API, which accepts biases when making the interpretation, based on suggestions by the application.

Figure 3-9. Data flow between the Tablet Input API, Ink Data Management API, and Ink Recognition API. It is possible to omit either (or both) the Tablet Input API or the Ink Recognition API from a particular application.

What s Not Covered by the Managed API

An equally important consideration when using the managed API is the set of features that are not supported. Although you may take the omission of some of these features for granted, others may surprise you. Several features explicitly not addressed by the managed API are

Hardware features
Does this Tablet PC have special buttons on the side meant to perform Tablet-specific functions? What type of pen and screen technology is being used? Is the pen in its holster? Does this Tablet have a built-in keyboard? None of these questions is addressed by the managed API, which gives very little access to hardware-specific information.
Document model
Collections of ink invariably have semantic structure or meaning. For instance, maybe your application is SuperDoodle, in which case the digital ink is most likely a series of small drawings. Or perhaps you re developing PowerJot, in which the user writes words and sentences. The managed API does not support (or alternatively, enforce) a particular document model. Although the API supports recognition of ink as text, there isn t much in the API that facilitates the development of semantically structured documents. Depending on your expectations, this may be a boon or an unpleasant surprise.

NOTE
Semantically structured hierarchical document models are prevalent in the SDK world. For instance, Microsoft Word s object model supports manipulations of pages, paragraphs, and lines (all semantic or hierarchical constructs) in addition to just supporting characters. Similarly, the Document Object Model (DOM) from HTML and XML imposes a specific hierarchical structure to its documents. In contrast, the Tablet PC Platform SDK managed API provides only a flat view of digital ink, in which you have ink strokes with no semantic relationships with one another.

Ink storage and organization
The managed API does not enforce the method by which digital ink is to be stored. You may choose to store ink data within the existing file format of your application on disk or in any other inventive place you devise. What the API provides is the ability to save ink to and restore ink from a binary stream. In similar fashion, the API does not provide an automatic way to organize or search ink that has been captured and stored.
Non-ink data types
This is probably obvious but bears calling out explicitly. There are no provisions for the handling or storage of non-ink objects natively by the managed API, although it does allow simple association of arbitrary data with ink. So if your application s document requires support for other data types (for instance, text or images), you ll have to handle that yourself. This restriction does not apply to the Ink controls, which do indeed support text or images in a limited manner.
Ink user interface
Although the SDK documentation makes many suggestions about the proper design of ink-based user interfaces, the managed API often does not directly support the suggested designs. This poses a particular predicament for developers. In the upcoming chapters, we have tried to address as many of these suggestions as possible so that the effects of these omissions are minimized. Here are two of the ink user interface elements missing from the API:
- Selection feedback and heuristics
  There is no support for the recommended selection feedback of both the area being selected and the selected ink. An example of the desired feedback is shown in Figure 3-10. If you want to render the recommended selection feedback, you ll have to implement your own way of manually achieving the same effect. Fortunately, your trusty authors have detailed how this can be done via samples in the next two chapters.
- Correcting misrecognition
  If your application supports recognition of ink into text, you may also want to provide a way for the user to correct the text when it s misrecognized. The Tablet Input Panel and Windows Journal provide the recommended way of doing this by means of a little green widget. This widget, shaped like an inverted L , appears when you hover the cursor over text in the Tablet Input Panel. Tapping the widget reveals a droplist containing alternative interpretations of recognized ink. The managed API, however, does not provide a way for you to programmatically display the correction widget in your own application.
  
  Figure 3-10. The suggested selection feedback includes a dotted trail to indicate the user s path of selection, as well as glowing ink to show what s been selected.

Managed API Object Survey

Now that you understand the design of the managed API and the basic grouping of its features, you re ready to survey in more detail the objects that are part of the managed API. Figure 3-11 shows where various objects in the managed API are conceptually grouped. Some objects are used in more than one group, so they are represented as straddling the groups in which they are used. The managed API contains many more objects than are shown in the figure, which shows only the key objects belonging to each group.

figure 3-11 objects in the managed api grouped into the main categories in which they are used.

Figure 3-11. Objects in the managed API grouped into the main categories in which they are used.

In the following sections we ll introduce the most frequently used objects from the Tablet Input, Ink Data Management, and Ink Recognition APIs.

Tablet Input API

The InkCollector and InkOverlay objects, as key components of the Tablet Input API, facilitate the capture and live rendering of digital ink as it is entered by the user. They do this by communicating directly with the hardware input drivers and abstracting hardware-generated activity into various events. The application responds to these events by prescribing a course of action to be taken, whether it be rendering a newly captured point, responding to a user s pen gesture, or even ignoring the event. A typical (and greatly simplified) Tablet Input API usage timeline is shown in Figure 3-12.

figure 3-12 a sample timeline of an interaction with the tablet input api

Figure 3-12. A sample timeline of an interaction with the Tablet Input API

Objects in the Tablet Input API fall into two domains: hardware-related objects and ink-capturing objects. Chapter 5 discusses these objects and their events in depth, but for now we ll just introduce them in Table 3-1 and Table 3-2 so you get a feel for what s available.

Table 3-1. Hardware-Related Objects in the Tablet Input API
Object	Description
Tablet	The naming of this object is a somewhat confusing and unfortunate holdover from calling pen-sensitive digitizer pads tablets. The Tablet object represents a two-dimensional hardware input device, such as a mouse, a touch pad, a digitizer tablet, or the pen-sensitive screen on a Tablet PC. You can query a Tablet object to determine the characteristics and capabilities of the hardware it represents.
Tablets	A collection of Tablet objects. Because a Tablet object represents a hardware input device, a system can easily have multiple Tablet objects, each representing a different device. In addition to supporting enumeration, the Tablets collection also lets you determine which capabilities are supported by all devices on the system.
Cursor	This is another easily misunderstood name in the API. The Cursor object represents the conceptual tip of a pen, not the arrow that moves around the screen as you move your mouse. To add to the blurring of terminologies, mice and other input devices also have Cursor objects representing them. The Cursor object represents their locus of interaction. Remember the Cursor object is not the visual representation of a pen s tip on screen!
Cursors	A collection of Cursor objects. Some pens may have multiple Cursor objects associated with them (like the nib and eraser ends of a pen). Because mice and other input devices also have Cursor representatives, a computer can easily have three or more Cursor objects.

Table 3-2. Ink-Capturing Objects in the Tablet Input API
Object	Description
InkCollector	The fundamental object used to collect and render ink as it is being entered by the user. InkCollector fires events to your application as they happen and also packages Cursor movements into ink strokes for you. You can tell it what events and what sorts of data you re interested in receiving, as well as whether you want it to paint ongoing ink strokes as they are collected.
InkOverlay	A superset of InkCollector that adds selecting and erasing of the ink that s been captured. For many applications, InkOverlay will be the object of choice to use because its implementation of selection-related features is a real time-saver.

Given the objects listed in Table 3-1, we can imagine what you d do with the Tablet Input API. You might start by investigating the Tablets object to determine what hardware capabilities are available. You d then create an InkOverlay or InkCollector, asking it to capture the user s ink strokes while simultaneously rendering them on screen. As the InkOverlay or InkCollector sends you more events, you might vary its behavior depending on the Cursor being used (for instance, perhaps you ve decided that the front end of the pen will write ink while the back end will erase). These examples are typical of how an application would use the Tablet Input API.

Ink Data Management API

Informally referred to as the Ink API, the Ink Data Management API provides access to the bulk of ink-related activities supported by the Tablet PC Platform. This API assists in such essential actions as changing ink strokes and their properties, merging and splitting strokes, rendering ink, and persisting ink. Surprisingly, there are relatively few objects that belong to this core portion of the managed API. However, don t let their numbers fool you the few objects in the Ink Data Management API are loaded with features!

Several objects belonging to this API are listed in Table 3-3. Note that some of the objects listed may also be used as part of the Tablet Input API or the Ink Recognition API, but the Ink Data Management API is considered to be their home.

Table 3-3. Several Objects from the Ink Data Management API
Object	Description
Ink	The mother of all ink-related activity, the Ink object is the one indispensable part of any Tablet PC programmer s arsenal. All digital ink lives within Ink objects, which support a myriad of methods to add and remove ink strokes to and from them. In addition, a key function of Ink objects is to support several hit detection methodologies to determine whether ink has been hit. Finally, Ink objects can save and load themselves from both binary streams and the clipboard.
Stroke	The Stroke object provides access to a single ink stroke within an Ink object. One of its primary uses is to obtain detailed point-level data about a particular ink stroke, such as its length or pressure. It also supports a small subset of behaviors normally available on an Ink object.
Strokes	A collection of Stroke objects that implement an eclectic subset of the functionality available through the Ink object. One thing to remember when working with a Strokes object is that it is only a collection of references to ink strokes contained within some Ink object. As a consequence, the object s strokes don t have a life of their own independent of the Ink object to which they belong.
DrawingAttributes	Each Stroke object allows for the setting of its DrawingAttributes, which determines the visual appearance of the ink stroke when drawn on screen. Major characteristics of interest include Color, Transparency, Width, and Antialiased, but there are many other tweaks that can be made to the appearance of Stroke objects.

Although much of the API is fairly straightforward, there are several important pitfalls to avoid when using Ink and Stroke objects. Much more will be said about the Ink Data Management API in Chapter 6.

Ink Recognition API

The final pillar of the managed API is the Ink Recognition API, which encapsulates technology originating from Microsoft s ink recognition team. The fundamental purpose of the Ink Recognition API is to suggest possible textual interpretations of written digital ink. Keeping in line with the managed API s flat view of ink, the Ink Recognition API does not provide interpretations regarding the semantic structure of ink, such as whether a set of ink is a drawing or whether a group of ink appears to be a paragraph. Instead, its only function is to suggest what words the writer might have intended when given some ink.

Stay On Target Stay On Target

The Ink Recognition API is unique in that language-specific recognition engines are provided for use in different countries. This creates a situation in which it s hard to discuss the accuracy of the Ink Recognition API in general. Instead, it s only meaningful to discuss the accuracy of any particular language s recognizer exposed via the Ink Recognition API.

Compared with other existing technologies, the Tablet PC Platform s English recognizer is fairly accurate when analyzing cursive English.

There are quite a few objects that belong in the Ink Recognition API. Those most often used are described in Table 3-4.

Table 3-4. The Most Frequently Used Objects in the Ink Recognition API
Object	Description
Recognizer	The Recognizer object represents a particular recognition engine installed on the user s machine. Recognition engines may support one or more languages and may also vary in the features they support.
RecognizerContext	RecognizerContext objects are the usual means by which you interact with recognition engines to perform ink recognition. An instance of this object is created each time you want ink to be recognized, and any hints about the likely content of the ink are set to give the recognizer its context. When all available information has been given to the RecognizerContext, a call to its Recognize method performs the recognition.
RecognitionResult	Results from performing recognition are returned in this object. Its primary use is to provide the text that was recognized as well as possible alternative interpretations of the ink.