Targeting and Hit-Testing Ink Strokes

Targeting and Hit-Testing Ink Strokes

In Chapter 2, we learned the importance of making it easy to select ink and manipulate it directly with the pen. Using the pen or mouse to target ink strokes to perform selection or another operation, such as erasing, is known as hit-testing.

Just as certain factors of human physiology must be taken into account when detecting tapping vs. dragging with the pen, we must also consider similar factors in the determination of whether ink has been targeted. The specific considerations depend on the kind of hit-testing being performed, so let s take a look at the various forms of hit-testing that apply to ink.

Different Types of Hit-Testing

A user can perform three main types of targeting, resulting in three types of hit-testing that the Tablet PC Platform supports: point-based, rectangle-based, and lasso-based.

Point-Based

Point-based hit-testing typically occurs when a user taps the pen stylus on the inking surface, and it is most often used to target objects. Ink strokes can often be quite thin, which makes it difficult for a user to consistently tap exactly within the ink of a stroke. We can alleviate this problem by introducing a hit radius, or circular area surrounding the tap point, that an ink stroke merely has to intersect for the point to be considered targeted.

Rectangle-Based

Rectangle-based hit-testing is useful for functions such as a selection tool or an insert/remove space tool. The size of the rectangle is often dynamically specified by the user dragging the pen on the ink surface, and this can result in the targeted ink being different from what the user intended.

The solution is to slightly relax the criteria used to compute whether a stroke is targeted by the rectangle. Instead of requiring an entire stroke to be contained by the rectangle, we can say that n percent of the ink making up a stroke must lie within the rectangle. We ll discuss this further very shortly.

Lasso-Based

Similar in use to rectangle-based hit-testing, lasso-based hit-testing is useful for a selection tool. Ink strokes that are enclosed by the polyline comprising the lasso are considered targeted. Usability studies showed that exactly enclosing ink strokes with a lasso required significant effort on the user s part; fortunately, lasso-based hit-testing criteria can be relaxed with the same method used by rectangle-based hit-testing criteria. If only a certain percentage of ink lies within the lasso, we can consider it targeted.

Percentage-Based Enclosure

What exactly does n percent of ink being enclosed by a rectangle or lasso really mean, anyway? Is it the ratio of the length of ink contained within the rectangle or lasso to the total length of the stroke? Or maybe the ratio of the viewable area of ink contained to the total viewable area? Interestingly, the answer that usability studies and the Tablet PC teams found was essentially, It doesn t matter as much as you might think.

The reason for this is quite simple: percentage-based enclosure is meant to combat the imprecise control that humans exhibit when using a pen to target objects. Because humans are mostly correct in their targeting, a percentage-based enclosure algorithm needs to worry about only most of the ink being enclosed by the rectangle or lasso polygon. This makes the idea of n percentage enclosure somewhat overkill given the problem being solved; my assertion is therefore that as long as an n percentage algorithm is kind of correct, users will be happy.

With that said, the algorithm used by the Tablet PC Platform to compute percentage enclosure for both rectangle and lasso polygons is quite accurate. It is actually a hybrid of the length of ink method mentioned earlier. An ink stroke is divided into a number of points, and if n percent of those points lie within the rectangle or lasso polygon, the ink is considered enclosed. Figure 6-7 shows a visual explanation of this algorithm.

figure 6-7 how percentage-based enclosure is actually calculated (not that it really matters).

Figure 6-7. How percentage-based enclosure is actually calculated (not that it really matters).

Hit-Testing Performance

Percentage-based enclosure is incredibly useful and considerably improves the user experience of targeting ink. As with most very cool things in software, though, there are performance implications to consider. Using enclosure hit-testing functionality is computationally intensive, and the performance can be affected by the following factors:
  • The number of Stroke objects contained by an Ink object

    The more ink strokes there are, the longer the computation will take.

  • The number of packets in a Stroke object

    The more packets there are, the longer the computation will take.

  • The enclosure percentage value

    The computation will take longer if the value is other than 100 (this applies only to rectangle-based hit-testing).

  • The number of points in a lasso polygon

    The more points there are, the longer the computation will take.

Don t get me wrong provided the end user sees a benefit, I heartily recommend using percentage-based enclosure hit-testing. This is just a heads up to the factors you might want to try to minimize to yield an even better targeting experience.

Hit-Testing Functions

Hit-testing functionality for ink strokes in an Ink object is achieved with these methods:

Strokes Ink.HitTest(Point point, float radius) Strokes Ink.HitTest(Rectangle rectangle, float percentage) Strokes Ink.HitTest(Point[] points, float percentage) Strokes Ink.HitTest(Point[] points, float percentage, out Point[] arrLassoPts)

The stroke objects that meet the target criteria are returned in a Strokes collection as the method s return value. The first three flavors of the HitTest method listed above perform point-based, rectangle-based, and lasso-based hit-testing, respectively. They are distinguished by the type of their first parameter: a point, a rectangle, and an array of points (a polygon). Notice that the point-based version of HitTest accepts a radius parameter (provided in ink space units) and the rectangle and lasso versions accept a percentage parameter.

The last version of the HitTest method performs lasso-based hit-testing, but it has a rather curious out parameter named arrLassoPts. Because the lasso polygon is specified as an array of arbitrarily valued points, the polygon can self-intersect. This is an entirely likely scenario, as users often will create all sorts of odd shapes when using a lasso selection tool. One or more self-intersections in a polygon make the user s intent ambiguous, so the arrLassoPts parameter returned is the array of points that was used in the actual hit-test computation. It can be handy for displaying the lasso polyline after the fact so the user is clear on what exactly happened to cause the result of the operation.

NOTE
In version 1 of the Tablet PC Platform, if the polygon specified for a lasso-based hit-test has one or more self-intersections, the first proper polygon encountered will be used for the computation. This should not be assumed to always be the case because other algorithms may be employed in the future.

The Stroke class also has one handy hit-testing function that simply says whether a Stroke object is within the radius of a point:

bool Stroke.HitTest(Point point, float radius)

The value returned indicates whether the ink stroke was hit true if it was, and false if it was not.

Computing the Nearest Point

A variant of hit-testing functionality is the computation of the closest Stroke or point along a Stroke to a target point:

Stroke Ink.NearestPoint(Point point) Stroke Ink.NearestPoint(Point point, out float indexOnStroke) Stroke Ink.NearestPoint(Point point, out float indexOnStroke, out float distance)

These methods return the Stroke object, if any, that is closest to the point specified. Optional out-parameters are the floating-point index of the point along the Stroke object that is closest and that point s distance away from the input point.

The Stroke class also has its own versions of the NearestPoint method:

float Stroke.NearestPoint(Point point) float Stroke.NearestPoint(Point point, out float distance)

The methods return the floating-point index of the closest point and optionally that point s distance away from the input point.

Sample Application InsertRemoveSpace

The InsertRemoveSpace sample application shows an implementation of the insert/remove space mode that can be found in the Windows Journal utility. Dragging the pen up or down a page of ink causes space to be inserted or removed, resulting in the ink below the pen-down location being moved up or down. Usability testing showed that without any tolerance implemented, users had difficulty reliably targeting the ink they wanted to be affected by the operation. That led to using rectangle-based percentage hit-testing when determining the ink strokes to move as a result of the operation.

InsertRemoveSpace.cs

//////////////////////////////////////////////////////////////////// // // InsertRemoveSpace.cs // // (c) 2002 Microsoft Press // by Rob Jarrett // // This program demonstrates an implementation of the "Insert/ // Remove Space" mode in Windows Journal. // //////////////////////////////////////////////////////////////////// using System; using System.Drawing; using System.Windows.Forms; using Microsoft.Ink; using MSPress.BuildingTabletApps; public class frmMain : Form { private InkInputPanel pnlInput; private Button btnFormat; private CheckBox cbSpaceMode; private InkCollector inkCollector; private Point ptStart; private Point ptPrev; private Strokes strksToMove; // Entry point of the program [STAThread] static void Main() { Application.Run(new frmMain()); } // Main form setup public frmMain() { SuspendLayout(); // Create and place all of our controls pnlInput = new InkInputPanel(); pnlInput.BackColor = Color.White; pnlInput.BorderStyle = BorderStyle.Fixed3D; pnlInput.Location = new Point(8, 8); pnlInput.Size = new Size(352, 192); btnFormat = new Button(); btnFormat.Location = new Point(8, 204); btnFormat.Size = new Size(50, 20); btnFormat.Text = "Format"; btnFormat.Click += new System.EventHandler(btnFormat_Click); cbSpaceMode = new CheckBox(); cbSpaceMode.Location = new Point(66, 204); cbSpaceMode.Size = new Size(172, 20); cbSpaceMode.Text = "Insert/Remove Space mode"; cbSpaceMode.Click += new System.EventHandler(cbSpaceMode_Click); // Configure the form itself ClientSize = new Size(368, 236); Controls.AddRange(new Control[] { pnlInput, btnFormat, cbSpaceMode}); FormBorderStyle = FormBorderStyle.FixedDialog; MaximizeBox = false; Text = "InsertRemoveSpace"; ResumeLayout(false); // Hook up to mouse events to perform custom mode behaviors pnlInput.MouseDown += new MouseEventHandler(pnlInput_MouseDown); pnlInput.MouseMove += new MouseEventHandler(pnlInput_MouseMove); pnlInput.MouseUp += new MouseEventHandler(pnlInput_MouseUp); pnlInput.Paint += new PaintEventHandler(pnlInput_Paint); // Create a new InkCollector, using pnlInput for the collection // area inkCollector = new InkCollector(pnlInput.Handle); inkCollector.AutoRedraw = false; // Start out in ink mode inkCollector.CollectionMode = CollectionMode.InkOnly; cbSpaceMode.Checked = false; // We're set to go, so enable tablet input inkCollector.Enabled = true; } // Handle the click of the format button private void btnFormat_Click(object sender, EventArgs e) { FormatInkDlg dlgFormat = new FormatInkDlg(); dlgFormat.DrawingAttributes = inkCollector.DefaultDrawingAttributes; if (dlgFormat.ShowDialog(this) == DialogResult.OK) { inkCollector.DefaultDrawingAttributes = dlgFormat.DrawingAttributes; } } // Handle the click of the Insert/Remove Space checkbox private void cbSpaceMode_Click(object sender, EventArgs e) { // Turn inking on or off inkCollector.Enabled = !cbSpaceMode.Checked; // Update the cursor shown if (inkCollector.Enabled) { pnlInput.Cursor = System.Windows.Forms.Cursors.Default; } else { pnlInput.Cursor = System.Windows.Forms.Cursors.HSplit; } } // Handle mouse down in the input panel private void pnlInput_MouseDown(object sender, MouseEventArgs e) { if (!inkCollector.Enabled) { ptStart = ptPrev = new Point(e.X, e.Y); // Compute the rectangle to use to hit-test the ink below // the mouse-down point - this is done by enumerating all // ink strokes and merging all of their rects together. // Then, the top of the rect is set to equal the mouse-down // point. Rectangle rcHitArea = new Rectangle(); foreach (Stroke s in inkCollector.Ink.Strokes) { Rectangle rcBounds = inkCollector.Renderer.Measure(s); if (rcHitArea.IsEmpty) { rcHitArea = rcBounds; } else { rcHitArea = Rectangle.Union(rcHitArea, rcBounds); } } // Convert the mouse-down point into ink coordinates Point ptTop = new Point(0, e.Y); Graphics g = pnlInput.CreateGraphics(); inkCollector.Renderer.PixelToInkSpace(g, ref ptTop); g.Dispose(); // Set the top of the hit area to be the mouse-down point int nBottom = rcHitArea.Bottom; rcHitArea.Y = ptTop.Y; rcHitArea.Height = nBottom - rcHitArea.Y; if (rcHitArea.Height > 0) { // Perform the initial hit-test to determine the // candidates to move strksToMove = inkCollector.Ink.HitTest(rcHitArea, 70); if (strksToMove.Count > 0) { // Reflect which strokes are candidates to move pnlInput.Invalidate(); } else { strksToMove = null; } } } } // Handle mouse move in the input panel private void pnlInput_MouseMove(object sender, MouseEventArgs e) { if (!inkCollector.Enabled) { if (e.Button == MouseButtons.Left) { // Erase the previous tracking rectangle Rectangle rcTrack = new Rectangle(0, ptStart.Y, pnlInput.ClientSize.Width, ptPrev.Y - ptStart.Y); Rectangle rcDraw = rcTrack; rcDraw.Location = pnlInput.PointToScreen(rcDraw.Location); ControlPaint.DrawReversibleFrame( rcDraw, BackColor, FrameStyle.Dashed); // Update rectangle with new mouse location, // constraining to the input panel's bounds rcTrack.Height = e.Y - ptStart.Y; if (rcTrack.Y + rcTrack.Height > pnlInput.ClientSize.Height) { rcTrack.Height = pnlInput.ClientSize.Height - rcTrack.Y; } else if (rcTrack.Y + rcTrack.Height < 0) { rcTrack.Height = -rcTrack.Y; } // Draw the new tracking rectangle rcDraw = rcTrack; rcDraw.Location = pnlInput.PointToScreen(rcDraw.Location); ControlPaint.DrawReversibleFrame( rcDraw, BackColor, FrameStyle.Dashed); ptPrev = new Point(e.X, rcTrack.Bottom); } } } // Handle mouse up in the input panel private void pnlInput_MouseUp(object sender, MouseEventArgs e) { if (!inkCollector.Enabled) { // Erase the previous tracking rectangle Rectangle rcTrack = new Rectangle(0, ptStart.Y, pnlInput.ClientSize.Width, ptPrev.Y - ptStart.Y); Rectangle rcDraw = rcTrack; rcDraw.Location = pnlInput.PointToScreen(rcDraw.Location); ControlPaint.DrawReversibleFrame( rcDraw, BackColor, FrameStyle.Dashed); if (strksToMove != null && strksToMove.Count > 0) { // Compute the distance to move the strokes by Point ptTop = ptStart; Point ptEnd = new Point(e.X, e.Y); Graphics g = pnlInput.CreateGraphics(); inkCollector.Renderer.PixelToInkSpace(g, ref ptTop); inkCollector.Renderer.PixelToInkSpace(g, ref ptEnd); g.Dispose(); // Perform the actual move operation strksToMove.Move(0, ptEnd.Y - ptTop.Y); strksToMove = null; // Show the result of the move pnlInput.Invalidate(); } } } // Handle painting in the input panel private void pnlInput_Paint(object sender, PaintEventArgs e) { if (strksToMove == null strksToMove.Count == 0) { // Draw all the strokes normally since no move tracking // operation is going on inkCollector.Renderer.Draw( e.Graphics, inkCollector.Ink.Strokes); } else { // Space tracking going on; show the candidates to be moved // by making them transparent foreach (Stroke s in inkCollector.Ink.Strokes) { if (strksToMove.Contains(s)) { // Draw the stroke partially transparent since it // is a candidate to be moved DrawingAttributes da = s.DrawingAttributes.Clone(); da.Transparency = 192; inkCollector.Renderer.Draw(e.Graphics, s, da); } else { // Draw the stroke normally inkCollector.Renderer.Draw(e.Graphics, s); } } } } }

The application doesn t use either the InkControl or InkControl2 class because it implements a custom input mode both of those classes only provide InkOverlay s editing mode functionality and hence cannot be modified. To reduce flicker, an InkInputPanel is used in conjunction with the manual rendering of ink strokes in the panel s Paint event.

The MouseDown event handler sets up the UI that reflects the amount of space to insert or remove. It then targets the ink strokes to move by computing the bounding box of all the ink strokes in the InkCollector, setting the resulting rectangle s top edge to the y-component of the pen-down/mouse-down location, and then calling Ink.HitTest with the rectangle:

// Perform the initial hit-test to determine the // candidates to move strksToMove = inkCollector.Ink.HitTest(rcHitArea, 70);

The input panel is then invalidated so the contents of the strksToMove collection can be rendered with a transparent style, showing the user the strokes that will be affected when the pen is lifted. The MouseMove event handler performs the actual animation for the UI by utilizing the ControlPaint.DrawReversibleFrame method.

Finally the MouseUp event handler terminates the space amount UI and performs the actual insert or remove space operation using the Strokes class s Move method:

// Perform the actual move operation strksToMove.Move(0, ptEnd.Y - ptTop.Y);

Once the space has been inserted or removed, the strksToMove collection is emptied and the input area is invalidated so the results of the operation are drawn.

Sample Application RealtimeLasso

Let s jump into another sample application to take a further look at the TabletPC Platform s hit-testing functionality. This sample implements lasso percentage-based selection, updating results in real time with selection feedback similar to Windows Journal. It also supports implicit mode switching out of ink mode with a right-tap or right-drag system gesture and provides point-based selection using the tap and right-tap system events.

The first code listing presented is that of a class found in the BuildingTabletApps helper library named LassoUI. This class implements the evenly spaced lasso ink dots that are drawn as a user performs a lasso operation. You may think that the way to implement this effect is to utilize a GDI+ pen style of a circle followed by a space, or something similar. However, this approach has a couple of drawbacks. The first is that we would have to draw the lasso ink in its entirety every single time a point was added to the lasso polyline. That means the longer the lasso ink gets, the longer it will take to render, and we ll see shortly that we already have enough performance issues to deal with. The other drawback is related to an optimization. Recall that I stated earlier that the number of points in a lasso polyline directly affects its performance. The number of points can be minimized if only the points resulting in a dot being drawn are added rather than all points used to trace the lasso s path. That takes some reasonable computation, as we ll soon see and if GDI+ polyline drawing with a pen-style is used to render lasso dots, we can t even be certain that the set of points we d compute will equal the dot positions.

The approach taken with the LassoUI class is that it computes the location of the dots itself and adds points to the lasso polyline when needed. The number of points used in the polyline is therefore automatically minimized.

LassoUI.cs

//////////////////////////////////////////////////////////////////// // // LassoUI.cs // // (c) 2002 Microsoft Press // by Rob Jarrett // // This class implements drawing of the lasso selection tool. // //////////////////////////////////////////////////////////////////// using System; using System.Collections; using System.Drawing; using System.Windows.Forms; namespace MSPress.BuildingTabletApps { public class LassoUI { private int nDotSpacing = 7; private int nDotSize = 4; private Brush brDotColor = Brushes.Orange; private ArrayList arrPts; private Point ptLast; private Rectangle rcBounds; public LassoUI() { arrPts = new ArrayList(); ptLast = new Point(0, 0); rcBounds = new Rectangle(0, 0, 0, 0); } // Start up a lasso public void Start(Graphics g, Point ptStart) { arrPts = new ArrayList(); arrPts.Add(ptStart); ptLast = ptStart; rcBounds = new Rectangle(ptStart, new Size(0, 0)); // Draw the first dot of the lasso DrawLassoDot(g, ptStart); } // Continue creating a lasso public bool Continue(Graphics g, Point ptNew) { // Compute how far the new point is from the last drawn dot int a = (ptNew.X - ptLast.X); int b = (ptNew.Y - ptLast.Y); double c = Math.Sqrt(a * a + b * b); // Is that distance less than the dot spacing? If so, we // can throw away this point if (c < (double)nDotSpacing) return false; // Compute how many dots will need to be drawn int nSegments = (int)(c / nDotSpacing); // Compute new rise and run values "snapped" to dot spacing double ap = a * ((nSegments*nDotSpacing) / c); double bp = b * ((nSegments*nDotSpacing) / c); // Draw lasso dots until the new endpoint is reached Point ptCurr = new Point(ptLast.X, ptLast.Y); for (int i = 1; i <= nSegments; i++) { double ratio = (double)i / nSegments; ptCurr = new Point( ptLast.X + (int)Math.Round(ap * ratio), ptLast.Y + (int)Math.Round(bp * ratio)); DrawLassoDot(g, ptCurr); } // Update the lasso's bounding rectangle if (ptCurr.X < rcBounds.X) { rcBounds.Width += rcBounds.X - ptCurr.X; rcBounds.X = ptCurr.X; } else if (ptCurr.X > rcBounds.Right) { rcBounds.Width = ptCurr.X - rcBounds.X; } if (ptCurr.Y < rcBounds.Y) { rcBounds.Height += rcBounds.Y - ptCurr.Y; rcBounds.Y = ptCurr.Y; } else if (ptCurr.Y > rcBounds.Bottom) { rcBounds.Height = ptCurr.Y - rcBounds.Y; } // Add the endpoint to the polyline arrPts.Add(ptCurr); ptLast = ptCurr; return true; } // Draw the entire lasso in one shot public void Render(Graphics g) { Render(g, new Rectangle()); } // Draw the entire lasso in one shot, using a clipping area to // improve performance public void Render(Graphics g, Rectangle rcClip) { if (arrPts.Count > 0) { // Draw the first point Point ptPrev = (Point)arrPts[0]; DrawLassoDot(g, ptPrev); foreach (Point ptNext in arrPts) { // Compute how far the new point is from the last // drawn dot int a = (ptNext.X - ptPrev.X); int b = (ptNext.Y - ptPrev.Y); double c = Math.Sqrt(a * a + b * b); // Compute how many dots need to be drawn within // this line int nSegments = (int)Math.Round(c / nDotSpacing); // Draw dots until the line's endpoint is reached Point ptCurr = new Point(ptPrev.X, ptPrev.Y); for (int i = 1; i <= nSegments; i++) { double ratio = (double)i / nSegments; ptCurr = new Point( ptPrev.X + (int)Math.Round(a * ratio), ptPrev.Y + (int)Math.Round(b * ratio)); // See if we need to bother drawing the dot Rectangle rcDot = new Rectangle( new Point(ptCurr.X - nDotSize/2, ptCurr.Y - nDotSize/2), new Size(nDotSize, nDotSize)); if (rcClip.IsEmpty rcClip.IntersectsWith(rcDot)) { DrawLassoDot(g, ptCurr); } } ptPrev = ptNext; } } } // Get the current lasso polyline public Point[] Points { get { return (Point[])arrPts.ToArray(typeof(Point)); } } // Get the current bounding rectangle of the dots drawn public Rectangle BoundingRect { get { Rectangle rcResult = new Rectangle(rcBounds.Location, rcBounds.Size); // Inflate the result by the dot size, taking into // account an odd-numbered size. i.e. if we just // performed an Inflate(nDotSize/2,nDotSize/2) and // the dot size were e.g. 5, the bounds would only // increase by 5/2*2 = 4. So we'll use our own // inflation method. rcResult.Offset(-nDotSize/2, -nDotSize/2); rcResult.Width += nDotSize; rcResult.Height += nDotSize; return rcResult; } } // Get or set the dot spacing of the lasso public int DotSpacing { get { return nDotSpacing; } set { if (nDotSpacing < 1) { throw new ArgumentOutOfRangeException(); } nDotSpacing = value; } } // Get or set the dot size of the lasso public int DotSize { get { return nDotSize; } set { if (nDotSize < 1) { throw new ArgumentOutOfRangeException(); } nDotSize = value; } } // Get or set the brush used to draw the lasso dots public Brush DotBrush { get { return brDotColor; } set { brDotColor = value; } } // Render a single lasso dot private void DrawLassoDot(Graphics g, Point pt) { pt.Offset(-nDotSize/2, -nDotSize/2); g.FillEllipse(brDotColor, new Rectangle(pt, new Size(nDotSize, nDotSize))); } } }

The class should be instantiated at the start of a lasso UI operation (for example, the start of a pen drag) and retained until the end of the operation (pen up). After creating the lasso class, the Start and Continue methods should be called to form the lasso polyline and draw the dots:

public void Start(Graphics g, Point ptStart) public bool Continue(Graphics g, Point ptNew)

The lasso ink dots are drawn to the Graphics object g, and the points (specified in pixel coordinates) are filtered to reduce the complexity of the lasso polyline. The Start method should be called only at the start of a lasso because it initializes the polyline. The Continue method is called throughout, returning if a point was added to the lasso polyline this is so a client of the class can conditionally perform a hit-test operation and avoid redundancy.

While a lasso is in progress, the input area might (and probably will) become invalidated as ink strokes rendered selected or unselected. The Render method should therefore be used in the input area s paint handler so that lasso ink dots that have already been drawn won t be erased:

public void Render(Graphics g) public void Render(Graphics g, Rectangle rcClip)

The method draws the current lasso polyline to the Graphics object g, and the optional rcClip parameter results in only the dots contained in rcClip being rendered. This parameter should be supplied if the invalid area is known typically the client s paint handler is provided a PaintEventArgs object containing a ClipRectangle property so rendering performance is improved by avoiding redundant dot drawing.

During the lasso operation, the lasso polyline is obtained with the Points property:

 public Point[] Points

The points returned are in pixels, so be sure to convert them to ink space before calling HitTest. To improve lasso hit test performance, the Continue method only adds points to the Points property when they are spaced far enough to matter. The Continue method does this by ignoring any points that are less than DotSpacing distance from the previous point captured.

When the lasso operation is complete, the BoundingRect property can be used to retrieve the region that should be invalidated for the lasso ink dots to disappear:

public Rectangle BoundingRect

Lastly, the LassoUI class provides some properties that are used to alter the look of the lasso ink dots:

public int DotSpacing public int DotSize public Brush DotBrush

The DotSpacing property gets or sets the distance (in pixels) between the dots, the DotSize property gets or sets the diameter (in pixels) of the dots, and the DotBrush property gets or sets the brush used to draw each dot. You shouldn t need to alter these under regular circumstances except for accessibility functionality (high contrast, for example).

Now that we understand how a lasso polyline is created, we can take a look at the sample application, which uses the LassoUI class.

RealtimeLasso.cs

//////////////////////////////////////////////////////////////////// // // RealtimeLasso.cs // // (c) 2002 Microsoft Press // by Rob Jarrett // // This program demonstrates an implementation of realtime selection // user feedback for a "lasso" operation. It also illustrates how // to render ink in a selected state. // //////////////////////////////////////////////////////////////////// using System; using System.Collections; using System.Drawing; using System.Drawing.Drawing2D; using System.Windows.Forms; using Microsoft.Ink; using MSPress.BuildingTabletApps; public class frmMain : Form { private int nTapDistance = 4; private int nSelThickness = 4; private InkInputPanel pnlInput; private Button btnFormat; private ComboBox cbxEditMode; private InkOverlay inkOverlay; private bool fSelectMode; private Stroke strkCurr; private LassoUI lasso; private Strokes strksSelected; private Strokes strksLassoed; // Entry point of the program [STAThread] static void Main() { Application.Run(new frmMain()); } // Main form setup public frmMain() { SuspendLayout(); // Create and place all of our controls pnlInput = new InkInputPanel(); pnlInput.BackColor = Color.White; pnlInput.BorderStyle = BorderStyle.Fixed3D; pnlInput.Location = new Point(8, 8); pnlInput.Size = new Size(352, 192); pnlInput.Paint += new PaintEventHandler(pnlInput_Paint); btnFormat = new Button(); btnFormat.Location = new Point(8, 204); btnFormat.Size = new Size(50, 20); btnFormat.Text = "Format"; btnFormat.Click += new System.EventHandler(btnFormat_Click); cbxEditMode = new ComboBox(); cbxEditMode.DropDownStyle = ComboBoxStyle.DropDownList; cbxEditMode.Location = new Point(66, 204); cbxEditMode.Size = new Size(72, 20); cbxEditMode.SelectedIndexChanged += new System.EventHandler(cbxEditMode_SelIndexChg); // Configure the form itself ClientSize = new Size(368, 236); Controls.AddRange(new Control[] { pnlInput, btnFormat, cbxEditMode}); FormBorderStyle = FormBorderStyle.FixedDialog; MaximizeBox = false; Text = "RealtimeLasso"; ResumeLayout(false); // Fill up the editing mode combobox cbxEditMode.Items.Add(InkOverlayEditingMode.Ink); cbxEditMode.Items.Add(InkOverlayEditingMode.Select); cbxEditMode.Items.Add(InkOverlayEditingMode.Delete); // Create a new InkOverlay, using pnlInput for the collection // area inkOverlay = new InkOverlay(pnlInput.Handle); inkOverlay.AutoRedraw = false; inkOverlay.EditingMode = InkOverlayEditingMode.Ink; // Don't want select mode yet fSelectMode = false; strkCurr = null; lasso = null; strksSelected = inkOverlay.Ink.CreateStrokes(); strksLassoed = inkOverlay.Ink.CreateStrokes(); // Set up event handlers to deal with lasso selection inkOverlay.CursorDown += new InkCollectorCursorDownEventHandler( inkOverlay_CursorDown); inkOverlay.SystemGesture += new InkCollectorSystemGestureEventHandler( inkOverlay_SystemGesture); inkOverlay.NewPackets += new InkCollectorNewPacketsEventHandler( inkOverlay_NewPackets); inkOverlay.Stroke += new InkCollectorStrokeEventHandler( inkOverlay_Stroke); // Select the current editing mode in the combobox cbxEditMode.SelectedItem = inkOverlay.EditingMode; // We're now set to go, so turn on tablet input inkOverlay.Enabled = true; } // Handle the click of the format button private void btnFormat_Click(object sender, System.EventArgs e) { // Edit the current drawing attributes using our Format Ink // dialog class FormatInkDlg dlgFormat = new FormatInkDlg(); dlgFormat.DrawingAttributes = inkOverlay.DefaultDrawingAttributes; if (dlgFormat.ShowDialog(this) == DialogResult.OK) { inkOverlay.DefaultDrawingAttributes = dlgFormat.DrawingAttributes; } } // Handle the selection change of the editing mode combobox private void cbxEditMode_SelIndexChg(object sender, System.EventArgs e) { // Clear the current selection UpdateStrokes( inkOverlay.Ink.CreateStrokes(), ref strksSelected); if ((InkOverlayEditingMode)cbxEditMode.SelectedItem == InkOverlayEditingMode.Select) { // We want CursorDown, NewPackets, and Stroke events, but // don't want ink drawn - hence we'll change InkOverlay // into Ink mode but make the ink invisible. Then, we'll // make sure it's not added to the Ink object by cancelling // it in the Stroke event handler. inkOverlay.EditingMode = InkOverlayEditingMode.Ink; inkOverlay.DefaultDrawingAttributes.Transparency = 255; inkOverlay.DynamicRendering = false; inkOverlay.Cursor = System.Windows.Forms.Cursors.Cross; fSelectMode = true; } else { // Restore the InkOverlay to it's previous state fSelectMode = false; inkOverlay.Cursor = null; inkOverlay.DefaultDrawingAttributes.Transparency = 0; inkOverlay.DynamicRendering = true; // Set the current editing mode to the selection chosen // in the combobox inkOverlay.EditingMode = (InkOverlayEditingMode)cbxEditMode.SelectedItem; } } // Handle the pen going down private void inkOverlay_CursorDown(object sender, InkCollectorCursorDownEventArgs e) { // Since this or might become an invisible stroke we're // creating, we'll save it off so it can be removed from // upcoming hit-test results strkCurr = e.Stroke; } // Handle system gestures private void inkOverlay_SystemGesture(object sender, InkCollectorSystemGestureEventArgs e) { // If we're in ink mode and a right-tap or drag occurs, then // switch into our custom select mode if (!fSelectMode && inkOverlay.EditingMode == InkOverlayEditingMode.Ink && (e.Id == SystemGesture.RightDrag e.Id == SystemGesture.RightTap)) { strkCurr.DrawingAttributes.Transparency = 255; cbxEditMode.SelectedItem = InkOverlayEditingMode.Select; } if (fSelectMode) { // Drag will start a lasso, tap will select if (e.Id == SystemGesture.Drag e.Id == SystemGesture.RightDrag) { //TODO: a drag or right-drag inside the bounds of the // current selection (if any) triggers drag-n-drop // Clear the current selection UpdateStrokes( inkOverlay.Ink.CreateStrokes(), ref strksSelected); // Create a new lasso UI object, and clear out the // in-progress lasso results lasso = new LassoUI(); strksLassoed = inkOverlay.Ink.CreateStrokes(); // Start up the lasso UI Point pt = new Point(e.Point.X, e.Point.Y); Graphics g = pnlInput.CreateGraphics(); inkOverlay.Renderer.InkSpaceToPixel(g, ref pt); lasso.Start(g, pt); g.Dispose(); } else if (e.Id == SystemGesture.Tap e.Id == SystemGesture.RightTap) { //TODO: a tap inside the bounds of the current selection // (if any) is a no-op and a right-tap triggers a // context menu // Compute the tap radius (i.e. how close the pen can be // to ink before it's considered hit) Point ptRadius = new Point(nTapDistance, nTapDistance); Graphics g = pnlInput.CreateGraphics(); inkOverlay.Renderer.PixelToInkSpace(g, ref ptRadius); g.Dispose(); // Do the hit test, removing the invisible stroke being // used for the tap itself, and select the result Strokes strks = inkOverlay.Ink.HitTest( e.Point, ptRadius.X); strks.Remove(strkCurr); UpdateStrokes(strks, ref strksSelected); } } } // Handle new packets private void inkOverlay_NewPackets(object sender, InkCollectorNewPacketsEventArgs e) { if (fSelectMode && lasso != null) { Graphics g = pnlInput.CreateGraphics(); // Continue on with the lasso UI bool fUpdated = false; int nPktLength = e.PacketData.Length / e.PacketCount; int nIndex = 0; for (int i = 0; i < e.PacketCount; i++) { Point pt = new Point( e.PacketData[nIndex], e.PacketData[nIndex+1]); inkOverlay.Renderer.InkSpaceToPixel(g, ref pt); fUpdated = lasso.Continue(g, pt); nIndex += nPktLength; } Point [] pts = lasso.Points; if (fUpdated && pts.Length >= 3) { // Convert the points to ink space inkOverlay.Renderer.PixelToInkSpace(g, ref pts); // Perform the in-progress lasso hit-test Strokes strks = inkOverlay.Ink.HitTest(pts, 60); strks.Remove(strkCurr); UpdateStrokes(strks, ref strksLassoed); } g.Dispose(); } } // Handle an ink stroke occurring private void inkOverlay_Stroke(object sender, InkCollectorStrokeEventArgs e) { // Don't need the in-progress stroke anymore strkCurr = null; if (fSelectMode) { // Erase the invisible ink stroke e.Cancel = true; inkOverlay.Ink.DeleteStroke(e.Stroke); if (lasso != null) { // Clear out the in-progress lasso results strksLassoed = inkOverlay.Ink.CreateStrokes(); Point [] pts = lasso.Points; // Erase the lasso UI Rectangle rcBounds = lasso.BoundingRect; lasso = null; pnlInput.Invalidate(rcBounds); if (pts.Length >= 3) { // Convert the points to ink space to perform the // hit-test operation Graphics g = pnlInput.CreateGraphics(); inkOverlay.Renderer.PixelToInkSpace(g, ref pts); g.Dispose(); // Perform the lasso hit-test resulting in a // selection change UpdateStrokes(inkOverlay.Ink.HitTest(pts, 60), ref strksSelected); } } } } // Handle the ink input panel painting private void pnlInput_Paint(object sender, PaintEventArgs e) { // Compute selection ink thickness Point ptThk = new Point(nSelThickness, nSelThickness); inkOverlay.Renderer.PixelToInkSpace(e.Graphics, ref ptThk); Rectangle rcBounds; // Paint each stroke in inkOverlay.Ink foreach (Stroke s in inkOverlay.Ink.Strokes) { // Make sure we're not going to try to paint strokes that // have been deleted if (s.Deleted) { continue; } // Should stroke be drawn selected? if (strksSelected.Contains(s) strksLassoed.Contains(s)) { GetBoundsInPixels(s, true, out rcBounds); if (e.ClipRectangle.IntersectsWith(rcBounds)) { RendererEx.DrawSelected(inkOverlay.Renderer, e.Graphics, s); } } else { // Draw an unselected stroke GetBoundsInPixels(s, false, out rcBounds); if (e.ClipRectangle.IntersectsWith(rcBounds)) { inkOverlay.Renderer.Draw(e.Graphics, s); } } } // Paint any selection bounding box if (strksSelected.Count > 0) { GetBoundsInPixels(strksSelected, true, out rcBounds); if (e.ClipRectangle.IntersectsWith(rcBounds)) { Pen p = new Pen(Color.DarkBlue); p.DashStyle = DashStyle.Dash; e.Graphics.DrawRectangle(p, rcBounds); } } // Paint any in-progress lasso UI if (lasso != null) { lasso.Render(e.Graphics, e.ClipRectangle); } } // Computes the bounds of strokes private void GetBoundsInPixels(Strokes strks, bool fSelected, out Rectangle rcBounds) { // Make sure the strokes collection has anything to measure if (strks == null strks.Count == 0) { rcBounds = Rectangle.Empty; return; } // First, get the bounds (in ink space) rcBounds = strks.GetBoundingBox(); // Convert the bounds to pixels Graphics g = pnlInput.CreateGraphics(); RendererEx.InkSpaceToPixel(inkOverlay.Renderer, g, ref rcBounds); g.Dispose(); // Inflate the bounds by the selection thickness if needed if (fSelected) { rcBounds.Inflate(nSelThickness*2, nSelThickness*2); } } // Computes the bounds of a stroke private void GetBoundsInPixels(Stroke strk, bool fSelected, out Rectangle rcBounds) { GetBoundsInPixels( inkOverlay.Ink.CreateStrokes(new int [] {strk.Id}), fSelected, out rcBounds); } // Copy one stroke set to another - invalidating strokes that are // not common between the two private void UpdateStrokes(Strokes strksNew, ref Strokes strksCurr) { // If we're dealing with selection (rather than lasso), we // should invalidate the bounds of the entire current and new // stroke set so the outline UI is drawn properly if (strksCurr == strksSelected) { if (strksCurr.Count > 0) { Rectangle rcBounds; GetBoundsInPixels(strksCurr, true, out rcBounds); rcBounds.Width++; rcBounds.Height++; pnlInput.Invalidate(rcBounds); } if (strksNew.Count > 0) { Rectangle rcBounds; GetBoundsInPixels(strksNew, true, out rcBounds); rcBounds.Width++; rcBounds.Height++; pnlInput.Invalidate(rcBounds); } strksCurr = strksNew; return; } // See if we need to continue - are the two stroke sets equal? if (strksNew.Count == strksCurr.Count) { bool fEqual = true; foreach (Stroke s in strksNew) { if (!strksCurr.Contains(s)) { fEqual = false; break; } } if (fEqual) { return; } } // We'll use temporary variables since the invalidate calls may // cause the Paint handler to execute while we're in the middle // of this function Strokes s1 = strksCurr; Strokes s2 = strksNew; strksCurr = strksNew; // Invalidate the strokes that are not common foreach (Stroke s in s1) { if (!s2.Contains(s)) { Rectangle rcBounds; GetBoundsInPixels(s, true, out rcBounds); pnlInput.Invalidate(rcBounds); } } foreach (Stroke s in s2) { if (!s1.Contains(s)) { Rectangle rcBounds; GetBoundsInPixels(s, true, out rcBounds); pnlInput.Invalidate(rcBounds); } } // Get the input panel to immediately redraw pnlInput.Update(); } }

This program uses an InkOverlay object rather than an InkControl because it implements its own custom Selection editing mode and the InkControl class has no provision for this. InkOverlay s version of the mode doesn t facilitate the user performing real-time lasso operations very well, so I took the opportunity to also show a little of how a custom Selection mode might be implemented.

The application owns two Strokes collections one used for the currently selected Stroke objects and the other used to track the Stroke objects that are currently enclosed by the lasso in case we want to add any sort of add to selection capability. For example, many keyboard-based applications allow the selection to be extended if the user holds down the Ctrl key, and in fact Windows Journal provides this ability. Because a current selection that we want to preserve may be present while a lasso is occurring, it is prudent to use two separate Strokes collections.

A lasso is started when the application is in its custom Selection mode and either a SystemGesture.Drag or SystemGesture.RightDrag occurs. An instance of the LassoUI class is created and is used until the lasso stroke ends.

This application highlights the performance issues that percentage-based lasso enclosure brings. To see it for yourself, try writing a sentence in the inking area and then lasso the whole thing you ll probably notice how the dots start to lag behind the pen toward the end of the lasso stroke. This is because of all the hit-testing computation and ink rendering going on, even though work was done to try to minimize it.

The UpdateStrokes method reduces the amount of rendering triggered while a lasso is in progress. Its purpose is to copy one Strokes collection to another and invalidate the bounding rectangles of any Stroke objects that are not in common between them. It could just invalidate the whole input area every time a stroke was added or removed from the current lasso selection set, but performance would suffer further because of all the redundant drawing going on.

The ideal implementation of real-time lasso selection should be multithreaded: one thread receives tablet events and computes and renders the lasso polyline while another thread performs the hit-test operation and ink rendering. This is the implementation Windows Journal appears to use (or something similar), but it s a little beyond the scope of this book in its implementation.



Building Tablet PC Applications
Building Tablet PC Applications (Pro-Developer)
ISBN: 0735617236
EAN: 2147483647
Year: 2001
Pages: 73

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net