Using Gestures to Control Tablet Media Player

 Download CD Content

Overview

If you recall from Chapter 16, Creating an MP3 Player, we created an MP3 player that allows the user to write out a command that is recognized and a certain behavior is executed, such as playing or stopping a file. Although it obviously works, it is definitely not an easy solution from a user standpoint, as the need to write the word 'play' to play a file is very tedious. In this chapter, we look at a much better way to control these types of applications by using gestures to control a media player (see Figure 18.1).

click to expand
Figure 18.1: Built-in gestures from Table 18.1, viewed from left to right.

  Note 

The source code for the projects are located on the CD-ROM in the PROJECTS folder. You can either type them in as you go or you can copy the projects from the CD-ROM to your hard drive for editing.

Tablet Gestures

If you have ever used a PalmPilot, you have seen the way a device can take advantage of a type of shorthand called graffiti. The Palm works by allowing the user to write a simple version of complex characters-making entering text an easier objective. The Tablet PC provides a much deeper implementation of a shorthand system known as gestures.

The gestures are available as part of the standard Tablet PC Platform SDK. When you develop an application, the gestures are not enabled by default. Rather, if you want to enable one of the built-in gestures, you must instruct the application that you are doing so. Most gestures have a 'hot point,' which is a distinguishing point in the geometry of the gesture. The hot point can be used to determine where the gesture was performed. The gestures APIs provide the HotPoint property of the Gesture event, making it possible to determine the hot point for a given gesture. Not all gestures have a specific hot point; for these gestures, the starting point is reported as the hot point. Using both Figure 18.1 and Table 18.1, you can see an example of the gestures and the location of the hot point.

Table 18.1: Gestures

Name

Hot Point

Notes

Scratch-out

Starting point

Make minimum three strokes close together

Triangle

Starting point

Don't lift pen while drawing

Square

Starting point

Start at upper-left corner

Star

Starting point

Draw five points

Check

Corner

Draw upward portion 2-4 times larger

Curlicue

Starting point

Angled from lower left to upper right

Double-Curlicue

Starting point

Same as previous but twice

Circle

Starting point

Single stroke starting at top

Double-circle

Starting point

Draw circles overlapping

Left-semicircle

Starting point

Draw from right to left

Right-semicircle

Starting point

Draw from left to right

Caret

Apex

Both sides equal length

Inverted-Caret

Apex

Same as previous but inverted

Chevron-left

Apex

Both sides equal length

Chevron-right

Apex

Both sides equal length

Arrow-up

Arrow head

Limit to two strokes

Arrow-down

Arrow head

Limit to two strokes

Arrow-left

Arrow head

Limit to two strokes

Arrow-right

Arrow head

Limit to two strokes

Up

Starting point

Single upward line (draw fast)

Down

Starting point

Single downward line (draw fast)

Left

Starting point

Single line to left (draw fast)

Right

Starting point

Single line to right (draw fast)

Up-left

Point of change

Single stroke starting with up stroke

Up-right

Point of change

Single stroke starting with up stroke

Down-left

Point of change

Single stroke starting with down stroke

Left-up

Point of change

Single stroke starting with left stroke

Left-down

Point of change

Single stroke starting with left stroke

Right-up

Point of change

Single stroke starting with right stroke

Right-down

Point of change

Single stroke starting with right stroke

Up-down

Point of change

Start with up drawing close together

Down-up

Point of change

Start with down drawing close together

Left-right

Point of change

Start with left drawing close together

Right-left

Point of change

Start with right drawing close together

Up-left-long

Point of change

Start up with left 2x longer

Up-right-long

Point of change

Start up with right 2x longer

Down-left-long

Point of change

Start down with left 2x longer

Down-right-long

Point of change

Start up with right 2x longer

Exclamation

Center of line

Draw dot soon after line

Tap

Starting point

Perform quickly

Double-tap

Starting point

Perform quickly

Building the Media Player

Microsoft provides the Media Player ActiveX control as a simple way to offer media player capabilities. We are going to take advantage of this control as we build an application that can open various movie files and that is controlled by gestures.

To begin this application, create a new Windows Forms application and add an OpenFileDialog to it. You can leave its name as the default given. In the middle of the form, add an InkPicture control, and in the upper right of the form, add a separate one (should look like Figure 18.2) leaving their names as InkPicture1 and InkPicture2.

click to expand
Figure 18.2: Position the controls on the form.

Now, let's add the media player control to the form. Choose Project | Add Reference to display the Add Reference dialog box. Next, choose the COM tab and then locate the Windows Media Player OCX control. You can position the media player control on the form as seen in Figure 18.3. This control should approximately cover the InkPlayer control we placed on the form earlier. We're going to use the InkPlayer control to allow us to annotate on the movie.

click to expand
Figure 18.3: The Media Player OCX positioned on the form.

The next step for this project is to add a Label control and position it above the InkPicture2 control. Change its Text property to read 'Gesture'. The final thing we need is a StatusBar control added to the form. When it's added, it automatically positions itself along the bottom of the form, which is where the status bar is normally placed.

Writing Some Code

Before we get to most of the code, we need to add a reference to the Microsoft Ink API. Therefore, we need to add the following line to the top of the code:

Imports Microsoft.Ink

We'll use the Form_Load event to set several properties very important to our application. First, set the InkPicture1 control to have a transparent background color, remove its border, and then bring it to the front so that it is available for annotating over the video. Here are the three lines of code:

InkPicture1.BackColor = Color.Transparent
InkPicture1.BorderStyle = BorderStyle.None
InkPicture1.BringToFront()

Now, let's turn our attention to the other InkPicture control. This one will be used to handle the gestures for our application, which will control the features such as opening, playing, and pausing a video. First, we'll set its borderstyle and backcolor so that it separates itself from the rest of the interface and is very visible to the user. As we are interested in using the control to capture gestures only, we are going to set its Collection mode to GestureOnly, and then we'll set the gestures that it will try to recognize. Our application is going to use the following gestures:

Square: Stop

UpDown: Pause

ChevronRight: Play

ArrowRight: Volume increase

ArrowLeft: Volume decrease

Circle: Open

Here is the code:

InkPicture2.BorderStyle = BorderStyle.Fixed3D
InkPicture2.BackColor = Color.White
InkPicture2.CollectionMode = CollectionMode.GestureOnly
InkPicture2.SetGestureStatus(ApplicationGesture.Square, True)
InkPicture2.SetGestureStatus(ApplicationGesture.UpDown, True)
InkPicture2.SetGestureStatus(ApplicationGesture.ChevronRight, True)
InkPicture2.SetGestureStatus(ApplicationGesture.ArrowRight, True)
InkPicture2.SetGestureStatus(ApplicationGesture.ArrowLeft, True)
InkPicture2.SetGestureStatus(ApplicationGesture.Circle, True)

The last two things we need to handle are the status bar's Text property, which we set to 'No file loaded.', and the filter for the open dialog box so that we can only open MPEG and AVI videos:

OpenFileDialog1.Filter = "MPEG Video (*.mpg)|*.mpg|Video for Windows (*.avi)|*.avi|All Files (*.*)|*.*"
StatusBar1.Text = "No file loaded."

Now that we have our gestures, we can use the InkPicture2_Gesture event to determine which gesture was written and then perform the appropriate actions. We'll use an If...Then statement at the beginning of the procedure to check the current gesture's confidence. We can test to see if the recognition confidence is strong, and if so, we will use a Case statement to determine which type of gesture was written and instruct the AxMediaPlayer control appropriately. If the gesture confidence is less than strong, we will exit without performing any activities with the AxMediaPlayer. Before we exit, we set the status bar's Text property to 'Command Not Recognized' so that the user realizes that the command they attempted did not perform any function.

We have already discussed the gestures and what each of them will do, so here is the code:

If e.Gestures(0).Confidence = RecognitionConfidence.Strong Then
 Select Case e.Gestures(0).Id
 Case ApplicationGesture.Square
 AxMediaPlayer1.Stop()
 StatusBar1.Text = "Stop"
 Case ApplicationGesture.ChevronRight
 AxMediaPlayer1.Play()
 StatusBar1.Text = "Play"
 Case ApplicationGesture.UpDown
 AxMediaPlayer1.Pause()
 StatusBar1.Text = "Pause"
 Case ApplicationGesture.ArrowLeft
 AxMediaPlayer1.Volume = AxMediaPlayer1.Volume - 1
 StatusBar1.Text = "<- Vol = " & AxMediaPlayer1.Volume.ToString
 Case ApplicationGesture.Circle
 OpenFileDialog1.ShowDialog()
 AxMediaPlayer1.Open(OpenFileDialog1.FileName)
 Case ApplicationGesture.ArrowRight
 AxMediaPlayer1.Volume = AxMediaPlayer1.Volume + 1
 StatusBar1.Text = "-> Vol = " & AxMediaPlayer1.Volume.ToString
 StatusBar1.Text = AxMediaPlayer1.FileName
 End Select
Else
 StatusBar1.Text = "Command Not Recognized"
End If

The application is functional at this time, but we have a few additional things we need to handle. First, we need to change the drawing color for both InkPicture controls, depending on which control the pen is entering. We'll use both MouseEnter events for this aspect of the application:

Private Sub InkPicture2_MouseEnter(ByVal sender As Object, ByVal e As System.EventArgs) Handles InkPicture2.MouseEnter
 InkPicture2.DefaultDrawingAttributes.Color = Color.Red
End Sub

Private Sub InkPicture1_MouseEnter(ByVal sender As Object, ByVal e As System.EventArgs) Handles InkPicture1.MouseEnter
 InkPicture1.DefaultDrawingAttributes.Color = Color.Blue
End Sub

The final thing we need to add to the application is a way to pause the AxMediaPlayer currently playing file so that we can annotate a given frame. We can use the MouseDown event procedure for InkPicture1 to test the current filename. If there is a filename, we then test to see if the media player is currently playing. If it is, we pause the player so that we can proceed to annotate it. Lastly, we set the status bar's Text property to 'Paused for annotation.'

Here is the code:

Private Sub InkPicture1_MouseDown(ByVal sender As Object, ByVal e As System.Windows.Forms.MouseEventArgs) Handles InkPicture1.MouseDown
 If axMediaPlayer1.FileName "" and AxMediaPlayer1.PlayState = MPPlayStateConstants.mpPlaying Then
 AxMediaPlayer1.Pause()
 StatusBar1.Text = "Paused for annotation."
 End If
End Sub

You can test the application to see if it works. Figure 18.4 displays an open gesture with an OpenFileDialog and Figure 18.5 shows a video being annotated.

click to expand
Figure 18.4: A file being opened.

click to expand
Figure 18.5: A video file being annotated.

Summary

In this chapter, we created a video player that was controlled using gestures. The application also used an additional InkPicture control to allow us to annotate over the content of the video. In Chapter 19, Getting Started with Microsoft Agent, we build our first speech-enabled application.



Developing Tablet PC Applications
Developing Tablet PC Applications (Charles River Media Programming)
ISBN: 1584502525
EAN: 2147483647
Year: 2003
Pages: 191

Flylib.com © 2008-2020.
If you may any questions please contact us: flylib@qtcs.net