The System.Net Namespace | Professional VB 2005 with .NET 3.0 (Programmer to Programmer)

Most of the functionality used when writing network applications is contained within the System.Net and System.Net.Sockets namespaces. The main classes in these namespaces that this chapter covers are as follows:

WebRequest and WebResponse, and their subclasses, including the new FtpWebRequest
WebClient, the simplified WebRequest for common scenarios
HttpListener, the new capability to create your own Web server

Tip

Additional classes, methods, properties, and events have been added to the System.Net and System.Net.Sockets namespaces in the .NET Framework 2.0. See the updated reference for these namespaces at http://msdn2.microsoft.com/library/system.net.aspx as of this writing.

Web Requests (and Responses)

When most people think of network programming these days, they’re really thinking of communication via a Web server or client, so it shouldn’t be surprising that a set of classes is needed for this communication. In this case, it is the abstract WebRequest class and the associated WebResponse. These two classes represent the concept of a request/response communication with a Web server or similar server. As these are abstract classes, that is, MustInherit classes, they cannot be created by themselves. Instead, you create the subclasses of WebRequest that are optimized for specific types of communication.

The most important properties and methods of the WebRequest class are described in the following table:

Open table as spreadsheet

Member	Description
Create	Method used to create a specific type of WebRequest. This method uses the URL (either as a string or as an Uri class) passed to identify and create a subclass of WebRequest.
GetRequestStream	Method that allows access to the outgoing request. This enables you to include additional information, such as POST data, to the request before sending.
GetResponse	Method used to perform the request and retrieve the corresponding WebResponse
Credentials	Property that enables you to set the user ID and password for the request if they are needed to perform the request
Headers	Property that enables you to change or add to the headers for the request
Method	Property used to identify the action for the request, such as GET or POST. The list of available methods is specific to each type of server.
Proxy	Property that enables you to identify a proxy server for the communication if needed. Note that you generally don’t need to set this property, as Visual Basic 2005 detects the settings for Internet Explorer and uses them by default.
Timeout	Property that enables you to define the time permitted for the request before you “give up” on the server

Each subclass of WebRequest supports these methods, providing a very consistent programming model for communication with a variety of server types. The basic model for working with any of the subclasses of WebRequest can be written in the following pseudocode:

  Declare variables as either WebRequest and WebResponse, or the specific child classes Create the variable based on the URL Make any changes to the Request object you may need Use the GetResponse method to retrieve the response from the server Get the Stream from the WebResponse Do something with the Stream

If you decide to change the protocol (for example, from HTTP to a file-based protocol), the only thing that needs to change is the URL used to retrieve the object.

Working with FileWebRequest and HttpWebRequest

The first two types of WebRequest that became available were FileWebRequest and HttpWebRequest. FileWebRequest is used less frequently; it represents a request to a local file, using the “file://” URL format. You may have seen this type of request if you attempted to open a local file using your Web browser, such as Internet Explorer, Firefox, or Navigator. Generally, however, the subclass most developers will use is HttpWebRequest. This class enables you to make HTTP requests to a Web server without requiring a browser. This could enable you to communicate with a Web server, or, using the time-honored tradition of screen scraping, to retrieve data available on the Web.

One hurdle many developers encounter when first working with HttpWebRequest is that there is no available constructor. Instead, you must use the WebRequest.Create method (or the Create method of your desired subclass) to create new instances of any of the subclasses. This method uses the URL requested to create the appropriate subtype of WebRequest. For example, this would create a new HttpWebRequest:

  Dim req As HttpWebRequest = WebRequest.Create("http://msdn.microsoft.com")

Note that if you have Option Strict turned on (and you should), the preceding code produces an error. Instead, you should explicitly cast the return value of Create to the desired type:

  Dim req As HttpWebRequest = _   DirectCast(WebRequest.Create("http://msdn.microsoft.com"), _   System.Net.HttpWebRequest)

Putting It Together

In order to demonstrate how to use WebRequest/WebResponse, the following example shows how to wrap a Web call into a Visual Basic class. In this case, we’ll wrap Google’s define: keyword, which enables you to retrieve a set of definitions for a word (for example, http://www.google.com/search?q=define%3A+isbn), and then use that in a sample application (see Figure 29-1.)

image from book
Figure 29-1

Create a new Windows application. The example project is named DefinePad.
Add a new class to the project. This will hold the actual WebRequest code. Call it GoogleClient.
Add a reference to the System.Web.DLL, as you will need access to some of its functionality later.

In the GoogleClient.vb file, add Imports statements to make the coding a little briefer:

  Imports System.IO Imports System.Net Imports System.Web Imports System.Collections.Generic

The main function in GoogleClient is a Define function that returns an array of strings. Each string will be one definition returned by Google:

  Public Function Define(ByVal word As String) As String()     Dim req As HttpWebRequest = Nothing     Dim resp As HttpWebResponse     Dim query As String     Dim result As New List(Of String)     query = "http://www.google.com/search?q=define%3A" & _       HttpUtility.UrlEncode(word)     Try         req = DirectCast(WebRequest.Create(query), HttpWebRequest)         With req             .Method = "GET"             resp = req.GetResponse             If resp.StatusCode = HttpStatusCode.OK Then                 ParseResponse(resp.GetResponseStream, result)             Else                 MessageBox.Show("Error calling definition service")             End If         End With     Catch ex As Exception       End Try     Return result.ToArray() End Function

The first task is to guarantee that no invalid characters appear in the query string when you send the request, such as a space, accented character, or other non-ASCII character. The System.Web.HttpUtility class has a number of handy shared methods for encoding strings, including the UrlEncode method. This replaces characters with a safe representation of the character that looks like %value, where the value is the Unicode code for the character. For example, in the definition of the preceding query variable, the %3A is actually the colon character (“:”) that has been encoded. Anytime you retrieve a URL based on user input, encode it because there is no guarantee the resulting URL is safe to send.
Once the query is ready, create the WebRequest. As the URL is for an HTTP resource, an HttpWebRequest is created. While the default method for WebRequest is a GET, it’s still good practice to set it. You’ll create the ParseResponse method shortly to process the stream returned from the server.
Also noteworthy in this piece of code is the return value for this method, and how it is created. In order to return arrays of a specific type (rather than return actual collections from a method), you must either know the actual size to initialize the array or use the new List generic type or the older ArrayList. These classes behave like the Visual Basic 6.0 Collection class, which enables you to add items, and it grows as needed. They also have a handy method that allows you to convert the array into an array of any type; you can see this in the return statement. The ArrayList requires you to do a bit more work. If you want to use an ArrayList for this method, then you must identify the type of array you’d like to return. The resulting return statement would look like this using an ArrayList:
```
  Return result.ToArray(GetType(String)) 
```

The ProcessRequest method parses the stream returned from the server and converts it into an array of items. Note that this is slightly simplified: In a real application, you would likely want to return an array of objects, where each object provides access to the definition and the URL of the site providing it:

  Private Sub ParseResponse(ByVal input As System.IO.Stream, _   ByRef output As List(Of String))         'definitions are in a block beginning with <p>Definitions for...         'then are marked with <li> tags         'yes, I should use Regular Expressions for this         'this format will also likely change in the future.         Dim reader As New StreamReader(input)         Dim work As String = reader.ReadToEnd         Dim blockStart As String = "<p>Definitions of"         Dim pos As Integer = work.IndexOf(blockStart)         Dim posEnd As Integer         Dim temp As String     Do         pos = work.IndexOf("<li>", pos + 1)         If pos > 0 Then             posEnd = work.IndexOf("<br>", pos)             temp = work.Substring(pos + 4, posEnd - pos - 4)             output.Add(ParseDefinition(temp))             pos = posEnd + 1         End If     Loop While pos > 0 End Sub

The code is fairly simple, using the time-honored tradition of screen scraping - processing the HTML of a page to find the section you need and then removing the HTML to produce the result.

The last part of the GoogleClient class is the ParseDefinition method that cleans up the definition, removing the link and other HTML tags:

  Private Function ParseDefinition(ByVal input As String) As String     Dim result As String = ""         Dim lineBreak As Integer         lineBreak = input.IndexOf("<br>")         If lineBreak > 0 Then             result = input.Substring(0, input.IndexOf("<br>"))         Else             result = input         End If         Return result.Trim End Function

Now, with class in hand, you can create a client to use it. In this case, you’ll create a simple text editor that adds the capability to retrieve definitions for words. Go back to the Form created for the application, and add controls as shown in Figure 29-2.

Figure 29-2

The user interface for DefinePad is simple: a TextBox and a ContextMenuStrip.

Open table as spreadsheet

Control	Property	Value
TextBox	Name	TextField
	Multiline	True
	Dock	Fill
	ContextMenuStrip	DefinitionMenu
ContextMenuStrip	Name	DefinitionMenu

The only code in the Form is for the Opening event of the ContextMenuStrip. Here, you add the definitions to the menu. Add the following code to the handler for the Opening event:

  Private Sub DefinitionMenu_Opening(ByVal sender As Object, _   ByVal e As System.ComponentModel.CancelEventArgs) _   Handles DefinitionMenu.Opening     Dim svc As New GoogleClient     Dim definitions() As String     Dim definitionCount As Integer     DefinitionMenu.Items.Clear()     Try         'define the currently selected word         If TextField.SelectionLength > 0 Then             definitions = svc.Define(TextField.SelectedText)             'build context menu of returned definitions             definitionCount = definitions.Length             If definitionCount > 6 Then                 definitionCount = 6             ElseIf definitionCount = 0 Then                 'we can't do anymore, so exit                 Dim item As New ToolStripButton                 item.Text = "Sorry, no definitions available"                 DefinitionMenu.Items.Add(item)                 Exit Sub             End If             For i As Integer = 1 To definitionCount                 Dim item As New ToolStripButton                 item.Text = definitions(i)                 DefinitionMenu.Items.Add(item)             Next         End If     Catch ex As Exception         MessageBox.Show(ex.Message, "Error getting definitions", _             MessageBoxButtons.OK, MessageBoxIcon.Error)     End Try End Sub

The bulk of the code in this event is to limit the number of items displayed in the menu. The actual functional part of the routine is the call to the Define method of the GoogleClient. If you trace through the code as you run, you’ll see the WebRequest generated, the call made, and the resulting response stream parsed into the individual items as desired. Finally, you can use the returned list to create a set of menu items (that don’t actually do anything), and display the “menu.” Clicking on any definition closes the menu.
To test the application, run it. Type or copy some text into the text box, select a word, and right-click on it. After a brief pause, you should see the definitions for the word (see Figure 29-3 for definitions of “developer”).

Figure 29-3

While they are not as “sexy” as Web Services, using this technique (WebRequest, screen scraping of the resulting HTML) can provide your applications with access to a great deal of the functionality of the Internet.

Working with FtpWebRequest

One of the new sets of classes added to the System.Net namespace in Visual Basic 2005 is another type of Web request - the FtpWebRequest. This class, and the related FtpWebResponse, is used to communicate with FTP servers. While the HttpWebRequest/Response can be used for simple file uploading and retrieving, the FtpWebRequest adds the capability to browse or create directories, delete files, and more. The following table describes some of the added functionality in the FtpWebRequest:

Open table as spreadsheet

Member	Description
Abort	Used when performing an asynchronous operation. This command terminates the current operation.
Abort	Used when performing an asynchronous operation. This command terminates the current operation.
Binary	A Boolean value that determines whether the data transfer should be treated as binary or text. Set to true when you are transferring a binary file, and text otherwise.
Method	While not new, the behavior of this method is quite important with the FtpWebRequest as it defines the action to perform. See the section below on WebRequestMethods.Ftp that defines the possible values.
Passive	Boolean value that determines how the client and server should communicate. If this is set to true, the server does not initiate communication back to the client. Instead, it waits until the client initiates the communication. This is typically needed when communicating through a firewall that might not allow the server to open a connection to the client machine.

WebRequestMethods.Ftp

As described above, the actual request made by the FtpWebRequest is identified by the Method property. This is a string property that can be set to any value recognized by your FTP server, but you will often want to set it to one of the values in the WebRequestMethods.Ftp structure:

Open table as spreadsheet

Field	Description
AppendFile	Adds content to an existing file
DeleteFile	Deletes a file from the server (if you have permission)
DownloadFile	Retrieves a file from the FTP server
GetDateTimeStamp	Gets the date and time the file was last modified
GetFileSize	Gets the size of the file on the FTP server
ListDirectory	Gets the file and directory names for a directory on the FTP server. The data returned is a list of the files, each on a line (that is, separated by CRLF characters). This method doesn’t provide an easy way to determine which of the items returned are directories or files.
ListDirectoryDetails	Gets the file and directory information for a directory on the FTP server. This method returns a good deal of information about each item, including attributes, permissions, date of last modification, and size. Just as with the ListDirectory method, each file’s (or directory’s) information is on a single line.
MakeDirectory	Creates a directory on the server
PrintWorkingDirectory	Gets the current path on the FTP server
RemoveDirectory	Removes a directory from the server (if you have permission)
UploadFile	Uploads a file to the FTP server
UploadFileWithUniqueName	Similar to UploadFile, but this method ensures that the new file has a unique filename. This is great when you allow the user to upload files but don’t want possible name collisions to happen, or if you don’t really care what name the file is (e.g., when the file contents just need processing but not saving).

Creating an FTP Client

In order to demonstrate using the FtpWebRequest, the next section covers how to create a simple FTP server browser. The application will enable you to connect to a server, browse the available files, and download files (see Figure 29-4.)

image from book
Figure 29-4

Even though this application is a Windows Forms application, we separate the FTP handling to a class for use in other applications:

Create a new Windows application called FTP Browser.
Before creating the user interface, define the class that will provide the functionality. Add a new class to the project called FtpClient.vb. This class will be used to create wrapper functionality to make working with FtpWebRequest easier. First, add the Imports statements for later use:
```
  Imports System.IO Imports System.Net Imports System.Text Imports System.Collections.Generic 
```

Add two properties to the class. This is for the user ID and password that will be used by the FtpClient:

  Private _user As String Private _pwd As String Public Property UserId() As String     Get         Return _user     End Get     Set(ByVal value As String)         _user = value     End Set End Property Public Property Password() As String     Get         Return _pwd     End Get     Set(ByVal value As String)         _pwd = value     End Set End Property

The Form will use two methods, GetDirectories and GetFiles. These two methods are basically the same:

  Public Function GetDirectories(ByVal url As String) As String()     'line should look like:     'dr-xr-xr-x   1 owner    group     ' 0 Nov 25  2002 bussys"     ' if the first character is a 'd', it's a directory     Return GetDirectoryEntries(url, "d") End Function Public Function GetFiles(ByVal url As String) As String()     'line should look like:     '-r-xr-xr-x   1 owner    group     ' 1715 May 20  1996 readme.txt     ' if the first character is a '-', it's a file     Return GetDirectoryEntries(url, "-") End Function

Obviously, both GetDirectories and GetFiles simply return the result of another helper routine, GetDirectoryEntries. The only difference between the information returned for a file and a directory is that directories have the directory attribute set to d, whereas files have a blank (-) in that position:

  Private Function GetDirectoryEntries(ByVal url As String, _     ByVal directoryAttribute As String) As String()     Dim result As New List(Of String)     Dim str As Stream = Nothing     Dim temp As String     Dim words() As String     Dim splitChars() As Char = {' 'c}     DoFtpRequest(url, _         WebRequestMethods.Ftp.ListDirectoryDetails, _         False, str)     Try         Using reader As StreamReader = New StreamReader(str)             Do                 temp = reader.ReadLine                 If temp <> Nothing Then                     'split into component parts                     words = temp.Split(splitChars, _                       StringSplitOptions.RemoveEmptyEntries)                     If words(0).StartsWith(directoryAttribute) Then                         result.Add(words(8))                     End If                 End If             Loop While temp <> Nothing         End Using     Catch ex As Exception         MessageBox.Show(ex.Message, "Error getting files from " & url)     End Try     Return result.ToArray() End Function

The GetDirectoryEntries method uses another helper method you’ll create shortly to execute the WebRequestMethods.Ftp.ListDirectoryDetails method on the FTP server. This method returns the resulting response stream in the str parameter. The code then loops through the returned content. Each of the directory entries appears on a separate line, so ReadLine is perfect here. The line is split on spaces, and then added to the return value if it has the desired value for the first character (that represents it if it’s a directory or a file).

The GetDirectoryEntries method calls a helper method that does the actual FtpWebRequest. This method returns the resulting stream by way of a ByRef parameter:

  Private Function DoFtpRequest(ByVal url As String, _   ByVal method As String, ByVal useBinary As Boolean, _   ByRef data As Stream) As FtpStatusCode     Dim result As FtpStatusCode     Dim req As FtpWebRequest     Dim resp As FtpWebResponse     Dim creds As New NetworkCredential(UserId, Password)     req = DirectCast(WebRequest.Create(url), FtpWebRequest)     With req         .Credentials = creds         .UseBinary = useBinary         .UsePassive = True         .KeepAlive = True         'make initial connection         .Method = method         resp = .GetResponse()         data = resp.GetResponseStream         result = resp.StatusCode     End With     Return result End Function

The appropriate type of WebRequest is created, the properties are set, and the final request is sent.

With the class created, we can move our attention back to the user interface. Return to the Form and add MenuStrip and SplitContainer controls. You can leave the names and other properties of these controls at their defaults. Create three items under the File menu: Connect, Download, and Exit. You may also want to add an ImageList control, and populate it with appropriate graphics for open and closed folders. The following table lists the properties set on the ImageList in the sample project:

Open table as spreadsheet

Property	Value
TransparentColor	Transparent
Images − open	Used the OpenFold.ico graphic from the Visual Studio 2005 Image Library (located in the icons\Win9x directory)
Images − closed	Used the ClsdFold.ico graphic from the Visual Studio 2005 Image Library (located in the icons\Win9x directory)

Add a TreeView control to the left side of the SplitContainer, and a ListView to the right side. Set the properties as shown in the following tables:

TreeView

Open table as spreadsheet

Property	Value
Name	DirectoryTree
Dock	Fill
PathSeparator	/
ImageList	The name of your ImageList control
SelectedImageKey	The open image’s name
ImageKey	The closed image’s name

ListView

Open table as spreadsheet

Property	Value
Name	FileList
Dock	Fill
MultiSelect	False
View	List

Open the code view for the form and add a few private variables to the Form class:

  Private ftp As New FtpClient Private baseUrl As String Private downloadPath As String

Next, add a handler for the Form_Load event; this will initialize the TreeView and FtpClient objects:

  Private Sub MainForm_Load(ByVal sender As Object, _   ByVal e As System.EventArgs) Handles Me.Load     'initialize form     With Me.DirectoryTree         .Nodes.Add("/")     End With     'initialize ftp client     With ftp         .UserId = My.Settings.user         .Password = My.Settings.email     End With     downloadPath = My.Settings.downloadPath End Sub

Notice the calls to My.Settings when initializing the FtpClient. The Settings collection is available to the My object when you have created settings values in the My Project dialog. Open the Solution Explorer and double-click My Project. Select the Settings tab and add the three values (see Figure 29-5).

Figure 29-5

Now return to adding the code to the form. Enable connecting to the FTP server and retrieving the initial list of directories to add to the TreeView. Add this to the Connect menu item:

  Private Sub ConnectToolStripMenuItem_Click(ByVal sender As System.Object, _   ByVal e As System.EventArgs) Handles ConnectToolStripMenuItem.Click     'makes a new connection to an FTP server     baseUrl = InputBox("Enter FTP site to open", "FTP Browser", __       "ftp://ftp.microsoft.com")     Me.DirectoryTree.Nodes.Clear()     'add the base node     Me.DirectoryTree.Nodes.Add(baseUrl)     AddNodes(Me.DirectoryTree.Nodes(0), baseUrl) End Sub

The event prompts the user for the address of the FTP server to connect with, and then adds it to the TreeView via a helper subroutine, AddNodes:

  Private Sub AddNodes(ByVal parent As TreeNode, ByVal url As String)     Dim dirs() As String     Me.Cursor = Cursors.WaitCursor     dirs = ftp.GetDirectories(url)     For Each dir As String In dirs         With parent.Nodes.Add(dir)             .Nodes.Add("NoNodeHere", "empty")         End With     Next     Me.Cursor = Cursors.Default End Sub

The AddNodes method retrieves the list of directories for the selected URL. In this, the first call for an FTP server, it retrieves the root directory. Later, the same method is used to retrieve subdirectories by requesting a URL containing the full path. Notice the addition of a fake node to each of the directories (the “NoNodeHere” item). This ensures that each of the added directories has the plus symbol next to it in the TreeView, implying that there is content below it. We will remove the empty node later when we request the actual subdirectories.

Initially, each of the directories is empty except for the “NoNodeHere” item. You can use the presence of this node to determine whether you need to request subdirectories. If it still exists, call AddNodes when the user attempts to expand the TreeView node:

  Private Sub DirectoryTree_BeforeExpand(ByVal sender As Object, _   ByVal e As System.Windows.Forms.TreeViewCancelEventArgs) _   Handles DirectoryTree.BeforeExpand     Dim thisNode As TreeNode     thisNode = e.Node     If thisNode.Nodes.ContainsKey("NoNodeHere") Then         'we haven't retrieved this nodes children yet         'remove the empty node         thisNode.Nodes("NoNodeHere").Remove()         'get the real children now         AddNodes(thisNode, baseUrl + thisNode.FullPath)     End If End Sub

If “NoNodeHere” still exists, remove it, and then call the AddNodes method again, passing this node and its path. This calls the FTP server again, retrieving the child directories of the selected directory. You perform this before the node is expanded, so before the user can see the “NoNodeHere” node. If the subdirectories have already been requested, then the “NoNodeHere” node won’t be in the TreeView anymore, and so the code to call the FTP server won’t be called again.

After the node has been expanded, it is selected. Now retrieve the list of files in that directory to display in the ListView control:

  Private Sub DirectoryTree_AfterSelect(ByVal sender As System.Object, _   ByVal e As System.Windows.Forms.TreeViewEventArgs) _   Handles DirectoryTree.AfterSelect     Dim thisNode As TreeNode     Dim files() As String     thisNode = e.Node     'we don't want to do this for the root node     If thisNode.Text <> "/" Then         'get files for this directory         Me.Cursor = Cursors.WaitCursor         'clear the current list         Me.FileList.Items.Clear()         files = ftp.GetFiles(baseUrl + thisNode.FullPath)         For Each fil As String In files             Me.FileList.Items.Add(fil)         Next         Me.Cursor = Cursors.Default     End If End Sub

This code is fairly simple. First, the ListView is cleared of existing files. Then, the FtpClient is called, retrieving the list of files in the selected directory. These are then added to the ListView.

You should now be able to run the application and browse an FTP server (see Figure 29-6). Note that because we haven’t added any credentials, only anonymous FTP servers can be browsed. If you want to connect to FTP servers that require authentication, set UserId and Password as appropriate, or query them from the user.

Figure 29-6
For a few finishing touches, we’ll set the Download menu item to be usable only if a file is selected, and add the code for the Exit menu item. Set the initial value for Enabled to false for the Download menu item, and add the following code to the handler for the ListView’s SelectedIndexChanged event:
```
  Private Sub FileList_SelectedIndexChanged(ByVal sender As System.Object, _   ByVal e As System.EventArgs) Handles FileList.SelectedIndexChanged     Me.DownloadToolStripMenuItem.Enabled = _       CBool(Me.FileList.SelectedItems.Count) End Sub 
```
When there is a selected item, the Count will be > 0, which converts to True. If 0 items are selected, then this will be False.

The code for the Exit menu item is simple enough:

  Private Sub ExitToolStripMenuItem_Click(ByVal sender As System.Object, _   ByVal e As System.EventArgs) Handles ExitToolStripMenuItem.Click     Me.Close() End Sub

Finally, add the code for the Download menu item:

  Private Sub DownloadToolStripMenuItem_Click(ByVal sender As Object, _   ByVal e As System.EventArgs) Handles DownloadToolStripMenuItem.Click     'download currently selected file (but only if something is selected)     ftp.DownloadFile(baseUrl & _       Me.DirectoryTree.SelectedNode.FullPath & _       "/" & Me.FileList.SelectedItems(0).Text, _       downloadPath & Me.FileList.SelectedItems(0).Text) End Sub

Obviously, we need to add the DownloadFile method to the FtpClient class. Here’s the code:

  Public Sub DownloadFile(ByVal url As String, _   ByVal destination As String)     Dim str As Stream = Nothing     DoFtpRequest(url, _         WebRequestMethods.Ftp.DownloadFile, _         True, _         str)     Using reader As StreamReader = New StreamReader(str)         Using writer As StreamWriter = _           New StreamWriter(File.OpenWrite(destination))             writer.Write(reader.ReadToEnd)         End Using     End Using End Sub

Note the repeat use of the DoFtpRequest method. However, this time, we pass True for the binary, just in case the file we’re transferring is not a text-based file. Using the new Using block, we create a new StreamReader around the output stream of the response, and a new StreamWriter to a local output file. By using the Using block, we guarantee that the associated readers, writers, and streams will all be closed when we’re done using them. The Using block is functionally identical to the following .NET Framework 1.1 code:

  Dim reader As StreamReader Try     reader = New StreamReader(str)     ... Finally     reader.Flush()     reader.Close()     reader = Nothing End Try

Now you can test out the new download code. Run the application again, connect to an FTP server, select a file, and then select Download from the File menu. You should see the newly created file appear in your download directory (see Figure 29-7).

image from book
Figure 29-7

While creating a full-blown FTP client would be a fair bit more work, it is hoped that you can see how the functionality of the FtpWebRequest and FtpWebResponse classes makes communicating with an FTP server much easier than before, not to mention writing the core functionality yourself using sockets.

Simplifying Common Web Requests with WebClient

Tip

When I first saw a demo of WebRequest in early 2000, I was delighted. Here was the capability to easily access Internet resources waiting for me to play with. However, one of the other attendees of the demo asked, “Why is that so difficult? You need to do so much to get it to work.” The next time I saw the same WebRequest demo, the presenter concluded with, “For those of you doing the common scenarios, we have an even easier way.” He then went on to show us how to use System.Net.WebClient.

For those times when you just want to send a GET or POST request and download a file or the resulting data, you can forget about WebRequest/WebResponse. WebClient abstracts away all of the little details of making Web requests, and makes it amazingly easy to grab data from the Web. The important methods and properties of the WebClient class are described in the following table:

Open table as spreadsheet

Member	Description
DownloadData	Returns a Byte array of data from the server. This is essentially the same as if you had called the Read method on the stream returned from GetResponse?Stream. You could then save this to a binary file, or convert to text using an appropriate encoding. However, see DownloadFile and DownloadString for two easier ways of performing these tasks.
DownloadFile	Retrieves a file from the server and saves it locally
DownloadString	Returns a block of text from the server
OpenRead	Returns a stream providing data from the server. This is essentially the same stream returned from the call to GetResponseStream.
OpenWrite	Returns a stream you can use to write to the server. This is essentially the same as creating a WebRequest and writing to the GetResponse stream.
UploadData	Sends a Byte array of data to the server. See UploadFile, UploadString, and UploadValues for easier ways of performing this task.
UploadFile	Sends a local file to the server for processing
UploadString	POSTs a string to the server. This is very handy when you are simulating HTML form input.
UploadValues	Sends a set of name/value pairs to the server. This is similar to the format used by QueryString values, and this method is quite useful for simulating HTML input.
BaseAddress	The base URL the WebClient will access - for example: http://www.example.com.
Credentials	Credentials that will be used when performing any request. You can create a new NetworkCredential to use this, or alternately, set the UseDefaultCredentials property to true to use the credentials the user has logged in with.
Headers	Collection of headers that will be used for the request
Proxy	Overrides the proxy settings from Internet Explorer if set. By default, you should never need to set this property, as the normal proxy settings are chosen by default.
QueryString	Collection of name/value pairs that will be sent with the request. This represents the values after the '?’ on a request.
ResponseHeaders	Collection of headers returned by the server after the request is completed

All of the DownloadX and UploadX methods also support an asynchronous version of the method, called DownloadXAsync, such as DownloadFileAsync or UploadValuesAsync. These methods perform the actual request on a background thread, and fire an event when the task is completed. If your application has some form of user interface, such as a Form, you should generally use these methods to keep your application responsive.

As WebClient uses the WebRequest classes to actually perform its magic, it can greatly simplify network coding. For example, just replace the code used in the WebRequest sample created earlier.

Before:

  Public Function Define(ByVal word As String) As String()     Dim req As HttpWebRequest = Nothing     Dim resp As HttpWebResponse     Dim query As String     Dim result As New List(Of String)     query = "http://www.google.com/search?q=define%3A" & _       HttpUtility.UrlEncode(word)     Try         req = DirectCast(WebRequest.Create(query), HttpWebRequest)         With req             .Method = "GET"             resp = req.GetResponse             If resp.StatusCode = HttpStatusCode.OK Then                 ParseResponse(resp.GetResponseStream, result)             Else                 MessageBox.Show("Error calling definition service")             End If         End With     Catch ex As Exception     End Try     Return result.ToArray() End Function

After:

  Public Function Define(ByVal word As String) As String()     Dim client As New WebClient     Dim query As String     Dim result As New List(Of String)     query = "http://www.google.com/search?q=define%3A" & _       HttpUtility.UrlEncode(word)     Try         result = ParseResponse(client.DownloadString(query))     Catch ex As Exception     End Try     Return result.ToArray() End Function

WebClient avoids all of the stream handling required for WebRequest, but you should still know how WebRequest operates, as this knowledge is directly applicable to WebClient.

Creating Your Own Web Server with HttpListener

One feature of the .NET Framework 2.0 that got me extremely excited was the new HttpListener class (and related classes). This class enables you to very easily create your own Web server. While it likely wouldn’t be a replacement for IIS, it can enable you to add Web server functionality to other applications. For example, rather than use remoting or MSMQ to create a communication channel between two applications, why not use HTTP? Each instance could host its own little Web server, and then you could use HttpWebRequest or WebClient to communicate between them. Alternately, many applications and hardware devices now provide a built-in Web application, enabling you to configure the device or application via a Web browser.

Tip

The fine print: The HttpListener class relies on the new Http.sys functionality built into IIS 6.0, so you must be using an operating system that includes http.sys as a systemwide HTTP service. Only Windows Server 2003, Windows XP SP2, and Vista (and future versions of the operating system) include this functionality - yet another reason to upgrade, and to install Service Packs. Future operating systems should all provide this functionality.

HttpListener works by registering one or more prefixes with http.sys. Once this is done, any requests intercepted by the HTTP subsystem are passed on to the registered listener. An HttpListenerContext object is created and passed to your listener. This context contains properties for the Request and Response objects, just as the Context object in ASP.NET does. Again, similar to Web applications, you read the request from the Request property, and write the response to the Response property. Closing the response sends the resulting page to the user’s browser. The following table describes the important members of HttpListener:

Open table as spreadsheet

Member	Description
Abort	Shuts down the server without finishing any existing requests
Close	Shuts down the server after finishing handling any existing requests
Start	Starts the listener receiving requests
Stop	Stops the listener from receiving requests
IsListening	Property that determines whether the listener is currently receiving requests
Prefixes	Collection of the types of requests that this listener will respond to. These are the “left-hand side” of the URL, such as “http://localhost:8080/” or “http://serverName:1234/vrootName/”. Note that you must end the prefix in a slash or you will receive a runtime error. If you have IIS installed on the same server, you can use port 80, as long as a vroot with the same name is not already defined by IIS.

Creating Your Web Server

To demonstrate using HttpListener, this example creates a Windows Service to host its functionality. This could simulate a management or monitoring interface to a Windows Service that would enable authenticated individuals to use the Windows Service remotely or to get other information out of it. Here are the steps:

Create a new Windows Service application called MiniServer. The server won’t do much on its own, but it will host an HttpListener.
From the Components section of the Toolbox, add a BackgroundWorker component named BackgroundWork. The other properties can remain at their defaults. This BackgroundWorker will be used to process HTTP requests on a background thread, simplifying the handling of the threads.
Switch to code view for the service. Add the Imports statements you need to the top of the file:
```
  Imports System.Net Imports System.IO Imports System.Web Imports System.Text 
```
Add the private members to the class. In addition, add a constant to identify the port number the service will use for listening. Select a port that currently isn’t in use on your computer. The example uses 9090:
```
  Private listener As New HttpListener() Private theService As String  Private Const PORT As Integer = 9090 
```

In the OnStart method, set up the list of prefixes to which the server will respond. This can be as simple as adding a port address to the URL, or include specific vroots. The sample includes examples of each:

  Protected Overrides Sub OnStart(ByVal args() As String)     Dim machineName As String     machineName = System.Environment.MachineName     theService = HttpUtility.UrlEncode(Me.ServiceName)     Me.EventLog.WriteEntry("Service Name: " & Me.ServiceName)     With listener         .Prefixes.Add(String.Format("http://{0}:{1}/", _           "localhost", PORT.ToString))         .Prefixes.Add(String.Format("http://{0}:{1}/", _           machineName, PORT.ToString))         .Prefixes.Add(String.Format("http://{0}/{1}/", _           "localhost", theService))         .Prefixes.Add(String.Format("http://{0}/{1}/", _           machineName, theService))         .Start()     End With     'start up the background thread     Me.BackgroundWork.RunWorkerAsync() End Sub

In this case, the server will respond to a prefix in any of the formats (the sample computer is called Tantalus):

  http://localhost:9090/ http://tantalus:9090/ http://localhost/sampleservice/ http://tantalus/sampleservice/

Important

There is one important point to keep in mind as you add prefixes: They must end in a slash (/) character. If not, you will get a runtime error when the listener attempts to add that prefix.

If you already have a Web server listening on port 80, such as IIS, do not include the last two prefixes. Only a single application can listen to each port, so this service will not be able to start if the other service is already monitoring port 80.

After initializing the Prefixes collection, calling the Start method binds the listener to the appropriate ports and vroots and starts it accepting requests. However, we don’t want to actually receive the requests in the OnStart handler. Remember that the service doesn’t actually start until after this method has completed, so having a lot of processing in the OnStart will actually prevent the service from completing. Therefore, we use another of the new features of Visual Basic 2005, the BackgroundWorker component, to handle the requests. Call its RunWorkerAsync method to start the background task (in our case, the HttpListener).

The OnStop method will serve to shut down the HttpListener:

  Protected Overrides Sub OnStop()     With listener         .Stop()         .Close()     End With End Sub

The background task performed by the BackgroundWorker component can be any process that you don’t want to interfere with the normal application’s processing. If this were a Windows Forms application, then having a long-running loop or other process running might prevent the application from drawing or responding to the user’s requests. Beyond that, you can do anything you want in the background task, with one exception: Because a Windows Forms application works in a single foreground task, one can’t directly access the controls on the Form from the background task. Instead, if the background task must change properties on the controls, it should fire events. The controls can then subscribe to those events, where you can access the properties. In this Windows Service, it has no such user interface, so that problem is avoided.

The actual work you want the BackgroundWorker to perform is in the DoWork event handler:
```
  Private Sub BackgroundWork_DoWork(ByVal sender As System.Object, _   ByVal e As System.ComponentModel.DoWorkEventArgs) Handles BackgroundWork.DoWork     Dim context As HttpListenerContext     Dim path As String     Dim defaultPage As String     'this is where we actually process requests     While listener.IsListening         context = listener.GetContext         path = context.Request.Url.AbsolutePath.ToLower         'strip out the serviceName if you're using the URL format:         'http://server/servicename/path         If path.StartsWith("/" & theService.ToLower) Then             path = path.Substring(theService.Length + 1)         End If         Me.EventLog.WriteEntry("Received request for " & path)         Select Case path             Case "/"                 'this would probably be a resource                 defaultPage = "Available pages<ul>" & _                     "<li><a href='/time'>Current server time</a></li>" & _                     "<li><a href='/date'>Current date</a></li>" & _                     "<li><a href='/random'>Random number</a></li></ul>"             Case "/time"                 SendPage(context.Response, DateTime.Now.ToLongTimeString)             Case "/date"                 SendPage(context.Response, DateTime.Now.ToLongDateString)             Case "/random"                 SendPage(context.Response, New Random().Next.ToString)             Case Else                 'if we don't understand the request, send a 404                 context.Response.StatusCode = 404         End Select     End While End Sub 
```
Our background task performs its work in a loop as long as the HttpListener is actively listening. Every developer knows that performing a set of tasks in a (relatively) tight loop is dangerous, possibly leading to computer or application lockup. However, the BackgroundWorker performs this on another thread, leaving our application responsive.

For this application, we first get access to the context for the listener. The context groups together one client’s set of communication with our listener. Similar to the HttpContext in ASP.NET, the HttpListenerContext provides access to the HttpListenerRequest and HttpListenerResponse objects, so the first step in handling a request should always be to get this context. Next, the code uses a very simple means of determining the request URL. In a more full-featured implementation, this could be more complex, separating any query values from the path requested, and so on. For this sample, the listener only responds to three main paths - /time, /date, and /random - to receive the current (server) time or date, or a random Integer value. If the user requests anything else, we return a 404.

The SendPage subroutine simply writes out a basic HTML page, and the value determined:

  Private Sub SendPage(ByVal response As HttpListenerResponse, _   ByVal message As String)     Dim sb As New StringBuilder     'build string     With sb         .Append("<html><body>")         .AppendFormat("<h3>{0}</h3>", message)         .Append("</body></html>")     End With     Me.EventLog.WriteEntry(sb.ToString)     'set up content headers     With response         .ContentType = "text/html"         .ContentEncoding = Encoding.UTF8         .ContentLength64 = sb.ToString.Length         Me.EventLog.WriteEntry(sb.ToString.Length.ToString)         Try              Using writer As New StreamWriter(.OutputStream)                  With writer                      .Write(sb.ToString)                      .Flush()                  End With              End Using         Catch ex As Exception             Me.EventLog.WriteEntry(ex.Message, EventLogEntryType.Error)         Finally             'close the response to end             .Close()         End Try     End With End Sub

There shouldn’t be anything surprising in this code. Using a StringBuilder, a response is built. Then the content is written back to the browser (see Figure 29-8) using a StreamWriter that is created on top of the Response.OutputStream. Remember to close the Response, or the request will never close until it times out.

image from book
Figure 29-8

Before you can test your Windows Service, however, it must be installed. On the Properties window for the actual service, click Add Installer (see Figure 29-9). This adds a new file to the project called ProjectInstaller.vb, and adds two components to the file, ServiceInstaller1 and ServiceProcessInstaller1. You can either keep these names or change them as you desire. In addition, set the properties as shown in the following table:

Open table as spreadsheet

Component	Property	Value
ServiceInstaller1	Description	Sample Service from Wrox Professional Visual Basic 2005
	DisplayName	Sample Service
	ServiceName	SampleService
ServiceProcessInstaller1	Account	LocalSystem

image from book
Figure 29-9

Most of these properties only affect the display values for the Windows Service, but the Account property of the ServiceProcessInstaller deserves special mention. Windows Services run on behalf of the user, so they can actually run under another user account. By setting the Account property to LocalSystem, you are setting the resulting Windows Service to run under the local system account. This account has a lot of access to the system, so you may want to instead use an account with more limited rights; however, you would have to create this account separately.

Build the Windows service. Unfortunately, if you attempt to run the service directly from Visual Basic, you will get an error message (see Figure 29-10).

Figure 29-10

A Windows Service can only run if it has been installed into the system, a task performed using a command-line utility InstallUtil.exe. Open the Visual Studio Command Prompt and navigate to the directory where you have built MiniServer.exe. Run installutil minis erver.exe, and with any luck you’ll be greeted with a success message (see Figure 29-11). Note that if you are running Windows Vista, then you need to run the Visual Studio Command Prompt as an administrator. You can do this by right-clicking on the Visual Studio 2005 Command Prompt icon and selecting Run As Administrator.

Figure 29-11
Start your new service. Open the Services application from Start All Programs Administrative Tools. Find the Sample Service in the list (see Figure 29-12) and click Start. You should now be able to request one of the items the service is listening to, such as http://localhost:9090/time (see Figure 29-13).

Figure 29-12

Figure 29-13

To confirm that all of the prefixes work, request one of the values using the vroot, rather than using the port (see Figure 29-14).

Figure 29-14

The HttpListener adds yet another powerful way for your applications to communicate. It enables you to extend the reach of your applications to Web browser clients, without requiring the additional administrative and management overhead of IIS.