Accessing the Internet with the System.Net Classes

Although not a common requirement, you sometime may need to access other resources on the Internet or a local network from within your ASP.NET code. This is easy using the wide range of classes in the System.Net namespace. While you can perform low-level network access through sockets in this way, the more common requirement is usually to be able to access another Web page. You may also want to, for example, look up the IP address for a specified host in a DNS server. This section looks at both of these topics.

The next chapter looks at the topic of using and consuming another type of network resourceWeb Services.

Retrieving Web Pages

Among the classes in the System.Net namespace are those that make it easy to send a request for a resource and then read the response. For HTTP requests, the classes you use are HttpWebRequest and HttpWebResponse. Other types are the base classes WebRequest and WebResponse, the FTP access classes FtpWebRequest and FtpWebReponse, and the classes FileWebRequest and FileWebResponse for accessing local files using the "file://" URI scheme.

Listing 15.25 shows the code the example page uses to fetch a Web page (the URL specified in the pageURL variable) from another site. The first stage consists of creating an HttpWebRequest instance using the static Create method of the WebRequest base class for the required URL. Then the code creates an HttpWebResponse instance over the return stream using the Getresponse method. The HttpWebResponse class exposes a series of properties, such as Status, that allow you to get information about the result of the request, as shown here.

To read the response, the code creates a StreamReader and initializes it by calling the GeTResponseStream method. It can then extract the entire response as a String by calling the ReadToEnd method of the Stream-Reader. The example application uses this code in the GetWebPage method of the file Ch15DataProcess.cs.

The HttpWebRequest class (and the other request class types) expose properties that allow you to closely control the request by specifying any credentials required for access to the remote site, specifying the request method (POST, GET, HEAD), and selecting the proxy server to use (if any). You can also add and modify the HTTP headers, add cookies to the request, set referrer and accepted file type information, and manage redirection of the request.

The HttpWebResponse class allows you to access information about the responding server, the HTTP headers and cookies, the character set and encoding used, and the size and last modified date of the page.

Listing 15.25. Requesting and Receiving a Web Page

// create a Request object to request the page HttpWebRequest req = (HttpWebRequest)WebRequest.Create(pageURL); // create a Response object to receive the response // by calling the GetResponse method of the Request HttpWebResponse result = (HttpWebResponse)req.GetResponse(); StreamReader reader = null; if (result.StatusCode == HttpStatusCode.OK) {   // response received, so read the stream   reader = new StreamReader(result.GetResponseStream());   sourceString = reader.ReadToEnd(); } else {   sourceString = "Failed request, " + result.StatusDescription; } reader.Close(); result.Close();

Figure 15.10 shows the result of selecting the Retrieved Web page option in the example and sending the results to a disk file. When you open the disk file, which the example automatically allocates the .htm file extension, you see the page rendered almost as it would be if accessed directly in the browser. However, associated resources such as images do not appear, because these are not part of the request and response. A Web browser automatically reads the links to associated content when it loads the page and issues separate requests in exactly the same way to retrieve each of them for display.

Figure 15.10. Retrieving a remote Web page and storing it as a disk file

To see what the returned content really looks like, select the Memory stream output option and repeat the request. As shown in Figure 15.11, the result is the source for the page as generated by the remote server.

Figure 15.11. A remote retrieved Web page displayed as a stream of bytes

From this short example, you should be able to see how easily you can build Web applications that request data behind the scenes or compile data from several Web-based sources.

Performing DNS Lookups

As a further example of accessing remote resources over the Internet, the code in Listing 15.26 shows how you can perform a DNS lookup on a specified domain. The Dns class in the System.Net namespace exposes the static GetHostEntry method that returns an instance of the IPHostEntry class.

This class provides a collection of IPAddress instances in its AddressList property and a list of alias host names in its Aliases property.

Listing 15.26. Performing a DNS Lookup Using the System.net Classes

// perform the DNS request IPHostEntry host = Dns.GetHostEntry(hostName); StringBuilder builder = new StringBuilder(); builder.Append("Results of DNS lookup for '"              + hostName + "'\n"); // iterate through the address list foreach (IPAddress addr in host.AddressList) {   builder.Append("IPAddress = " + addr.ToString() + "\n"); } // iterate through any list of aliases foreach (String alias in host.Aliases) {   builder.Append("Alias = " + alias + "\n"); } sourceString = builder.ToString();

The example page uses the code shown in Listing 15.26 in the GetdNSLookup method of Ch15DataProcess.cs. Figure 15.12 shows the result of a DNS lookup for "microsoft.com" (though the actual addresses are hidden in the screenshot).

Figure 15.12. The result of a DNS lookup for "microsoft.com"