17.6. Creating a Web Client with WebRequest and WebResponse

 < Day Day Up > 

As you would expect in a chapter on ASP.NET, the role of the Web client does not receive much attention. It's usually assumed to be a browser that requests a Web page specified by the user, and renders the returned HTML content. Despite the evolution of browser features, its primary role remains to display what it receives. It is of less use if you want to parse and extract portions of the received content or want to examine the HTTP message headers. For this, you need a Web client a program that communicates with an HTTP server and includes custom methods to perform operations on the response object. This section demonstrates how easy it is to write a Web client using .NET classes to handle the HTTP request/response duties.

WebRequest and WebResponse Classes

The System.Net namespace contains classes that are intended for applications involved with network communications. Included in these are the abstract WebRequest and WebResponse classes that contain generic properties to upload and download data given a specific Uniform Resource Identifier (URI). These classes are designed to support the response/request model not a specific protocol. The details of that are left to descendant classes. To handle the HTTP protocol, we call on the HttpWebRequest and HttpWebResponse classes. Note that other classes are available to support the file:// URI scheme and FTP operations.

Web Client Example

This example accepts a URL, sends a request for its Web page, and uses the response object to display the server description, IP address(es) of the server, and source code for the requested Web page. Figure 17-10 shows the interface for the application.

Figure 17-10. Example using HttpWebRequest and HttpWebResponse classes

The basic steps for communicating synchronously with the HTTP server are quite simple:


Create an HttpWebRequest object that identifies the URL using the Create method:

 request = (HttpWebRequest) WebRequest.Create(url);  


Get a response from the Internet resource using the WebRequest.GetResponse method:

 response = (HttpWebResponse) request.GetResponse();  


Get the Stream used to read the body of the response and read its contents.

 Stream s = response.GetResponseStream(); string strContents = new StreamReader(s).ReadToEnd(); 

Note that request and response are cast to HttpWebRequest and HttpWebResponse types, respectively. As mentioned, these subclasses deal specifically with the HTTP protocol (see Listing 17-9).

Listing 17-9. Using WebRequest and WebResponse to Scrape a Web Page
 private void btnURL_Click(object sender, System.EventArgs e) {    // Fetch web page for requested URL    HttpWebRequest request;     HttpWebResponse response;     if(txt_URL.Text.Length>0)    {       lblServer.Text="";       tbIP.Text="";       string serverPath= txt_URL.Text;       string url="http://"+serverPath;       // create a request to the url        request = (HttpWebRequest) WebRequest.Create(url);        request.Timeout= 7000; // timeout after 7 seconds       try       {          response = (HttpWebResponse)                    request.GetResponse();           lblServer.Text= response.Server;          // Get a stream to send the web page source           Stream s = response.GetResponseStream();           string strContents = new                     StreamReader(s).ReadToEnd();           // Place Web page source in text box          HTMLViewer.Text= strContents;            s.Close();          ListIP(serverPath);   // List IP address(es)       }        catch ( Exception ex)       {          lblServer.Text= ex.Message;       }    }    else     {       lblServer.Text= "Please enter a domain name.";    } } private void ListIP(string uri) {    // List IP addresses for this domain    // Use only server name part of URI for IP resolution    int ndx= uri.IndexOf("/");    if(ndx>0) uri= uri.Substring(0,ndx);    string ips="";    // Get a list of IP addresses for the URI    // Dns contacts the Internet Domain Name System    IPHostEntry IPHost = Dns.GetHostByName(uri);    foreach(IPAddress addr in IPHost.AddressList)    ips+= addr.ToString()+"\r\n";    tbIP.Text= ips; } 

You may encounter Web pages that check the UserAgent property of the request object and do not allow their pages to be downloaded unless they can identify the browser. You can assign a legitimate browser value to overcome this.

     < Day Day Up > 

    Core C# and  .NET
    Core C# and .NET
    ISBN: 131472275
    EAN: N/A
    Year: 2005
    Pages: 219

    flylib.com © 2008-2017.
    If you may any questions please contact us: flylib@qtcs.net