17.6. Creating a Web Client with WebRequest and WebResponse
| < Day Day Up > |
| As you would expect in a chapter on ASP.NET, the role of the Web client does not receive much attention. It's usually assumed to be a browser that requests a Web page specified by the user, and renders the returned HTML content. Despite the evolution of browser features, its primary role remains to display what it receives. It is of less use if you want to parse and extract portions of the received content or want to examine the HTTP message headers. For this, you need a Web client a program that communicates with an HTTP server and includes custom methods to perform operations on the response object. This section demonstrates how easy it is to write a Web client using .NET classes to handle the HTTP request/response duties. WebRequest and WebResponse ClassesThe System.Net namespace contains classes that are intended for applications involved with network communications. Included in these are the abstract WebRequest and WebResponse classes that contain generic properties to upload and download data given a specific Uniform Resource Identifier (URI). These classes are designed to support the response/request model not a specific protocol. The details of that are left to descendant classes. To handle the HTTP protocol, we call on the HttpWebRequest and HttpWebResponse classes. Note that other classes are available to support the file:// URI scheme and FTP operations. Web Client ExampleThis example accepts a URL, sends a request for its Web page, and uses the response object to display the server description, IP address(es) of the server, and source code for the requested Web page. Figure 17-10 shows the interface for the application. Figure 17-10. Example using HttpWebRequest and HttpWebResponse classes The basic steps for communicating synchronously with the HTTP server are quite simple:
Note that request and response are cast to HttpWebRequest and HttpWebResponse types, respectively. As mentioned, these subclasses deal specifically with the HTTP protocol (see Listing 17-9). Listing 17-9. Using WebRequest and WebResponse to Scrape a Web Page private void btnURL_Click(object sender, System.EventArgs e) { // Fetch web page for requested URL HttpWebRequest request; HttpWebResponse response; if(txt_URL.Text.Length>0) { lblServer.Text=""; tbIP.Text=""; string serverPath= txt_URL.Text; string url="http://"+serverPath; // create a request to the url request = (HttpWebRequest) WebRequest.Create(url); request.Timeout= 7000; // timeout after 7 seconds try { response = (HttpWebResponse) request.GetResponse(); lblServer.Text= response.Server; // Get a stream to send the web page source Stream s = response.GetResponseStream(); string strContents = new StreamReader(s).ReadToEnd(); // Place Web page source in text box HTMLViewer.Text= strContents; s.Close(); ListIP(serverPath); // List IP address(es) } catch ( Exception ex) { lblServer.Text= ex.Message; } } else { lblServer.Text= "Please enter a domain name."; } } private void ListIP(string uri) { // List IP addresses for this domain // Use only server name part of URI for IP resolution int ndx= uri.IndexOf("/"); if(ndx>0) uri= uri.Substring(0,ndx); string ips=""; // Get a list of IP addresses for the URI // Dns contacts the Internet Domain Name System IPHostEntry IPHost = Dns.GetHostByName(uri); foreach(IPAddress addr in IPHost.AddressList) ips+= addr.ToString()+"\r\n"; tbIP.Text= ips; } You may encounter Web pages that check the UserAgent property of the request object and do not allow their pages to be downloaded unless they can identify the browser. You can assign a legitimate browser value to overcome this. |
| < Day Day Up > |
