Chapter 3. HttpClient

It goes without saying that there is a staggering amount of data available on the Internetspecifically, available via the World Wide Web (HTTP). One of the wonderful things about the JDK is the built-in ability to access resources via HTTPspecifically, the java.net.HttpURLConnection class shows that simple HTTP resources can easily be downloaded by Java applications.

Unfortunately, the built-in HTTP connectivity in the JDK suffers from some significant limitations (for a partial feature set comparison, see http://www.nogoop.com/product_16.html#compare). Fortunately, these limitations are overcome when using the Commons HttpClient. A full list of features supported by HttpClient (including links to the relevant RFCs) can be found at http://jakarta.apache.org/commons/httpclient/features.html, but some of the most interesting features include:

Automatic Cookie ManagementAllowing access to web resources that require web form authentication and session tracking via cookies.

Multipart form POST for file uploadSee Chapter 2, "FileUpload," for more information on multipart form posts and file upload.

True request/response streamsThe 1.4.2 implementation of Java copies data into a byte array before sending and receiving data, whereas HttpClient passes the streams directly to the application.

Built-in support for transparently handling redirects.

Glancing at Figure 3-1, which shows the non-Exception classes just in the org.apache.commons.httpclient package, you might think that the HttpClient package is complex and difficult to use. While exercising the full range of capability may be complex, we'll look at how we can use the HttpClient classes to easily deal with cookies and posting forms.

Figure 3-1. Main HttpClient classes.

The biggest difference between HttpClient and the default JDK HTTP connectivity is the use of classes to implement each of the various HTTP requests. These classes are then passed to a persistent HTTP client object to execute. Listing 3-1 shows a simple GET request. The response to a particular method invocation is bound to the method object, not the client.

Listing 3-1. Simple GET Request

 HttpClient client = new HttpClient(); HttpMethod method = new GetMethod("http://www.cascadetg.com/"); client.executeMethod(method); System.out.println(method.getStatusCode()); System.out.println(method.getResponseBodyAsString()); method.releaseConnection(); 

Redirects

One of the nuisances of dealing with the default JDK HTTP connectivity is the lack of support for server redirects. The server uses a redirect to indicate to a client that the requested page is no longer available and that the client should load an alternative page. Unlike the JDK implementation, this redirect is transparently followed by HttpClient (i.e., your application receives the content from the redirected URL). You can optionally configure the handling of redirects on a per-request basis with the HttpMethod.setFollowRedirects() method. You also can use the HttpMethod.getPath() method after the execute() to obtain the path that was actually finally read.

If you only want to follow a certain number of redirects before failing, you should turn setFollowRedirects() to false and detect redirect status responses manually.