Customizing HTTP Request Headers

Problem

When you make an HTTP request, you want to specify custom HTTP headers like " User-Agent" or " Accept-Language".

Solution

Pass in a Hash of header values to Net::HTTP#get or Net::HTTP#post:

	require 
et/http
	require uri

	#A simple wrapper method that accepts either strings or URI objects
	#and performs an HTTP GET.
	module Net
	 class HTTP
	 def HTTP.get_with_headers(uri, headers=nil)
	 uri = URI.parse(uri) if uri.respond_to? :to_str
	 start(uri.host, uri.port) do | 
http|
	 return 
http.get(uri.path, headers)
	 end
	 end
	 end
	end

	#Lets get a web page in German.
	res = Net::HTTP.get_with_headers(http://www.google.com/,
	 { 
Accept-Language => de})

	#Check a bit of the body to make sure its really in German.
	s = res.body.size
	res.body[s-200..s-140]
	# => "ngebote</a> - <a href=/intl/de/about.html>Alles 374ber Google




Discussion
Usually you can retrieve the web pages you want without specifying any custom HTTP headers. As you start performing more complicated interactions with web servers, youll find yourself customizing the headers more.
For instance, if you write a web spider or client, youll want it to send a " 
User-Agent" header on every request, identifying itself to the web server. Unlike the HTTP client libraries for other programming languages, the net/http library doesn	 send a "User-Agent" header by default; its your reponsibility to send one.
	Net::HTTP.get_with_headers(url, {User-Agent => Ruby Web Browser v1.0})



You can often save bandwidth (at the expense of computer time) by sending an "Accept-Encoding" header, requesting that a web server compress data before sending it to you. Gzip compression is the most common way a server compresses HTTP response data; you can reverse it with Rubys zlib library:
	uncompressed = Net::HTTP.get_with_headers(http://www.cnn.com/)
	uncompressed.body.size
	# => 65150

	gzipped = Net::HTTP.get_with_headers(http://www.cnn.com/,
	 {Accept-Encoding => gzip})
	gzipped[Content-Encoding]
	# => "gzip"
	gzipped.body.size
	# => 14600

	require zlib
	require stringio
	body_io = StringIO.new(gzipped.body)
	unzipped_body = Zlib::GzipReader.new(body_io).read()
	unzipped_body.size
	# => 65150



If you want to build up a 
HTTP request with multiple values for the same 
HTTP header, you can construct a Net:: 
HTTP::Get (or Net::HTTP::Post) object and call the add_field method multiple times. The example in the Solution used the " 
Accept-Language" header to request a document in a specific language. The following code fetches the same URL, but its "Accept-Language" header indicates that it will accept a document written in any of four different dialects:
	uri = URI.parse(http://www.google.com/)

	request = Net::HTTP::Get.new(uri.path)
	[en_us, en, en_gb, ja].each do |language|
	 request.add_field(Accept-Language, language)
	end
	request[Accept-Language]
	# => "en_us, en, en_gb, ja"

	Net::HTTP.start(uri.host, uri.port) do |http|
	 response = http.request(request)
	 # … process the HTTPResponse object here
	end





See Also
Recipe 12.10, "Compressing and Archiving Files with Gzip and Tar," for more about the zlib library
Recipe 14.1, "Grabbing the Contents of a Web Page"
Recipe 14.20, "A Real-World HTTP Client," covers a lot of edge cases youll need to handle if you want to write a general-purpose client
REST web services often use the value of the "Accept" header to provide multiple representations of the same resource; Joe Gregorios article "Should you use Content Negotiation in your Web Services?" explains why its a better idea to provide a different URL for each representation (http://bitworking.org/news/WebServicesAndContentNegotiation)
Recipe 16.1 for more on REST web services