Searching the Web with Googles SOAP Service

Problem

You want to use Googles web services to perform searches and grab their results within your Ruby application.

Solution

Google exposes a SOAP API to its search functionality, and some other miscellaneous methods like spellcheck. Call these methods with the SOAP client that comes with Rubys standard library:

	$KCODE = u # This lets us parse UTF characters
	require soap/wsdlDriver

	class Google
	 @@key = JW/JqyXMzCsv7k/dxqR9E9HF+jiSgbDL
	# Get a key at http://www.google.com/apis/
	 @@driver = SOAP::WSDLDriverFactory.
	 new(http://api.google.com/GoogleSearch.wsdl).create_rpc_driver

	 def self.search(query, options={})
	 @@driver.doGoogleSearch(
	 @@key,
	 query,
	 options[:offset] || 0,
	 options[:limit] || 10, # Note that this value cannot exceed 10
	 options[:filter] || true,
	 options[:restricts] ||  ,
	 options[:safe_search] || false,
	 options[:lr] ||  ,
	 options[:ie] ||  ,
	 options[:oe] ||  
	 )
	 end

	 def self.count(query, options={})
	 search(query, options).estimatedTotalResultsCount
	 end

	 def self.spell(phrase)
	 @@driver.doSpellingSuggestion(@@key, phrase)
	 end
	end

Here it is in action:

	 
Google.count "Ruby Cookbook site:oreilly.com"
	# => 368

	results = 
Google.search "Ruby Cookbook site:oreilly.com", :limit => 7
	results.resultElements.size
	# => 7

	results.resultElements.first["title"]
	# => "oreilly.com -- Online Catalog: Ruby Cookbook…"

	results.resultElements.first["URL"]
	# => "http://www.oreilly.com/catalog/rubyckbk/"

	results.resultElements.first["snippet"]
	# => "The Ruby Cookbook is a new addition to …"

	 
Google.spell "tis is te centence"
	# => "this is the sentence"

Discussion

Each of the options defined in Google.search corresponds to an option in the Google search API.

Table 16-2.

Name	Description
key	Unique key provided when you sign up with Googles web services.
query	The search query.
limit	How many results to grab; the maximum is 10.
offset	Which result in the list to start from.
filter	Whether or not to let Google group together similar results.
restricts	Further restrict search results to those containing this string.
safe_search	Whether or not to enable the SafeSearch filtering feature.
lr	Language restriction: lets you search for pages in specific languages.
ie	Input encoding: lets you choose the character encoding for the query.
oe	Output encoding: lets you choose the character encoding for the returned results.

See Also

• For a simpler API, see Recipe 16.7, "Using a WSDL File to Make SOAP Calls Easier"
• http://www.google.com/apis/reference.html
• http://www.google.com/help/refinesearch.html