Hack 85. Geocode U.S. Locations with the GNIS

Make ordinary place names mappable.

As you become a mapping fanatic, you'll want to put all kinds of things on a map. The problem is that most things, although they obviously represent a place on Earth, don't come with geographic coordinates. This is why it's handy to have a toolkit of geocoding resources at your disposal whenever you're creating a new database.

This isn't a difficult thing to do, but it can be a very time-consuming and monotonous process. We'll make use of a terrific government resource, the USGS Geographic Names Information System (GNIS) at http://geonames.usgs.gov/gnishome.html. This site is a nice gateway into the wealth of geographic information available from the federal government about your neighborhood. More important for our purposes is its ability to give us geographic coordinates for any of "2 million physical and cultural geographic features in the United States."

Unfortunately, the GNIS doesn't have an API, so in order to automate the lookup of place names, we'll have to do some HTML form emulation and screen-scraping. Here's a Java program that does exactly this. First, we have a class called GNISCoords.java, shown in Example 7-1. Given a state and place name, it knows how to look up the geographic coordinates in the GNIS.

Example 7-1. GNISCoords.java

package org.rajsingh.maphacks; import java.io.*; import java.net.*; import java.util.*; /** * Get the latitude and longitude for a place name from the USGS * Geographic Names Information Service (USGS GNIS) */ public class GNISCoords { public String gnisQueryService = "http://geonames.usgs.gov/pls/gnis/"; /** * Make a request to the USGS GNIS. Parse the table that is returned * and return the coordinates in a 2-member String[ ]. The first element * will be the longitude (a negative number in the continental US), and * the second will be the latitude. * @param gnisRequest call to the USGS GNIS * @return longitude and latitude */ private String[ ] requestCoords(URL gnisRequest) { String[ ] coords = new String[2]; try { BufferedReader in = new BufferedReader(new InputStreamReader String l = ""; Vector webPage = new Vector( ); while ( (l=in.readLine( )) != null ) webPage.add(l); if ( getNumCols(webPage) > 2 ) { // find the URL for requesting the 1st matching place URL newRequest = getFirstOption(webPage); // try again to make the request coords = requestCoords(newRequest); } else { // the table has the coordinates we want coords = findCoords(webPage); } } catch (IOException ioe) { ioe.printStackTrace( ); } return coords; } /** * Make a request to the USGS GNIS to get the geographic coordinates * of a place in the US and return the coordinates in a 2-member String[ ]. * The first element will be the longitude (a negative number in the * continental US), and the second will be the latitude. * @param townName town name, e.g. Cambridge * @param stateName full state name, e.g. Massachusetts * @return longitude and latitude */ public String[ ] requestCoords(String townName, String stateName) { try { String req = gnisQueryService; req += "web_query.gnisprod?f_name="+URLEncoder.encode(townName, req += "&f_state="+URLEncoder.encode(stateName,"UTF-8"); // these don't change (for our purposes) req += "&f_ty=populated+place"; req += "&variant=N"; req += "&f_cnty=&f_ty=&elev1=&elev2=&cell=&my_function=Send+Query"; req += "&last_name=&last_state=&last_cnty=&page_cnt=&record_ URL gnis = new URL(req); return requestCoords(gnis); } catch (MalformedURLException mue) { mue.printStackTrace( ); } catch (IOException ioe) { ioe.printStackTrace( ); } return null; } /** * Use the coordinates from the link called, * "View USGS Digital Raster Graphic (DRG)" * @param webPageTable * @return longitude and latitude */ private String[ ] findCoords(Vector webPage) { String[ ] coords = new String[2]; for (Iterator iter = webPage.iterator( ); iter.hasNext( );) { String line = iter.next( ).toString( ).toLowerCase( ); if ( line.indexOf("view usgs digital raster graphic") > 0 ) { int begin = line.indexOf("lon=")+4; int end = line.indexOf(""", begin); coords[0] = line.substring(begin, end); begin = line.indexOf("lat=")+4; end = line.indexOf("&lon=", begin); coords[1] = line.substring(begin, end); return coords; } } return null; } /** * Parse a table listing matches for a town lookup. * The first URL on the page will be the one we want * @param webPageTable * @return */ private URL getFirstOption(Vector webPage) { String req = null; for (Iterator iter = webPage.iterator( ); iter.hasNext( );) { String line = iter.next( ).toString( ); if ( line.indexOf("web_query.GetDetail?") > 0 ) { int begin = line.indexOf("web_query.GetDetail"); int end = line.indexOf(""", begin); req = line.substring(begin, end); break; } } if ( req != null ) { try { URL ru = new URL(gnisQueryService + req); return ru; } catch (MalformedURLException e) { e.printStackTrace( ); } } return null; } private int getNumCols(Vector webPage) { int colcount = 0; boolean inrow = false; for (Iterator iter = webPage.iterator( ); iter.hasNext( );) { String line = iter.next( ).toString( ).toLowerCase( ); if ( line.startsWith("

7.9.1. Running the Hack

Compile the Java file like this.

% javac org/rajsingh/maphacks /GNISCoords.java

 

Now you can run the program. Raj Singh