Hack 65. Extract Data from Maps with Manifold

Just getting map data isn't enough to give you all the information the map might contain. If you're working with complex data, a GUI interface that lets you select the areas you consider relevant and then calculate the results can be extremely convenientif you can do it cheaply!

Manifold (http://manifold.net), a Windows-based GIS application, costs $245 but offers much of the same functionality as its "traditional" but much more expensive GIS counterparts. It can read a wide variety of map formats and provides a user interface that's deliberately familiar to people who work in Windows. You can open a map, select the bits in which you're interested, and do quick calculations on the data associated with your selected areas. Manifold also lets you create maps and perform sophisticated analysis, though this hack doesn't have the space to go into that.

For a recent project, I needed to know the population and number of houses in a specific area. While I've usually been able to get that kind of information from the American Fact Finder web site (http://factfinder.census.gov/home/saff/main.html?_lang=en), making effective use of that prepackaged data means that the areas you're interested in have to match up with the places defined by the Census Bureau. In my case, since I was trying to calculate the population of the hamlet of Varna, New York, a place with no municipal government, the prepackaged data wasn't much help. Instead, I turned to census block data available through maps at the Cornell University Geospatial Repository (CUGIR, at http://cugir.mannlib.cornell.edu/) and created a cluster of blocks that roughly corresponded to the boundaries of the hamlet.

The first step was getting the information I needed. In my case, that meant visiting http://new-gis.mannlib.cornell.edu/CUGIR_Data/cens2000/109blk00s.zip (a file that contains census block maps and data for Tompkins County, where Varna is) and extracting its contents. There are several files in there, but the primary ones I needed to work with were tgr109blk00s.shp (the shapefile) and tgr109blk00s.dbf (the underlying datafile).

To try this project yourself, download and extract the census block maps and open Manifold. Create a new project (through File images/ent/U2192.GIF border=0> New) and go to File images/ent/U2192.GIF border=0> Import images/ent/U2192.GIF border=0> Drawing to open the shapefile. When you choose the shapefile, Manifold also imports the data from the database file. It asks which pieces you want, as shown in Figure 6-8.

Figure 6-8. Importing data with a shapefile in Manifold

You can then open the drawing and double-click on "Tgr109blks00s Drawing" in the Project pane at right, displaying the census blocks of Tompkins County, as shown in Figure 6-9.

Figure 6-9. A county worth of census blocks

To zoom in, click the magnifying glass with the plus sign in it and then click on the map where you'd like to see more detail. Figure 6-10 shows a closer view of Varna, which is centered roughly on -76.4363, 42.4561.

Figure 6-10. Zooming in on Varna

After turning off the magnification cursor (by clicking on the magnifying glass with the plus sign in it again), you can select a parcel and then examine its field. Click on a parcel to select it (it will turn red) and then right-click it and select "Fields..." If you scroll down the dialog box that appears, you'll see the data for that particular census block, as shown in Figure 6-11.

Figure 6-11. Data for one census block in Varna

While this general approach will work for any map you import into Manifold, the metadata information describing this map at http://cugir2.mannlib.cornell.edu/Isite/CUGIR_METADATA/blk00s.html indicates that this map was created for New York State, and you may or may not be able to find similarly convenient maps for census blocks for other places.

The next step was to select census blocks that correspond to Varna. In the Comprehensive Plan I was working with, the "hamlet area" of Varna was defined as the set of parcels shown in Figure 6-12.

Figure 6-12. Varna, defined as tax parcels

Those parcels don't precisely map to census blocks, unfortunately, but the problem is manageable because the parts of the blocks that extend beyond the parcels are largely uninhabited. (It's much easier to do reliable map processing when you can check out the area physically on occasion.) Selecting all the parcels that contain the Varna hamlet area by Control-clicking produces the selection shown in Figure 6-13.

Figure 6-13. Varna, defined as a collection of census blocks

The easiest way to calculate the total population for this area is combine all of these blocks into a single selection. Rather than making the change to the original map, copy the blocks. Then, go to File images/ent/U2192.GIF border=0> Create images/ent/U2192.GIF border=0> Drawing and create a drawing named VarnaCoreBlocks or something similar. You'll need to double-click it in the Project pane at right to open it, and then paste in your blocks. Manifold will show you that it's adding data to the table with a dialog box like that shown in Figure 6-14, and the result will look like Figure 6-15.

Figure 6-14. Data being pasted

Figure 6-15. Varna's census blocks, isolated

The easiest way to get totals for the information in these blocks is to dissolve them into a single large piece. The Drawing menu's "Dissolve..." option does just this. When you select it, you'll see a dialog box like that shown in Figure 6-16.

Figure 6-16. Choosing a field for dissolving boundaries

In this case, the point is simply to dissolve all of the boundaries, but Manifold expects that people will be somewhat more selective. You can choose between [All Objects] and [Selected Objects], letting you dissolve across an entire drawing or just selected parts. You can also choose a field to match on for combining objects. To dissolve all the boundaries, choose a field in which all the values for the field are identical. In this case, STATE will work fine, since all of the pieces are in New York State. Clicking OK produces the result shown in Figure 6-17, and showing the fields for that one area produces the result shown in Figure 6-18.

Figure 6-17. Varna census blocks dissolved into a single object

Figure 6-18. The totaled fields for that single object's census data

The population of Varna, as defined by the plan noted earlier, is 679 people, according to the 2000 census. (Because Manifold adds all the values, the averages included in the data don't come out so well and need to be recalculated.) One final nice touch is that you can combine the new drawing with the old one to create a map that highlights the area chosen for calculation. In Manifold, go to File images/ent/U2192.GIF border=0> Create images/ent/U2192.GIF border=0> Map..., and select the two drawings, as shown in Figure 6-19.

Figure 6-19. Creating a map with two layers of drawings

Open the map you've created and then right-click on the tab for "Tgr109blk00s Drawing." Choose "Opacity..." from the menu that appears, and set it to 30%. If you then zoom in a little, you'll see the darker Varna area standing out against the surrounding blocks, and the divisions for the blocks it contains are still visible, as shown in Figure 6-20.

Figure 6-20. Two related maps at different levels of opacity, highlighting the area on which calculations were made

This is barely scratching the surface of the kinds of things you can do with Manifold, which lets you create, edit, and analyze maps in ways this hack can't begin to explain.

Simon St.Laurent