Hack 99. Map Your Friend-of-a-Friend Network

Spider and map your social network on the Web in easy steps.

What's a Friend of a Friend (FOAF)? Is it the RDF vocabulary for making statements about people; the shadowy originator of urban myths; or the myriad of friend-of-a-friend social networking sites, a few of which support inclusion of FOAF and XFN in XML? For our purposes, we'll concentrate on just the first definition

Before this gets too acronym-heavy, let's explain a few basic principles of FOAF networks. FOAF is the vocabulary that I use for making statements about myself, my "digital identity," and about my social network. In my FOAF file, I give my name, nicknames, web site addresses, and encrypted versions of my email addresses. I also publish a list of people I know, using the FOAF knows property. This doesn't distinguish how well I know different people or imply any reciprocity about the relationship.

Both the group-blogging services Typepad and LiveJournal export FOAF information for their users, and there is increasing take-up of it by social networking sites such as tribe.net. Not only can you describe and share data about and via a social network with this information, but for free you get the benefits of many other semantic web vocabularies to describe events, spatial locations, and bits of associated media.

9.8.1. A Simple FOAF File

 A. Person
 ap
 
 
 
 
 
 
 zool
 
 
 

This file describes A. Person and tells you a little about his geospatial location and contact details, as well as providing a small snippet of his social network. It can live on a web server, and it's usual to use the .rdf extension and text/rdf+xml MIME type for it.

The FOAF specification, detailed at http://xmlns.com/foaf/0.1/, provides some simple ways to indicate your approximate location. You can say you are foaf:based_near a location. The semantics of "near" are intentionally vague. What foaf:based_near points to is a geo:SpatialThing, something that generally has a latitude and longitude property in WGS84.

Alternatively, one can convey approximate location in FOAF using the contact:nearestAirport property. This comes from the "Contact" personal information management vocabulary published by the w3 (http://www.w3.org/2000/10/swap/pim/contact#). Airports are referred to by their three-letter international airport code (LHR, SFO, LAX, etc.). The nearestAirport property was designed to allow people to indicate where they are, in an international context, without revealing too much personal contact information.

Each airport has a URL associated with it in RDF, which is defined in an airport schema:

I can also create a link to another FOAF file by attaching:

 

9.8.2. Crawling the FOAF Web

The following quick Perl script is a semantic web "scutter" or spider; it accepts a list of files and builds a list of pages to index as it traverses the rdfs:seeAlso list. It uses Class::RDF, a module that is available from CPAN:

#!/usr/bin/perl
 
use strict;
use Class::RDF;
Class::RDF->set_db('dbi:SQLite:foaf.db');
Class::RDF->define('rdfs' => 'http://www.w3.org/2000/01/rdf-schema#',
 'foaf' => 'http://xmlns.com/foaf/0.1/',
 'airport' => 'http://www.daml.org/2001/10/html/airport-ont#',
 'contact' => 'http://www.w3.org/2000/10/swap/pim/contact#', );
Class::RDF::NS->export('rdfs','foaf','airport','contact');
 
my $air_base = 'http://jibbering.com/rdf/airports.1?';
 
my @seen = ( );
my @files = (shift);
 
while (my $traverse = shift @files) {
 print "reading $traverse
";
 my @objects;
 eval {
 @objects = Class::RDF->parse( uri => $traverse );
 };
 my @follow = grep {$_->rdfs::seeAlso} @objects; 
 foreach my $url (map {$_->rdfs::seeAlso->uri->value } @follow) {
 my ($seen) = grep {$_ eq $url} @seen;
 if (not $seen) {
 push @files, $url;
 push @seen, $url;
 } 
 } 
 my @air = grep {$_->contact::nearestAirport} @objects;
 foreach my $airport (@air) {
 eval {
 Class::RDF->parse( uri => $air_base.$airport->
 airport::iataCode ); 
 };
 }
}

While it's collecting files from the semantic web, this script is looking for airports in the graphs it gets back. When it finds an IATA code property, it looks it up at the service (e.g., http://jibbering.com/rdf/airports.1?BRS). This returns a bigger RDF graph with the airport's location, time zone, name, and other features.

Jim Ley used a crawl of the FOAF web and an SVG world base map to draw the "FOAF people map" shown in Figure 9-16; you can see the SVG version along with other interesting FOAF experiments at http://www.jibbering.com/.

Figure 9-16. Jim Ley's FOAF people map

 

9.8.3. Making Your Own FOAF World Map

Featured in this book are a couple of different web services that will accept a feed of points in RDF/XML or RSS and return an interactive map showing the points plotted over a world map; one is worldKit [Hack #39], another is RDFMapper [Hack #21] .

The RDFMapper service at Map Bureau takes an RDF file with georeferenced entries, parses key information on it, and presents a set of annotated points on an interactive map. You can also post to it the URLs of functions that you want to use to extract and display information from the RDF model. The functions are written in Fabl, a simple special-purpose RDF-processing language.

The following short Fabl function reads geo-annotated FOAF and looks for the latitude and longitude of things that are foaf:based_near or contact:nearestAirport to something else. This code lives in a file on a web server, just like a regular HTML page, and you send the RDFMapper web service its address:

install(namespace('foaf'));
namespace('contact','http://www.w3.org/2000/10/swap/pim/contact#');
install(namespace('contact'));
 
geom2d:Point function extractFoafLocation(Resource itm)
{
 
 var Resource geom2d:Point;
 if (count(itm,foaf:based_near)>0)
 return extractGeoLatLong(itm.foaf:based_near);
 if (count(itm,contact:nearestAirport)>0)
 return extractGeoLatLong(itm.contact:nearestAirport);
 return nil ~ geom2d:Point;
}

Now we need to make some RDF for RDFMapper to eat. This scrap of Perl finds all the geolocated things in the data model that our scutter built from the semantic web, and all the things that the geolocs are a property of:

# same Class::RDF headers as before
 
my @near = Class::RDF::Object->search(geo->lat);
 
push @out, @near;
 
foreach (@near) {
 my @n = Class::RDF::Statement->search(object => $_->uri->value);
 push @out, map { Class::RDF::Object->new($_->subject->value) } @n;
}
print Class::RDF->serialise(@out);

We drop this file on a web server somewhere and point the RDFMapper web service at it. The full instructions, including details on customizing the display and page layout, are at http://www.mapbureau.com/rdfmapper/.

All the scripts accompanying this hack, and a demo showing the current state of geo-annotated FOAF world, are at http://mappinghacks.com/foafworld/.

9.8.4. Identifying People with FOAF

If you have an eye for RDF/XML, you'll notice that the two Persons in our first example are not identified by URL, but by temporary nodes that have temporary IDs relative only to that documentwhat's known as a bnode in RDF terms. In the FOAF world, it can be considered impolite to refer to people by URLs. I am not a number, nor do I want to be assigned a barcode from birth (although the FOAF spec does describe a dna_checksum property). Because we can't, or don't want to, assign a unique global identifier to each person, we use a technique called smushing to connect all the information we spidered from the FOAF web into one big graph and figure out which people are actually the same people, mentioned several times.

At http://xmlns.com/foaf/0.1/, the FOAF vocabulary lists the classes and properties of people (and other "Agents," such as organizations) that you can talk about in FOAF terms. If you View Source on that URL, you'll see a machine-readable RDF version of the document, which applies a few logical constraints to these properties. These are described in OWL, the W3C's Ontology Web Language.

The FOAF schema states that both foaf:mbox and foaf:homepage are of type owl:InverseFunctionalProperty. This implies that given the pairs of statements X foaf:mbox 'mailto:[e-mail] and Y foaf:mbox 'mailto:[e-mail], we can infer that X and Y refer to the same thing.

I describe my friends' locations in my FOAF file, and they describe them in theirs; we thus end up with two people on the map who are in fact the same person. To give our scutter a bit more integrity, we need to provide it with a smusher, an algorithm that will go through our model and join the graphs we found in different files together at their Inverse Functional Properties. A smushing extension to our simple Perl scutter is available with the rest of this hack's background material at http://www.mappinghacks.com/foafworld/.