2.3. Working with Directories
It's a not-so-well-kept secret that the best approach for getting into the search engine listings is to enter through a back door by working with the two most important structured directories: the Open Directory Project (ODP) and the Yahoo! Directory .
2.3.1. Understanding Taxonomies
A directory differs from the index used by a search engine because a directory uses a structured way to categorize sites, sometimes called a taxonomy. In addition, sites are included in a particular category in the ODP and Yahoo! directories only after they have been reviewed by human editors. You can search within directories (just as you can search in a web index, such as the one compiled by Google). But it's common to use a directory, following its taxonomy by drilling down through subjects to find what you want. For example, suppose you wanted to find resources related to alternative photo processes, such as creating daguerreotypes (a nineteenth-century print technology). Using the Open Directory taxonomy, shown in Figure 2-2, you would drill down through the following categories: Arts Photography Techniques and Style Alternative Processes.
Figure 2-2. You can find "Alternative Processes" by drilling down through Arts, Photography, and Techniques and Styles
2.3.2. Getting Open Directory Project Listings
The Open Directory Project (ODP), http://dmoz.org, is the most important taxonomic directory on the Web. Formally hosted and administered by the Netscape division of AOL, the ODP is run along the lines of an open source project and is inspired by the Debian Social Contract (see http://www.debian.org/social_contract.html).
The credo behind the ODP is that "humans do it better." The ODP believes that web automated search is ineffective, and getting worse, and that the small contingent of paid editors at commercial web search engine companies cannot keep up with the staggering rate of change on the Webdecaying, stagnant sites, link rot, new sites, sites intended as search spam, and so on.
The ODP is run and maintained by a vast army of volunteer editors. These editors follow internal checks and balances to preserve the integrity of the directory. See http://dmoz.org/guidelines/ for more information about the ODP review process and guidelines for site inclusion.
The ODP taxonomy (categorization system) and the sites included in the categories are freely available as data for use by anyone who wants to run his or her own search engine, as long as the terms of the ODP's free-use license are complied with (for terms of the license, in case you want to use the ODP data in a search engine of your own, see http://dmoz.org/license.html).
Google, and most of the major search engines, do use information derived from the ODP, but of course they use it in their own way. With Google in particular, information from the ODP is used to form the Google Directory , http://directory.google.com.
Most significant, inclusion within an ODP category means that your site is very likely to be included within the Google web index (as well as the Google Directory and in other major web indices).
So it's worth submitting your site to the ODP, if only because it's the best way to get indexed (and appropriately categorized) by Google. You'll find a FAQ about how to add your site at http://www.dmoz.org/add.html (this FAQ is also available via a link from the ODP home page).
The first step is to locate the best category for your site. For example, suppose you have a site like Syndication Viewer, shown in Figure 2-3, whose purpose is to catalog and display selected RSS and Atom feeds as HTML.
The best category I can find for this site on ODP is Reference Libraries Library and Information Science Technical Services Cataloguing Metadata RDF Applications RSS. The category page is shown in Figure 2-4.
On the category page that you think is best for your site, click the suggest URL link, shown in the upper center of Figure 2-4.
On the Submit a Site to the Open Directory Project page, you will be asked to verify the category you selected (as determined by the page from which you clicked the suggest URL link). You'll need to enter your site's URL, title, a brief site description, and
Figure 2-3. The Syndication Viewer site catalogs RSS and Atom feeds
your email address. As the editors note, "A well-written, objective description will make listing your site easier."
When your listing page is complete, click Submit. This process is now complete, except for the waiting. You should check back in four to six months to see if you've been listed.
2.3.3. Getting Yahoo! Directory Listings
The Yahoo! Directory, a somewhat lesser-known part of Yahoo!, works in pretty much the same way as the ODP, except that it is privately maintained. Sites added to the Yahoo! Directory tend to end up in the Yahoo! index, as well as other important search indices.
Figure 2-4. ODP category pages are where you find the suggest URL links
To suggest your site for inclusion in the Yahoo! Directory , open the Yahoo! Directory's home page, http://dir.yahoo.com/, shown in Figure 2-5.
Next, find the best category for your site, either by drilling down through the Yahoo! Directory taxonomy (shown on the left of Figure 2-5), or by searching within the Yahoo! Directory using the search box shown at the top of Figure 2-6.
Figure 2-5. Yahoo! Directory is not Yahoo!; here's the Yahoo! Directory home page
When you find the right category page (for example, for a digital photography site, the category page for Directory Arts Visual Arts Photography Digital), click the Suggest a Site link shown in Figure 2-6.
Figure 2-6. From the appropriate category page, click the Suggest a Site link to propose your site for inclusion in the Yahoo! Directory
Clicking the Suggest a Site link starts the site submission process. You'll first be asked if you want to pay for inclusion or continue for free. I suggest you do not pay for inclusion.
The next step is to verify that the site you want to suggest does not already appear in the Yahoo! Directory. This is verified using a Yahoo! Directory search. If your site cannot be found, you can continue.
You'll be asked to verify the listing category, and to log in with your Yahoo! username. (If you don't have a Yahoo! account, you'll need to create one.) Finally, you can complete the form with information about your site shown in Figure 2-7 and submit.
Figure 2-7. You can use your elevator pitch story to fill out the Yahoo! Directory Suggest a Site form