Section 2.3. Working with Directories

2.3. Working with Directories

It's a not-so-well-kept secret that the best approach for getting into the search engine listings is to enter through a back door by working with the two most important structured directories: the Open Directory Project (ODP) and the Yahoo! Directory .

2.3.1. Understanding Taxonomies

A directory differs from the index used by a search engine because a directory uses a structured way to categorize sites, sometimes called a taxonomy. In addition, sites are included in a particular category in the ODP and Yahoo! directories only after they have been reviewed by human editors. You can search within directories (just as you can search in a web index, such as the one compiled by Google). But it's common to use a directory, following its taxonomy by drilling down through subjects to find what you want. For example, suppose you wanted to find resources related to alternative photo processes, such as creating daguerreotypes (a nineteenth-century print technology). Using the Open Directory taxonomy, shown in Figure 2-2, you would drill down through the following categories: Arts Photography Techniques and Style Alternative Processes.

You can think of the index of the Web compiled by search engines such as Google as being like the index to a nonfiction book. In contrast, a taxonomic directory is much more like the table of contents to the book: it is organized according to the structure of the book, and you can drill down by part, chapter (within the part), heading (within the chapter), and subtopic to find the information you need.

Figure 2-2. You can find "Alternative Processes" by drilling down through Arts, Photography, and Techniques and Styles

2.3.2. Getting Open Directory Project Listings

The Open Directory Project (ODP), http://dmoz.org, is the most important taxonomic directory on the Web. Formally hosted and administered by the Netscape division of AOL, the ODP is run along the lines of an open source project and is inspired by the Debian Social Contract (see http://www.debian.org/social_contract.html).

The credo behind the ODP is that "humans do it better." The ODP believes that web automated search is ineffective, and getting worse, and that the small contingent of paid editors at commercial web search engine companies cannot keep up with the staggering rate of change on the Webdecaying, stagnant sites, link rot, new sites, sites intended as search spam, and so on.

The ODP is run and maintained by a vast army of volunteer editors. These editors follow internal checks and balances to preserve the integrity of the directory. See http://dmoz.org/guidelines/ for more information about the ODP review process and guidelines for site inclusion.

You, too, can become an ODP editor in an area of your interest and expertise. See http://dmoz.org/help/become.html for more information about becoming an ODP editor.

The ODP taxonomy (categorization system) and the sites included in the categories are freely available as data for use by anyone who wants to run his or her own search engine, as long as the terms of the ODP's free-use license are complied with (for terms of the license, in case you want to use the ODP data in a search engine of your own, see http://dmoz.org/license.html).

Google, and most of the major search engines, do use information derived from the ODP, but of course they use it in their own way. With Google in particular, information from the ODP is used to form the Google Directory , http://directory.google.com.

Google uses its own search technology for searches within the Google Directory.

Most significant, inclusion within an ODP category means that your site is very likely to be included within the Google web index (as well as the Google Directory and in other major web indices).

So it's worth submitting your site to the ODP, if only because it's the best way to get indexed (and appropriately categorized) by Google. You'll find a FAQ about how to add your site at http://www.dmoz.org/add.html (this FAQ is also available via a link from the ODP home page).

The first step is to locate the best category for your site. For example, suppose you have a site like Syndication Viewer, shown in Figure 2-3, whose purpose is to catalog and display selected RSS and Atom feeds as HTML.

The best category I can find for this site on ODP is Reference Libraries Library and Information Science Technical Services Cataloguing Metadata RDF Applications RSS. The category page is shown in Figure 2-4.

You can start your look for the right category (to get close to the best possible category) using a search term, for example, "RSS."

On the category page that you think is best for your site, click the suggest URL link, shown in the upper center of Figure 2-4.

On the Submit a Site to the Open Directory Project page, you will be asked to verify the category you selected (as determined by the page from which you clicked the suggest URL link). You'll need to enter your site's URL, title, a brief site description, and

Figure 2-3. The Syndication Viewer site catalogs RSS and Atom feeds

your email address. As the editors note, "A well-written, objective description will make listing your site easier."

When your listing page is complete, click Submit. This process is now complete, except for the waiting. You should check back in four to six months to see if you've been listed.

If I make it sound like you might have to wait a long time to get listed in the ODP, well, you might! The ODP depends on volunteer labor, and rumor is that it is getting slower and slower. However, inclusion in the ODP is a virtual guarantee of inclusion in many search engine indexes and other directories. So have patience! The ODP is worth it.

2.3.3. Getting Yahoo! Directory Listings

The Yahoo! Directory, a somewhat lesser-known part of Yahoo!, works in pretty much the same way as the ODP, except that it is privately maintained. Sites added to the Yahoo! Directory tend to end up in the Yahoo! index, as well as other important search indices.

Figure 2-4. ODP category pages are where you find the suggest URL links

To suggest your site for inclusion in the Yahoo! Directory , open the Yahoo! Directory's home page, http://dir.yahoo.com/, shown in Figure 2-5.

You can also find the Yahoo Directory by opening the main Yahoo! home page, selecting Directory as your search category, and searching for a term. The search results you will be presented with are from the Yahoo! Directory (not the Yahoo! web index), and the display will show where in the taxonomy you are, so you can browse through related categories.

Next, find the best category for your site, either by drilling down through the Yahoo! Directory taxonomy (shown on the left of Figure 2-5), or by searching within the Yahoo! Directory using the search box shown at the top of Figure 2-6.

Figure 2-5. Yahoo! Directory is not Yahoo!; here's the Yahoo! Directory home page

You can use directory search results as the starting place for pinpointing the perfect category.

When you find the right category page (for example, for a digital photography site, the category page for Directory Arts Visual Arts Photography Digital), click the Suggest a Site link shown in Figure 2-6.

Figure 2-6. From the appropriate category page, click the Suggest a Site link to propose your site for inclusion in the Yahoo! Directory

Clicking the Suggest a Site link starts the site submission process. You'll first be asked if you want to pay for inclusion or continue for free. I suggest you do not pay for inclusion.

The next step is to verify that the site you want to suggest does not already appear in the Yahoo! Directory. This is verified using a Yahoo! Directory search. If your site cannot be found, you can continue.

You'll be asked to verify the listing category, and to log in with your Yahoo! username. (If you don't have a Yahoo! account, you'll need to create one.) Finally, you can complete the form with information about your site shown in Figure 2-7 and submit.

Figure 2-7. You can use your elevator pitch story to fill out the Yahoo! Directory Suggest a Site form

Keep track of the email address you supply on the Yahoo! Directory Suggest a Site form. You'll need the address to change information about any of your sites listed in the directory.