Gluing the Directory Together: Knowledge References

So far, we've described directory partitions in terms of the entries they contain. However, we still need some way to describe the relationships between directory partitions. The LDAP and X.500 standards define these relationships in terms of knowledge references pointers to directory information held in another partition. There are two types of knowledge references:

1. Immediate superior knowledge references . This type of knowledge reference points upward in the DIT toward the root and ties the directory partition to its ancestor. In effect, it provides the "hook" at the top of the directory partition that you use to hang a partition from its ancestor .

2. Subordinate references . These knowledge references point downward in the DIT to other partitions. They are the hooks onto which you hang other directory partitions.

Figure 10.6 illustrates how directory partitions fit together into a larger DIT. In this example, the partition root ou=London,dc=example,dc=com denotes the top of the partition. The partition contains the partition root and all the entries underneath it, except for those entries held in partitions pointed to by the two subordinate references, ou=Accounting,ou=London,dc=example,dc=com and ou=Engineering, ou=London,dc=example,dc=com . An immediate superior knowledge reference also points upward in the DIT, connecting this partition to its parent partition. Knowledge references always come in pairs; although not shown, the parent partition has a subordinate reference pointing to the partition in the figure.

Figure 10.6. A Directory Partition in a DIT

Note

Directory servers based on the LDAPv3 standard advertise the partitions they hold in a special entry called the root DSE ( DSA-specific entry ; DSA stands for Directory System Agent and is the X.500 term for a directory server). As explained in Chapter 2, Introduction to LDAP, the root DSE is a special entry that contains information about the directory server, including its capabilities and configuration.

To discover the partitions held by a given server, you can search for the entry with a zero-length name , with a scope of base and a search filter of (objectclass=*) . The command ldapsearch h host -s base b "" "(objectclass=*)" will display the root DSE on the host named host . The attribute namingContexts of the returned entry lists the partition root of each partition held by the server.

Your directory server software may use these same terms, or it may have an alternative way of talking about how partitions are named and related . But in virtually all cases, this model is a good basis for understanding how your server software can be used to construct real-world directories.

Dividing a directory in this fashion isn't useful all by itself. If we just left the partitions as a series of isolated islands, locating an entry or searching across a series of partitions would be tedious . The user of the directory would need to know which servers to contact to perform an operation that spanned multiple partitions. Clearly, directory partitions need to know about each other to work together as a single, cohesive unit. In the next section we discuss how directory partitions work together to appear as a single unit.

Name Resolution in the Distributed Directory

Now that you understand partitions and knowledge references, the basic building blocks of a distributed directory, let's examine an important concept: name resolution . This is the process by which a directory maps a distinguished name (DN) provided by a client to an actual object in the directory. We use this process in the following circumstances:

• When locating the base object of an LDAP search or compare operation

• When locating the target entry for a modify, delete, rename, or bind operation

• When locating the parent of an entry to be added to the directory

A DN presented in this fashion is called a purported name . The client purports (assumes) that it exists, and it is up to the directory system to check whether this is true.

In the simplest casea client contacts a directory server and presents the DN of an entry contained in that serverthe name resolution process is simple: The server simply checks whether the entry exists and takes the appropriate action. For example, if a directory client attempts to read an entry via an LDAP base object search, the server checks whether the DN of the entry is within one of the directory partitions it holds; if it is, it checks its database for the entry. If the entry exists, any requested attributes are returned. Otherwise, a "no such object" error is returned to the client.

If the client presents a purported name whose DN is not within any of the directory partitions held by a server, the server must use any available knowledge references to resolve the name itself or aid the client in locating the server that can resolve the name. Conceptually, the server resolves the name by "walking up" or "walking down" the directory tree until the appropriate server is located. The knowledge references fully describe how a given directory partition fits into the DIT, so a server always knows whether the next step in the name resolution process involves walking up or walking down the DIT.

For example, suppose that the directory client depicted in Figure 10.7 contacted Server 1 and presented the purported name cn=Barbara Jensen,ou=PCB Design, ou=Engineering, dc=example,dc=com . Name resolution would proceed as follows :

1. The directory client would present the purported name.

2. Because Server 1 does not hold a directory partition that could contain the entry, the immediate superior knowledge reference would allow the name resolution process to walk up the tree to Server 2.

3. Server 2 contains a subordinate reference for the directory partition ou=Engineering, dc=example,dc=com , so the name resolution process would walk down the tree to Server 3.

4. Server 3 holds a subordinate reference for the directory partition ou=PCB Design,ou=Engineering,dc=example,dc=com , so the process would walk down the tree again.

5. Finally, Server 4 would be consulted to check whether the entry actually exists.

Figure 10.7. Name Resolution in a Distributed Directory

Up to this point, we've been discussing knowledge references and name resolution in an abstract sense, and you may be curious about the software in the directory system that actually performs this tree-walking operation. LDAPv3 supports client-side processing of knowledge references, as well as server-side processing. In client-side processing, the knowledge reference information is provided to the client, which uses that information to contact additional servers to fulfill the request. In server-side processing, the server contacts other servers on the client's behalf and provides the complete set of results. In the next sections we discuss these two approaches in detail, examine the benefits and drawbacks of each approach, and offer advice on how to choose between them.

Handling Distribution in the Client: LDAP Referrals and Search Result Continuation References

LDAP referrals and search result continuation references are pieces of knowledge reference information sent from an LDAP server to an LDAP client indicating that other servers need to be contacted to fulfill the client's request. Referrals and search result continuation references are defined as follows:

• LDAP referrals . When an LDAP client performs an LDAP add, modify, delete, modify DN, compare, or search operation with a base-level scope, and the server does not hold the target entry, the knowledge reference information is returned to the client in an LDAP referral. An LDAP referral is a special type of LDAP result message (LDAP errors are also carried in result messages) that directs the client to another server that may be able to service the operation.

• LDAP search result continuation references . When an LDAP client performs a one-level or subtree search and the server has knowledge of other subordinate directory partitions, knowledge references are returned as search result continuation references. A search result continuation reference is a search result (an entry is another type of search result) that carries information about the subordinate directory partitions and how to contact servers that hold those contexts.

LDAP referrals and search result continuation references are very similar. The only significant difference between them is how the information is packaged: Whereas LDAP referrals are contained in LDAP result messages, search result continuation references are contained in search result messages. Normally, the LDAP client libraries automatically handle processing of referrals and search result continuation references, and the subtle differences are hidden.

Figure 10.8 illustrates with an example. Suppose that an LDAP client issues a search operation to Server 1 with a search base of ou=Engineering,dc=example,dc=com , a scope of subtree , and a filter of (&(objectclass=person)(sn=smith)) . In other words, the client wants to find all persons within ou=Engineering,dc=example, dc=com whose surname is Smith.

Figure 10.8. Distributed Searching with Client-Side Processing of Knowledge References

To service this search request, the distributed directory first uses name resolution to find the base object of the search. It then uses any knowledge references to refer the client to all the servers it needs to contact to obtain the entire set of matching entries.

The following sequence describes this search procedure in detail:

Part 1: Resolving the name of the base object of the search.

Step 1. The client submits the search request to Server 1.

Step 2. Server 1 does not hold an appropriate directory partition, so it examines the superior reference and returns to the client a referral to Server 2.

Step 3. The client submits the search operation to Server 2.

Step 4. Server 2 contains a subordinate reference to Server 3 for an appropriate directory partition, so it returns to the client a search result continuation reference that points to Server 3.

Step 5. The client submits the search operation to Server 3.

Part 2: Servicing the search operation.

Step 6. Server 3 searches its database and returns any matching entries to the client. Because it also contains a subordinate reference to Server 4, it returns to the client a search result continuation reference that points to Server 4.

Step 7. The client submits the search operation to Server 4.

Step 8. Server 4 returns any matching entries to the client.

Step 9. The client combines the lists of entries returned from Server 3 and Server 4, and presents them to the user.

This example shows how the directory, using knowledge references, can direct a client to the correct serverseven if the client happens to ask the wrong server. In fact, the whole point is that LDAP clients should not have to worry about which server contains which data within the directory service.

The Structure of LDAP Referrals and Search Result Continuation References

The information in LDAP referrals and search result continuation references is in the format of an LDAP Uniform Resource Locator (URL), as defined in RFC 2255. The following information is contained in a referral or reference:

• The host name of the server to contact.

• The port number of the server.

• The base DN (if processing a search operation) or target DN (if processing an add, delete, compare, or modify DN operation). This part is optional; if absent, the client should use the same base DN it used when performing the operation that resulted in the referral.

This information is all that is required for the client to chase the referral. For example, if a client contacts Server 2 and searches the subtree dc=example,dc=com for all entries with the surname Smith (as in step 3 of the preceding example), the referral would be returned as the following LDAP URL:

 ldap://server3.example.com:389/ou=Engineering%2Cdc=example%2Cdc=com 

This URL indicates that the client should contact the host server3.example.com on port 389 and submit a search rooted at ou=Engineering,dc=example,dc=com to collect the rest of the search results (each comma within the DN is represented by the hexadecimal escape sequence "%2C" as required by the URL specifications). Notice that the base DN in the referral is different from the one originally supplied by the client. When a server changes the search base before sending the referral or search result continuation reference, it is indicating to the client that, in addition to contacting a different server, it must search another part of the DIT to complete the search processing.

Note

Support for LDAP referrals and search result continuation references is standardized as part of the LDAPv3 specification. Servers and clients based on the older U-M LDAPv2 source code distribution support an experimental implementation that supports only referrals and is incompatible with the LDAPv3 standard. Netscape Directory Server 6 operates with both v2 and v3 clients by sending LDAPv3-format referrals to LDAPv3 clients and LDAPv2-format referrals to LDAPv2 clients.

Tip

If you are developing an application using the Netscape C LDAP SDK or the Netscape Java LDAP SDK, you can either instruct the SDK to follow, or "chase," referrals automatically (the default with the C SDK), or you can tell it to pass referral information back to your program for action.

One reason you might not want to chase referrals automatically is if you wanted to provide feedback to the user indicating that a referral was being followed. Or you might want to warn the user if a referral outside your corporate firewall is returned.

As a developer, you have total control over how your application handles referrals. Good directory client software even allows the end user to configure whether referrals should be automatically followed and how to handle authentication when referrals are being followed.

Handling Distribution in the Server: Chaining

Another way to glue together directory partitions is with chaining . When this approach is used, the client starts by submitting an operation to a server in the usual way. If the server is unable to process the request completely because it does not hold all the required data, it contacts other servers on behalf of the client (forming a chain of connections between servers). After the other servers have completed the operation, the server that was contacted by the client returns the combined results to the client. Figure 10.9 shows how a directory distributed by means of chaining would respond to the query shown in Figure 10.8.

Figure 10.9. Distributed Searching with Chaining

When using chaining, servers perform the following steps to service a search operation, as illustrated in Figure 10.9:

Part 1: Resolving the name of the base object of the search.

Step 1. The client submits the search request to Server 1.

Step 2. Server 1 does not hold an appropriate directory partition, so it consults its immediate superior reference and chains the operation to Server 2.

Step 3. Server 2 contains a subordinate reference to Server 3 for an appropriate directory partition, so it chains the operation to Server 3.

Step 4a. Server 3 verifies that the base object exists.

Part 2: Servicing the search operation.

Step 4b. Server 3 searches its database for any matching entries. It also discovers that it holds a subordinate reference to Server 4, so it chains the operation to Server 4.

Step 5. Server 4 searches its database and returns any matching entries to Server 3.

Step 6. Server 3 combines the entries retrieved from its own database with those returned by Server 4 and sends the resulting set to Server 2.

Step 7. Server 2 returns the set of entries to Server 1.

Step 8. Server 1 returns the set of entries to the client.

Step 9. The client presents the set of entries to the user.

As you can see, the same number of steps is involved in client-side and server-side processing of knowledge reference information. The major difference is which directory service component performs the processing necessary to retrieve the distributed information. In a chained system, the processing is done by the servers; in a system that uses referrals, the processing is done by the clients.

Deciding between Client-Side and Server-Side Processing of Knowledge Reference Information

Each method of linking distributed directory servers has advantages and disadvantages. Server-side processing generally reduces client complexity, but at the cost of increased server complexity. Servers that support chaining must be able to collect search results from remote servers and send the resulting set to directory clients. Client-side processing, on the other hand, requires more client complexity to handle the chasing of referrals and collation of search results. However, it offers more flexibility for client application writers and allows a developer to provide more feedback to an application's users about the progress of a distributed directory operation.

You may not have a choice about which method you use if your directory server supports only one method. Some directory server software, such as Microsoft Active Directory, supports only referrals, and the Microsoft LDAP client software automatically follows referrals generated by Active Directory. Other directory software, such as Novell eDirectory and Netscape Directory Server 6, allows you to glue your directory together using either referrals or chaining, or a combination of the two.

Servers based on X.500 standards usually support chaining and may also support referrals. The X.500 specification does not actually mandate that servers support chaining, so the type of distribution method (chaining or referrals) again depends on the software in use. Consult your server software documentation to determine which methods are supported and recommended.

Configuring a Distributed Directory

The procedure by which you configure a distributed directory depends on the type of directory server software you're using. The following are the four most common types of directory server software:

1. Microsoft Active Directory . Active Directory manages the relationships between domains using proprietary methods. Tools provided with Active Directory allow you to add and remove domains as needed.

2. Novell eDirectory . Novell eDirectory is modeled on the X.500 standards, but it uses proprietary methods to establish new partitions and set up the hierarchical relationships between them. Novell's Partition Manager utility allows a directory administrator to create new partitions and assign them to Novell eDirectory servers.

3. X.500 . X.500 servers based on the 1993 standard define a protocol for setting up and tearing down relationships between servers. The actual protocol is known as the Hierarchical Operational Binding Protocol. As the name implies, it is used to establish a hierarchical relationship between two X.500 naming contexts and sets up the superior and subordinate references. Your software may not use this rather frightening terminology (in fact, one would hope not!); instead, it may provide a more user-friendly interface for specifying partitions and how they relate to one another. Consult your server's documentation for specific procedures.

4. Netscape Directory Server 6 . Netscape Directory Server 6 requires that you manually configure the relationship of each server to other directory servers in your organization. You can configure distribution via client-side or server-side processing. With client-side processing, knowledge reference information is returned to the client in referrals and search result continuation references. With server-side processing, the server chains operations to other servers as needed to service the client's request, and then sends the combined result to the client. We'll discuss Netscape Directory Server 6 in more detail next.

Configuring Distribution with Netscape Directory Server 6

Before you configure your distributed directory, you must decide how to apportion your directory data across multiple servers. That topic is complex, and it is discussed later in this chapter, in the section titled Designing Your Directory Server Topology. For the purposes of this example, let's assume we have decided that our topology will be as shown in Figure 10.3. Also assume that the host name of Server 1 is server1.example.com and the host name of Server 2 is server2.example.com . Server 1 holds the directory partition dc=example,dc=com . This partition contains all of the corporate data for example.com . Server 2 holds the directory partition ou=External Customers, dc=example,dc=com . This partition holds information about example.com 's customers. Note that this partition is subordinate to the previous partition.

We would like to be able to search across the entire directory without worrying about the fact that it's split across two servers. Therefore, we need to connect these two servers into a single directory topology. As mentioned previously, we can accomplish this with client-side or server-side processing of knowledge reference information.

Configuring Distribution with Client-Side Processing

Client-side processing of distribution information is configured in three steps. To configure client-side processing for the topology that was depicted in Figure 10.3, for example, first you create a suffix for ou=External Customers,dc=example,dc=com on Server 1 (a suffix is another name for a directory partition). Second, you configure that suffix to generate referrals to the subordinate partition by providing the LDAP URL ldap://server2.example.com:389 . Third, on the server holding the subordinate partition (Server 2), you set a default referral that directs clients to the superior partition's server by providing the LDAP URL ldap://server1.example.com:389 .

Once the servers have been configured in this manner, clients will be referred to the appropriate server, no matter which server they submit their search operation to. Here are some examples of this behavior:

• If a client contacts Server 1 and performs a subtree search with a search base of dc=example,dc=com and a filter of (sn=smith) , Server 1 will return any matching entries in its database, along with a search result continuation reference to Server 2. The client will then contact Server 2 and submit the same search operation, but with a search base of ou=External Customers,dc=example,dc=com , and Server 2 will return any matching entries.

• If a client contacts Server 2 and submits this LDAP search, Server 2 will realize that it does not hold the partition dc=example,dc=com . Therefore, it will return its default referral, which directs the client to contact Server 1. The client will submit the operation to Server 1, and the process will proceed as in the previous example.

Complete instructions for configuring client-side processing can be found in the section titled Creating Suffix Referrals in Chapter 3 of the Netscape Directory Server 6 Administrator's Guide .

Configuring Distribution with Server-Side Processing

To configure server-side processing of distribution information, you establish a relationship between two directory servers. The Netscape term for this relationship is a database link . A database link is configured on the server that holds the superior directory partitionin our example, Server 1. A database link contains the following configuration information:

• The host name and TCP port number of the server containing the subordinate partition

• Whether to use SSL when contacting that server

• The DN of the entry to bind as, and the password to use when binding

You can configure database links using Netscape Console.

After the database link is configured on the superior server, you must perform two more configuration steps on the server holding the subordinate partition (Server 2 in Figure 10.3):

Step 1. Create the entry that corresponds to the bind DN that the superior server will use when connecting, and set its password. Creating this entry allows Server 1 to authenticate to Server 2 when chaining an operation.

Step 2. Set an access control instruction (ACI) in the entry at the top of the subordinate partition. This ACI must grant proxy authorization rights to the entry you just created. The proxy authorization rights allow the server holding the superior partition to impersonate other users. In our example, Server 1 will authenticate to Server 2 using the configured bind DN. However, when Server 1 chains an LDAP operation to Server 2, it includes an LDAPv3 Proxied Authorization control that instructs Server 2 to perform the operation as if it had been performed by the user who originally submitted the operation to Server 1. For more information on the LDAPv3 Proxied Authorization control, see Chapter 3, LDAPv3 Extensions.

At this point the two servers are configured so that Server 1 will chain search operations to Server 2. However, there is still another configuration step that must be performed. Server 2 must be configured so that it refers search operations outside of ou=External Customers,dc=example,dc=com to Server 1. Once this step has been performed, client operations will be processed as follows:

Step 1. If a client contacts Server 1 and performs a subtree search with a search base of dc=example,dc=com and a filter of (sn=smith) , Server 1 will chain the operation to Server 2, collect the entries returned by Server 2, and combine those entries with any matching entries in its database. The complete result set will be returned to the client.

Step 2. If a client contacts Server 2 and submits this LDAP search, Server 2 will realize that it does not hold the partition dc=example,dc=com . Therefore, it will return its default referral, which directs the client to contact Server 1. The client will submit the operation to Server 1, and the process will proceed as in the previous example.

Complete instructions for configuring server-side topology processing can be found in the section titled Creating and Maintaining Database Links in Chapter 3 of the Netscape Directory Server 6 Administrator's Guide .