Understanding Open Directory Replication | Apple Training Series: Mac OS X System Administration Reference, Volume 1

In a replication system, master and replica servers are dynamically and automatically synchronized to keep their contents identical. If computers go down, this ensures that services remain available and helps to avoid costly downtime. Mac OS X computers in your network can use either the master or any replica for authentication and other directory service functions. Any changes on a master or replica will be distributed to all replicas.

The process is described in the following figure:

The top Mac OS X computer is attempting to contact an Open Directory master whose LDAP database is not currently running.

Because it is configured to also look at a Replica of the Open Directory master, the Mac OS X computer can still access the LDAP information.

The second Mac OS X computer is simply configured to initially access the replica.

Replicas can greatly increase the robustness of your organization's network in two ways:

Redundancy: Mac OS X computers will automatically use a replica system for authentication when the master system is down.
Load sharing: Replication allows Open Directory servers to be load-shared. When a Mac OS X computer seeks authentication, it can query all known Open Directory systems. The Mac OS X computer will then use the first system to respond.

Essentially, an Open Directory replica is a server that contains a duplicate of the directory data stored on a master server. If you are configuring a Mac OS X computer to retrieve records from a replica server, you should add one replica to the Authentication and Contacts search paths. If a server failure should occur, Mac OS X will use other information to reconfigure itself to connect to another replica or master server. You'll learn more about this process later in this lesson.

Replication Topology

One Open Directory master can maintain several different replicas, and replicas can reside in the same local network as the master server or in remote networks, as illustrated in the following figure. Each replica is replicating the data from the Open Directory master with the IP address of 17.1.1.5.

When designing a replication topology, consider the following needs:

Redundancy: Redundancy dictates that every Open Directory master support at least one replica. If the master were to go offline for any reason, the replica would take over responsibilities until the master came back online.
Performance: In a healthy network of even up to a few hundred computers, you should not need more than two or three replicas. If a remote location requires authentication services, you should install a replica at that remote location.
Network bandwidth: Each replica adds to the amount of traffic on a network.
Although there is no software limitation to the number of replicas you can create, don't burden the network unnecessarily.

Tip

The actual number of replicas you need depends upon the needs of your network. As a rule of thumb, use 1 replica for every 250 computers, or 1 for every remote location. Effective planning will help you determine the right number for your organization.

Creating a Replica

Using the Open Directory module of Server Admin as seen in the following figure, you can define a Mac OS X server as a replica. The actual setup takes place on the computer you wish to assign as a replica, rather than on the existing master. You'll be asked for the IP address of the master Open Directory server, the system's root password on that server, and an LDAP directory administrator user name and password on that server. Because replication requires changes on the master server and to the master server's LDAP database, you need both the root and administrator passwords for the master. If replication is successful, you'll see the master server's address entered into the Master field on the replica.

Before creating a replica, make sure that your current Open Directory master is configured as you would like it to be, including that the KDC is running and that the naming context of your LDAP database is correct. Once you have established a replication system, it is a complex process to update existing replicas to be consistent with a significantly reconfigured Open Directory master. For example, if you did not have Kerberos enabled at the time replicas were created, and then you later enabled Kerberos on the master, you would need to create a Kerberos record for each of the replicas to allow them to fully participate in the Kerberos realm. This entails creating host keys for the replicas and entering the host principal into the KDC. This process would have already been done for you if the KDC was running on the master server before the replicas were created.

Note

Each replica must have a serial number separate from that of the master, or replication will fail.

Replica Creation Process

As with establishing an Open Directory master, slapconfig is the engine that drives the replica creation process. Replica creation is logged in the /Library/Logs/slapconfig.log log file on both the master and the replica. These two logs, when viewed together, completely document the automated replica creation process, which is deceptively simple in design and well thought out.

Create a Secure Shell Protocol (SSH) connection between the replica and master computers.
Before these steps can be initiated, an SSH connection must be created. On the replica, slapconfig initiates SSH connections to the master as the master's root user to check the supplied password for root and the LDAP administrator user on the master. If either the root or the domain administrator's password is wrong, or the supplied administrator user does not exist in the LDAP domain on the master, the replication process cannot continue. If there are any issues with the SSH keys between these two computers, the process will also fail. The server software serial numbers on the two computers are also checked at this time. If they are the same, the replication process will fail since the serial numbers are used to identify the servers during replication.

Destroy any LDAP and KDC databases.
Any existing LDAP and KDC databases on the replica are destroyed and their servers are shut down.
Note

If you need any information stored in the replica's LDAP or KDC databases, export it before beginning the replication process and import it back in when the procedure is done.

The Password Server's database on the replica is retained since this will contain passwords for local users on the replica computer in addition to the replicated network users.

Export the master computer's LDAP database.
slapconfig stops the master computer's LDAP server and calls slapcat to dump the entire LDAP database to an LDAP Interchange Format (LDIF) file.

Copy the LDIF file to the replica computer using scp, which is called by slapconfig.

Update the master computer's configuration.
The master adds the replica computer's IP address and a replication-specific user to a number of configuration files, including /etc/openldap/slapd_macosxserver.conf file, which correspond to an entry in the replica's copy of this file. All LDAP replication between the two servers is authenticated by this user, which is not listed in the actual LDAP database.
Entries are also made in the LDAP database at cn=passwordserver,cn=config and cn=ldapreplicas,cn=config, which alert Mac OS X computers and other replicas to the existence of the replica. At this time, the master computer's LDAP server is restarted so that Mac OS X computers can continue to use the master for directory services while the replication process continues.
Note

The replication process for a large system of 30,000 users should take only a few minutes to complete over a LAN connection.

Generate a new LDAP database using the LDIF file.
The LDIF file that was copied from the master computer is now used to generate a fresh LDAP database on the replica computer using slapadd. Once the LDAP database is created, the replica computer's LDAP database is started.

Start the replicator on the master computer.
If slurpd is not already running on the master, it starts to read the change log generated by slapd. Then, using the replication user that is shared between the master computer and replica computer (created in step 5), it will add the changes into the replica computer's LDAP database.

Enable Password Server and KDC on the replica computer.
The replica computer's Password Server is made aware that it is now in a multimaster situation. At this point, Password Server will attempt to contact the master Password Server in the replication system when any password change is initiated on this particular replica.
Password Server copies the KDC database on the Open Directory master in its entirety from the master to the replica. The sso_util command is then run to generate service keys for the services hosted on the local replica. This will also change the appropriate service configuration files to allow the master and replica computers to be fully integrated into the Kerberos realm.

The following procedure details the replica creation process as driven by slapconfig and illustrated in the preceding figure.

After the KDC is created, the replica creation process is complete. Your next task is to configure replication settings on the master computer.

Replica Configuration

Once a replica has been established, you will see the server listed on the Open Directory master in Server Admin. In the General pane of the Open Directory module, you can force a replication to ensure that a replica is up-to-date, which is useful when a replica has been offline for a period of time and has just come back online.

Note

Clicking the Replicate Now button forces a replication to all replicas, not just the one you have selected in the list of replicas.

To customize the replication interval, services can either hold all changes and replicate once every interval or replicate whenever the directory is modified. If all your replicas are on the same local area network (LAN), you should probably set them to replicate when modified. However, if you have a number of remote networks, it is advisable to set the replication interval to a few hours to reduce traffic across wide area network (WAN) connections. Replication times set for under 60 minutes will not be recognized, and the replication will take place immediately.

Note

Updates to the replication interval might not be recognized immediately by the replicas. Also, only major problems occurring during the replication process will be displayed in the Result column of the Server Admin interface. If the replica is offline, the interface status will display OK.

Tip

A longer replication interval might cause some confusion because a changed password, for example, will not be immediately replicated to the other Password Servers.

Any confusion due to a longer replication interval will be mitigated by using single sign-on, since authentication is tied to a ticket instead of the user's password. The user can authenticate to the replica with the changed password and acquire a valid ticket that will be honored by all other servers in the Kerberos realm.

Connecting the Mac OS X Computer

In order for a Mac OS X version 10.4 computer to connect to a server in a replication system, the computer must have a search path listed in the Authentication pane of Directory Access. This allows the computer to discover the replication system and build its list of replicas.

As a general rule, a Mac OS X computer should not have more than one server from the same replication system listed in the Authentication pane of Directory Access, or that computer will search the same list of users multiple times, thus negating some of the benefits of actually having the replicas in the system.

When the Mac OS X computer first contacts a server in a replication system, it downloads a list of LDAP replicas from the LDAP server to the computer and stores it in /Library/Preferences/DirectoryService/DSLDAPv3PlugInConfig.plist. In addition, the Writable key identifies the Open Directory master in this file.

Note

Mac OS X v10.2 computers and earlier are unable to automatically find a replica. However, you can sequentially list the replicas in Directory Access. Then the Mac OS X computer will try each server sequentially as necessary when attempting to authenticate a user.

The Mac OS X computer caches all known Password Servers in the replication system in /var/db/authserverreplicas.local. When the Mac OS X computer attempts to authenticate using Password Server, it looks at the Password Server public key that is stored in the user's authentication authority and references that key in this file to find out what Password Servers can be used for authentication.

The Mac OS X computer also keeps a cache of all known KDCs in the replication system in the edu.mit.kerberos file located in /Library/Preferences. Whenever a new KDC replica is added to the replication system, that server is automatically added to the default Kerberos config file stored in the Open Directory database. Mac OS X computers will check this serialized file to see if any changes have been made, and if so, the computers will download this modified file to /Library/Preferences/edu.mit.kerberos, thus giving them access to this new replica.

Once the Mac OS X computer has the listing of all servers in the replication system, it uses the server that is specified in the LDAP configuration plug-in. If that server is unresponsive, the Mac OS X computer will progressively contact all LDAP servers, Password Servers, and KDCs. The first of each type of server to respond is cached in memory and will be used for lookups and authentication. Ideally, this will result in the Mac OS X computer using the closest serveror servers, since it is entirely possible that you will use separate servers for LDAP, Password Server, and the KDCto it in the replication system.

The Mac OS X computer will continue to use the servers cached in memory until it restarts, its network settings change, or a cached server becomes unresponsive. At this time, it will initiate the discovery process again. If the Mac OS X computer cannot contact any of the known Password Servers in its list, it will then search over Bonjour for any other servers that share the same public key. Failing that, the Mac OS X computer will attempt to connect to a Password Server running on local computer. This last step is of benefit only when the Mac OS X computer is actually a Mac OS X Server system that has had its IP address changed.