Repairing eDirectory with DSRepair

Repairing eDirectory with `DSRepair`

Every database needs a tool for repairing inconsistencies when they occur. DSRepair has been serving in this capacity as long as eDirectory has existed. Even though Novell is shifting its focus toward web-based tools, DSRepair is still essential for working on your eDirectory on a day-to-day basis. DSRepair offers three main groups of features:

Unattended full repair
eDirectory monitor operations
eDirectory repair operations

To load DSRepair on your OES Linux server, execute ndsrepair at the server console. This utility requires at least one startup switch to determine what operation will be performed. These startup switches can be confusing at first, but the man page and utility usage screen are available to help with command syntax. To access the ndsrepair utility usage screen, execute ndsrepair with the -help startup option, as shown in Figure B.4.

Figure B.4. The `ndsrepair --help` screen.

All DSRepair operations and results can be logged to a file for review. The default log file is /var/nds/ndsrepair.log. The ndsrepair utility also supports a startup option (-F) used for configuring the name and location of this log file.

The following three sections describe the features available in each of the three main categories of DSRepair operations: Unattended Full Repair, eDirectory Monitor Operations, and eDirectory Repair Operations.

Unattended Full Repair (`ndsrepair U`)

The Unattended Full Repair (UFR) is probably the most-used feature in DSRepairalthough the huge database sizes now being supported by eDirectory might change that. UFR checks for and repairs most non-critical eDirectory errors in the eDirectory database files of a given server. UFR is activated by executing ndsrepair with a U startup parameter.

The UFR performs seven primary operations each time it is run, none of which requires any intervention by the administrator. These operations are described in Table B.2. During some of these operations, the local database is locked. UFR builds a temporary set of local database files and runs the repair operations against those files. That way, if a serious problem develops, the original files are still intact. When these operations are complete, a UFR log file will be generated that outlines all the activities that have gone on, errors that were encountered, and other useful information in reviewing the state of your eDirectory environment.

Table B.2. Operations Performed by Unattended Full Repair
OPERATION	LOCKED?	DESCRIPTION
Database structure and index check	Yes	Reviews the structure and format of database records and indexes. This ensures that no structural corruption has been introduced into the eDirectory environment at the database level.
Rebuild the entire database	Yes	This operation is used to resolve errors found during structure and index checks. It restores proper data structures and re-creates the eDirectory database and index files.
Perform tree structure check	Yes	Examines the links between database records to make sure that each child record has a valid parent. This helps ensure database consistency. Invalid records are marked so that they can be restored from another partition replica during the eDirectory replica synchronization process.
Repair all local replicas	Yes	This operation resolves eDirectory database inconsistencies by checking each object and attribute against schema definitions. It also checks the format of all internal data structures. This operation can also resolve inconsistencies found during the tree structure check by removing invalid records from the database. As a result, all child records linked through the invalid record are marked as orphans. These orphan records are not lost, but this process could potentially generate a large number of errors while the database is being rebuilt. Do not be overly alarmed. This is normal and the orphan objects will be reorganized automatically over the course of replica synchronization.
Check local references	Yes	Local references are pointers to other objects maintained in the eDirectory database on this server. This operation will evaluate the internal database pointers to make sure that they are pointing to the correct eDirectory objects. If invalid references are found, an error is reported in `ndsrepair.log`.
Repair network addresses	No	This operation checks server network addresses stored in eDirectory against the values maintained in local SAP or SLP tables to make sure that eDirectory still has accurate information. If a discrepancy is found, eDirectory is updated with the correct information.
Validate stream syntax files	Yes	Stream syntax files, such as login scripts, are stored in a special area of the eDirectory database. Validate stream syntax files checks to make sure that each stream syntax file is associated with a valid eDirectory object. If not, the stream syntax file is deleted.

TIP

Rebuilding the operational indexes used by eDirectory is possible only when the local database is locked. Given this, it is good to schedule a locked database repair on a regular basis, even in large eDirectory environments.

WARNING

When the local database is locked, no changes are permitted while the operations execute. Some of these operations, when performed on very large eDirectory databases, will take an extended period of time to complete. When working with a large eDirectory database, it is best to schedule these types of operations carefully so as not to disrupt network operations.

`DSRepair` Monitor Operations

DSRepair offers several partition, replica, and server operations that are available to monitor the health of the eDirectory environment. These operations can be performed individually or as groups to help keep eDirectory stable and healthy.

Some of the DSRepair operations described in this section are available only when DSRepair is loaded in advanced mode. This is done by typing the following at the server console:

 ndsrepair P -Ad

The first category of operations can be loosely grouped into monitor operations that are designed to report eDirectory status and health. You will likely perform most of these tasks from iMonitor, but they are still available using the ndsrepair utility. Table B.3 describes the monitor operations available with ndsrepair.

Table B.3. `DSRepair` Monitor Operations
OPERATION	HOW TO ACCESS	DESCRIPTION
Report sync status	`ndsrepair -E`	Reports the sync status for every partition that hosts a replica on this server.
Report sync status of all servers	`ndsrepair P` Select a partition number and then select Report Synchronization Status of all servers	Queries each server hosting a replica of the selected partition and reports the sync status of each replica.
Report sync status on selected server	`ndsrepair P` Select a partition number and then select View Replica Ring Select a server number Select Report synchronization status on the selected server	Reports the sync status of the replica hosted by this server for the selected partition.
Report time synchronization	`ndsrepair T`	Reports status of time synchronization.
Servers known to this database	`ndsrepair -N`	Queries the local database and compiles a list of servers known to this database.

`DSRepair` Repair Operations

Although monitoring the condition of the eDirectory database is important, it does little good if there are no tools for repairing inconsistencies when they occur. DSRepair offers several eDirectory repair operations. These repair operations can be organized into three categories:

Database repair operations
Partition and replica repair operations
Global Schema operations
Other repair operations

DATABASE REPAIR OPERATIONS

All database repair options are accessed using the Repair Local Database option (-R) startup switch with ndsrepair. In addition to that main startup switch, secondary switches may be used to enable or disable specific types of database rebuild operations. An example of using these secondary switches is as follows:

 ndsrepair R t yes i no

Each secondary option used with the R repair option is explained in Table B.4.

Table B.4. `DSRepair` Database Repair Operations
OPERATION	SECONDARY NDSREPAIR SWITCH (WITH R)	DESCRIPTION
Lock NDS database during repair	`-l <yes/no>`	Causes the database to be locked throughout the repair process. This will prevent access to eDirectory from clients, but may be necessary to resolve some database issues.
Use temporary NDS database during repair	`-u <yes/no>`	Causes the repair process to be performed on a temporary copy of the database. After successful completion of the repair, the new database is copied over the original.
Maintain original unrepaired database	`-m <yes/no>`	Causes the original database to be backed up prior to the repair.
Perform database structure and index check	`-i <yes/no>`	Reviews the structure and format of database records and indexes. This ensures that no structural corruption has been introduced into the eDirectory environment at the database level.
Reclaim database free space	`-f <yes/no>`	This operation searches for unused database records and deletes them to free up disk space.
Rebuild entire database	`-d <yes/no>`	This operation is used to resolve errors found during structure and index checks. It restores proper data structures and re-creates the eDirectory database and index files.
Perform Tree structure check	`-t <yes/no>`	Examines the links between database records to make sure that each child record has a valid parent. This helps ensure database consistency. Invalid records are marked so that they can be restored from another partition replica during the eDirectory replica synchronization process.
Rebuild operational schema	`-o <yes/no>`	This operation rebuilds the base schema classes and attributes needed by eDirectory for basic functionality.
Repair all local replicas	`-r <yes/no>`	This operation resolves eDirectory database inconsistencies by checking each object and attribute against schema definitions. It also checks the format of all internal data structures. Repairing all local replicas can also resolve inconsistencies found during the tree structure check by removing invalid records from the database. As a result, all child records linked through the invalid record are marked as orphans. These orphan records are not lost, but this process could potentially generate a large number of errors while the database is being rebuilt. Do not be overly alarmed. This is normal and the orphan objects will be reorganized automatically over the course of replica synchronization.
Validate stream files	`-v <yes/no>`	Stream syntax files, such as login scripts, are stored in a special area of the eDirectory database. Validate Stream Syntax Files checks to make sure that each stream syntax file is associated with a valid eDirectory object. If not, the stream syntax file is deleted.
Check local references	`-c <yes/no>`	Local references are pointers to other objects maintained in the eDirectory database on this file server. Check Local References evaluates the internal database pointers to make sure they are pointing to the correct eDirectory objects. If invalid references are found, an error is reported in `ndsrepair.log`.

NOTE

Executing ndsrepair R will automatically cause the following options to be set to Yes: -i, -d, -t, -r, -v, and c.

PARTITION AND REPLICA REPAIR

In addition to the database repair options, DSRepair offers partition and replica operations designed to keep the distributed eDirectory environment functioning properly. This changes the focus from the local database to the partitionand all the replicas of that partition stored on servers across the network. To access these operations, execute ndsrepair P, and then select the partition with which you want to work, as shown in Figure B.5.

Figure B.5. `DSRepair` replica and partition options.

Table B.5 describes the various partition and replica operations available.

Table B.5. `DSRepair` Partition and Replica Operations
OPERATION	DESCRIPTION
Repair ring all replicas	Performs the replica ring repair operation for each replica ring in which this server participates.
Repair selected replica	Performs a replica repair on the selected replica only.
Schedule immediate synchronization	Initiates a replica synchronization cycle for each partition with a replica hosted on this server. This is useful for forcing the recognition of recent database changes.
Cancel partition operation	Causes the current partition operation (such as a move subtree) to be cancelled. Note: If the operation has progressed too far, it may not be possible to cancel the operation.
Designate this server as new master replica	If the master replica of a given partition is lost due to hardware failure, this operation can be used to designate a new master in order for partition operations to function normally.
Report synchronization status of all servers	Reports the synchronization status for all servers that host a replica of this partition.
Synchronize the replica on all servers	Each server holding a replica of the selected partition is contacted and then a synchronization cycle is initiated.
Repair ring, all replicas	Performs a replica ring repair operation of all replicas on each server in the replica table.
Repair ring selected replica	Performs a replica ring repair operation of the selected replica on each server in the replica table.
View replica ring	Displays all servers in the current partition replica ring.
View entire partition name	Displays the full distinguished name of the current partition.
Repair time stamps and declare a new epoch	Advanced option visible with Ad Increments the epoch counter and timestamps all objects within the partition. Objects are then synchronized from the Master to all other replica servers.
Destroy the selected replica on this server	Advanced option visible with Ad Removes the selected replica from the server. This option should only be used under the guidance of Novell Technical Support.
Delete unknown leaf objects	Advanced option visible with Ad Deletes all objects in the replica that have an Unknown object class. Use this option only if the objects cannot be deleted through iManager.

Five other replica operations are available by doing the following:

1.	Start `ndsrepair` using the `P` startup parameter.
2.	Start Select a replica and then choose View Replica Ring.
3.	Start Select a server from the list.

These five operations are described in Table B.6.

Table B.6. `DSRepair` Replica Ring Operations
OPERATION	DESCRIPTION
Report synchronization status on the selected server	Reports the synchronization status of the selected partition's replica that is hosted on this server.
Synchronize the replica on the selected server	Causes the replica for the selected partition to begin a synchronization cycle.
Send all objects to every replica in the ring	The operation rebuilds every replica in the ring according to the objects found in this server's replica. Warning: Any changes made to other replicas that have not yet updated to this server will be lost.
Receive all objects from the master to this replica	This operation rebuilds the local replica from object information received from the master replica. Warning: Any changes made to this replica that have not yet updated to the master replica will lost.
View entire servers name	Displays the distinguished eDirectory name for the specified server.

GLOBAL SCHEMA OPERATIONS

The schema in eDirectory is the set of rules that govern how objects are created and organized within an eDirectory tree. Though not normally necessary, ndsrepair has been provided with schema repair functionality should the local server become out of sync with other servers throughout the tree. To access these operations, start ndsrepair using the S startup option. As schema operations can have consequences throughout the entire tree, authorization of an administrator is required to perform any of these functions.

Table B.7 describes the schema operations available with ndsrepair.

Table B.7. DSRepair Global Schema Operations
OPERATION	DESCRIPTION
Request schema from Master server	Requests the Master replica of the `[Root]` partition to synchronize schema to the local server.
Reset local schema	Resets the timestamps on the local schema and requests inbound schema synchronization. This operation is not permitted on the Master of `[Root]`.
Post NetWare 5 schema update	Extends the schema for compatibility with Post-NetWare 5 schema changes. This option will only take effect on a server that holds a replica of the `[Root]` partition. This option should not be necessary on an OES server.
Optional schema enhancements	Extends the containment definitions of schema. This option will only take effect on a server that holds a replica of the `[Root]` partition. This option should not be necessary on an OES server.
Import schema from Tree	Advanced option visible with Ad This option allows the import of schema from another eDirectory tree.
Declare a new epoch	Advanced option visible with Ad This causes the Master of `[Root]` to update timestamps on schema and re-synchronize schema throughout the tree. This operation will cause a large amount of network traffic. Warning: If the Master of `[Root]` contains damaged or incorrect schema, this operation will propagate bad schema throughout the tree. This can cause extreme damage to the objects stored in the tree. This option should only be used under direct supervision of Novell Technical Support.

OTHER REPAIRS

Finally, there are four miscellaneous repair operations that are possible using the ndsrepair utility. Table B.8 describes these operations.

Table B.8. Other `DSRepair` Operations
OPERATION	HOW TO ACCESS	DESCRIPTION
Repair all network addresses	`ndsrepair N` Select any server, and then select Repair all network addresses.	This operation checks server network addresses stored in all Root Partition objects in the tree against the values maintained in local SAP or SLP tables. If a discrepancy is found, eDirectory is updated with the correct information. If no corresponding SAP or SLP entry is found, DSRepair reports an error.
Repair selected server's network addresses	`ndsrepair N` Select the desired server, and then select Repair selected server's network address.	Same as above, but only the Root Partition objects on the local server are checked.
Check external references	`ndsrepair -C`	External references are pointers to eDirectory objects not stored in partition replicas on this server. Check External References evaluates each reference to an external object to make sure that it is pointing to a valid eDirectory object. The external reference check also verifies the need for all obituaries maintained in the local database. An obituary is used to maintain database consistency while eDirectory is replicating changes such as object moves, deletes, or name changes. If a replica attempts to reference the changed object using old information because it has not received the replica sync yet, the obituary entry permits it to do so with- out generating an error. Once all replicas have synchronized with the new information, the Janitor process eliminates the obituary.
Repair single object	`ndsrepair J <EID>`	Repairs a single object on the local server. This option requires the Entry ID (in hexadecimal format) of the object that is to be repaired. The Entry ID can be located using iMonitor.

By using the operations described in this section, you will be able to manage most noncatastrophic problems in your eDirectory environment.

`DSRepair` Advanced Options

In addition to the numerous functions described in the previous sections, DSRepair also has some advanced features that are hidden from normal use. These advanced features are enabled through switches when loading the ndsrepair utility. Table B.9 provides an overview of the advanced functionality available on each platform.

Table B.9. Advanced `DSRepair` Features
SWITCH	DESCRIPTION
`-P -Ad`	Loads `ndsrepair` with advanced Partition and Replica options available. After you select a partition, the following additional menu items will be available: Repair timestamps and declare a new epoch Destroy the selected replica on this server Delete unknown leaf objects This switch also allows the Designate This Server as the New Master Replica option to assign a Subordinate Reference as the new Master replica. Be careful!! If you select the View Replica Ring option and select a replica, you will see an additional option on that menu as well: Remove this server from the replica ring
`-S -Ad`	Loads `ndsrepair` with advanced schema operations. After entering the admin name and password, you are presented with the following additional menu items: Import schema from Tree Declare a new epoch
`-R Ad -OT`	Causes all obituaries to be timestamped. This can resolve issues with obituaries not advancing due to prolonged server outages.
`-R Ad -XK2`	Kill all eDirectory objects in this server's eDirectory database. This operation is used only to destroy a corrupt replica that cannot be removed in any other way.
`-R Ad -XK3`	Kill all external references in this server's eDirectory database. This operation is used to destroy all external references in a nonfunctioning replica. If the references are the source of the problem, eDirectory can then re-create the references as needed.

WARNING

The features described in this section canand willcause irreversible damage to your eDirectory tree if used improperly. These features should only be used only under the guidance of eDirectory professionals, such as the Novell Technical Services team, in order to resolve serious database issues. Always make a full backup of the eDirectory tree before using any of these features on a production tree. If you are going to use these features be sure you understand all the consequences before proceeding.

These advanced options are seldom used because they are needed for only the most serious of cases. However, it is nice to know they exist when you get into a jam.

It is highly recommended that you work with these switches in a test environment and carefully study the ramifications of these radical operations. Sometimes the cure can be worse than the problem.