Cache Memory Considerations for eDirectory

     

Entry caching was added starting with NDS 8.73 and for eDirectory 8.5 and later. The entry (or record ) cache contains logical entries in the eDirectory tree rather than physical blocks of records from the eDirectory database. While traversing the database, eDirectory searches the entry cache stored in memory for the next requested entry. If the entry is found, a cache hit occurs, and the data can be retrieved from memory rather than from disk. If the data is not found in the entry cache, a cache fault occurs, and the entry record must be retrieved from the block cache (and from disk, if the required block is not in the block cache). The record is then added to the entry cache to prevent subsequent cache faults on the same entry record.

Although there is some redundancy between the block and entry caches, each cache is designed to boost performance for different operations. The block cache is most useful for update operations, whereas the entry cache is most useful for operations that browse the eDirectory tree by reading through entries, such as name resolution operations. Both the block and entry caches are useful in improving query performance. The block cache speeds up index searching, and the entry cache speeds up the retrieval of entries referenced from an index. (eDirectory indexes are discussed later in this chapter.)

NOTE

eDirectory caches information for the entire block ”which may be more than that asked for in the block cache. However, it will cache only the entry information asked for in entry cache. Stream files (such as login scripts) are only cached on demand and are cached in the file system cache and not in the eDirectory cache.

For instance, if eDirectory is asked for an octet attribute that is only 4 bytes in size , it will cache the entire block, usually 4KB, in the block cache. However, only the entry husk (which includes the entry's ID), not the entire entry, and attribute asked for are placed in the entry cache. (An entry husk is similar to a pointer in that it carries only enough information to link it to the real object.)


Distributing Memory Between the Entry and Block Caches

With an entry cache and a block cache, the total available memory for caching is shared between the two caches. The default is an equal division (50% each). To maintain the amount of block cache available in versions of NDS prior to 8.73, you need to double the total cache size for eDirectory (because of the entry cache you now have). If you use the cache to boost LDIF-import performance, for example, you can either double the total cache size or change the default cache settings (as discussed later in this chapter).

The more blocks and entries that can be cached, the better the overall performance will be. The ideal configuration is to cache the entire database in both the entry and block caches. However, this is usually not possible especially when you have extremely large databases. Generally , the rule of thumb is to try to get as close to a 1:1 ratio of block cache to DIB set size as possible. For the entry cache, you should try to get close to a 1:2 or 1:4 ratio. For the best performance, you should exceed these ratios where possible.

Calculating the Cache Limits

eDirectory provides two methods for controlling cache memory consumption: a dynamically adjusting limit (Dynamic Adjust mode) and a hard memory limit (Hard Limit mode). The Hard Limit mode is the method that versions of eDirectory prior to 8.5 use to regulate memory consumption. You set a hard memory limit by specifying one of the following:

  • A fixed number of bytes

  • A percentage of total physical memory

  • A percentage of available physical memory

When a hard memory limit is specified in percentages, it is translated to a fixed number of bytes based on the amount of memory at the time the setting is made.

NOTE

The hard memory limit size is for the block and entry caches combined.


The Dynamic Adjust mode causes eDirectory to periodically adjust its memory consumption in response to the change in memory usage by other processes. You specify the limit as a percentage of available physical memory. Using this percentage, eDirectory recalculates a new memory limit at fixed intervals. The new memory limit will be the percentage of physical memory available at the time.

Along with this percentage, you can set maximum and minimum thresholds. Such a threshold is the number of bytes that eDirectory will adjust to. It can be set as either the number of bytes to use or the number of bytes to leave available. The minimum threshold default is 16MB, and the maximum threshold default is 4GB.

If the minimum and maximum threshold limits are not compatible with one another, the minimum threshold limit is followed. For example, suppose you specified the following settings:

  • Minimum threshold: 8MB

  • Percentage of available physical memory to use: 75%

  • Maximum threshold: Keep 10MB available

When eDirectory adjusts its cache limit, there is 16MB of available physical memory. eDirectory calculates a new limit of 12MB (75% of 16MB). eDirectory then checks to see whether the new limit falls within the range of the minimum and maximum thresholds. In this example, the maximum threshold requires that 10MB remain available, so eDirectory lowers the limit to 6MB (leaving 10MB available). However, the minimum threshold is 8MB, so eDirectory resets the final limit to 8MB.

NOTE

Remember: Whenever there is a conflict resulting from the specified settings, the minimum threshold value gets priority over the maximum value.


Setting the Cache Limits

You can specify upper limits for the block cache and the entry cache separately. If no previously permanent cache settings are found when the DS agent (DSA) starts up, the cache defaults to a hard limit of 16MB for the first 10 minutes. Because the DSA usually loads when a server is restarted, this default behavior allows other applications to load and request system resources first. After 10 minutes, the behavior (by default) switches to Dynamic Adjust mode, based on the amount of available memory.

With the dynamically adjusting limit, you can also specify the interval length at which the memory limit is recalculated. The default interval is 15 seconds.

NOTE

The shorter the recalculation interval, the more the memory consumption is based on current conditions. However, shorter intervals are not necessarily better because the percentage recalculation will create more memory allocation and freeing.


REAL WORLD: The Cache Cleaner

Because of the multithreading nature of the DS module, NDS 8.73 and later create multiple versions of blocks and entries in the cache for transaction integrity purposes. For instance, if one thread is performing a read transaction while another is doing an update to the DIB , old versions of blocks changed by the writing thread are maintained on behalf of the reading thread. This occurs so that the reading thread will have a consistent view of the data during the lifetime of its transaction, even though modifications are taking place during that time.

Prior to eDirectory 8.5, this cached data did not get removed when it was no longer needed. eDirectory 8.5 and later add a background process that periodically browses the cache and cleans out older versions. This helps to minimize cache memory consumption. The default checking interval is 15 seconds and is configurable.

Do not confuse this cache cleaner background process ”for maintaining caching performance ”with the Flat Cleaner background process, which is used for DS database maintenance.


If the server does not have a replica, and no dynamic adjustments are specified, a hard memory limit of 16MB (with 8MB for the block cache and 8MB for the entry cache) is used. On the other hand, if the server contains one or more replicas, the default Dynamic Adjust mode uses a limit of 51% of available memory, with a minimum threshold of 8MB and a maximum threshold of keeping 24MB available to the operating system (that is, total available memory minus 24MB).

WARNING

Novell's tuning guides recommend very large caches (four times the DIB size). On machines with gigabytes of RAM, this is likely to lead to instability if it is not capped to 2GB. At the time of this writing, eDirectory is a 32-bit process that has a 4GB limit in its virtual address space. If you set the cache to 2GB or more, during peaks of high activity, the address space may exceed the operating system limits for a 32-bit process and lead to its abrupt termination (and result in core dumps or abends). On Linux 2.4 kernels , the process may run out of virtual address map entries and freeze.

When you're working on high-end tree deployments, you should set the cache to an absolute figure of 1.5GB or less. You should not use dynamic or hard settings.


You can use either the Hard Limit mode or the Dynamic Adjust mode, but you can only use one at a time because the two are mutually exclusive. The last method selected will always replace any prior settings. As discussed in the following sections, there are three ways you can set the cache limits:

  • By using the _NDSDB.INI file

  • By using the SET DSTRACE command

  • By using NDS iMonitor

Setting Cache Configuration via _NDSDB.INI

At startup, eDirectory looks for the _NDSDB.INI file in the directory where DIB files are stored. This file is a simple text file that can be created or modified with any text editor.

NOTE

The _NDSDB.INI file is read only when eDirectory starts up. Therefore, any changes made after eDirectory is running will not take effect until the eDirectory module has been restarted. On the other hand, no restart is necessary if the changes were made via DSTrace or NDS iMonitor.

Cache settings made via DSTrace and NDS iMonitor will automatically be populated in the _NDSDB.INI file.


The following is the syntax for cache memory settings for eDirectory:

 cache=  option1,option2,option3  ,... 

WARNING

Do not include any whitespace on either side of the = sign or between the options. Whitespace prevents the value from being set.


NOTE

None of the commands and options in _NDSDB.INI are case-sensitive, and they can be specified in any order.


This command sets a hard memory limit or dynamically adjusting limit. Multiple (optional) options may be specified, in any order, separated by commas. These are the allowable options:

  • DYN or HARD ” This option specifies to use a dynamically adjusting limit or a hard limit.

  • cache_in_bytes ” If just a number is specified, it is taken as the upper cache limit for the Hard Limit mode.

  • %: percentage ” This option specifies the percentage of available or physical memory to use for cache. (The default is 51%.)

  • AVAIL or TOTAL ” This option indicates whether the specified percentage value is based on the available physical memory or on total physical memory. (The default is AVAIL .)

  • MIN: bytes ” This option specifies the minimum number of bytes to use. (The default is 8388608, or 8MB.)

  • MAX: bytes ” This option specifies the maximum number of bytes to use.

  • LEAVE: bytes ” This option specifies the minimum number of bytes to leave. (The default is 25165824, or 24MB.)

NOTE

The MIN , MAX , and LEAVE values are ignored for a dynamically adjusting limit. Dynamic Adjust mode always bases its calculation on the available physical memory.


The following example sets a dynamically adjusting cache limit of 60% of available memory, with a minimum of 16MB:

 cache=DYN,%:60,MIN:16711680 

The following example sets a hard memory limit of 75% of available physical memory, with a minimum of 32MB:

 cache=HARD,%:75,MIN:33423360,AVAIL 

The following example sets a hard memory limit of 24MB:

 cache=25165824 

In addition to the cache settings, there are two settings that control the dynamic adjust interval and the interval at which the cache cleaner background process runs:

  • CacheAdjustInterval= seconds ” The default is 15 seconds.

  • CacheCleanupInterval= seconds ” The default is 15 seconds.

The final setting allows you to control the percentage split between the entry and block caches:

  • BlockCachePercent= percent ” The percent value indicates how much of the cache will be used for block caching. This needs to be between 0 and 100 (inclusive). A value of 70 means that 70% of cache memory will be used for the block cache, and the remaining 30% for the entry cache. The Default value is 50.

WARNING

Never set the BlockCachePercent value to zero because that would seriously degrade eDirectory performance. Novell also recommends that no more than 75% of the cache memory should be allocated to either the entry cache or the block cache for typical day-to-day operations.


TIP

Novell recommends setting the BlockCachePercent value to between 70% and 90%, depending on the proportion of updates in the total operations. And you should set it to 90% for operations such as bulk creations or deletions; you should set it to 50% if you do not expect too many update bursts.


eDirectory 8.7 introduced a new method for specifying the maximum dirty cache ( MaxDirtyCache ) and the low dirty cache ( LowDirtyCache ) for the eDirectory cache. By default, the value of MaxDirtyCache is unlimited (that is, using all of the available eDirectory cache; flush the dirty cache when this limit is reached) and the LowDirtyCache value is set to zero (that is, don't flush the dirty cache if it is less than this value). Setting the amount of dirty cache at any given instant below a particular value helps to even out the disk writing instead of burdening the checkpoint thread in the forced mode, which essentially writes the whole cache to the disk, thereby creating an I/O bottleneck.

NOTE

A checkpoint occurs when all dirty cache buffers are used and eDirectory must flush them to disk.


Normally, you don't have to set the MaxDirtyCache and LowDirtyCache values. However, if you are bulk-loading to populate, depopulate, or modify the DS objects, you should set them in the _NDSDB.INI file, as follows :

 MaxDirtyCache=  value_in_bytes  LowDirtyCache=  value_in_bytes  

Then you should restart the DSA in order for the settings to take effect. You should make sure to take them out after your bulk-load operation.

WARNING

Novell's testing suggests that for platforms other than HP /UX, setting MaxDirtyCache and LowDirtyCache is useful only for bulk-loading for less than 1.5 million objects. For higher values, there might be performance degradation if these values are changed from the defaults.


NOTE

There is no "hard value" recommended for the dirty cache settings because they very much depend on the server hardware. On most systems, the MaxDirtyCache value is between 1MB and 10MB, while 20MB may be used for fiber channel storage area networks. On HP /UX, setting MaxDirtyCache to 340MB and LowDirtyCache to 335MB worked well for all scenarios tested .

You can use the following procedure to determine what the MaxDirtyCache setting for your server may be. First, measure the random I/O write speed to the disk. Set the MaxDirtyCache value such that all modified buffers in a 3-minute interval can be flushed to the DIB volume in 10 seconds (10,000 ms) or less. Set the value of LowDirtyCache to about half that of MaxDirtyCache . For example, if the random write speed is 10 ms per block (4KB), you set MaxDirtyCache to (10,000ms x 4KB/10), or 4,000KB. Alternatively, you can use a simple trial-and-error method: You can set the MaxDirtyCache value to 5MB and observe the max update response time during a burst of updates. Then you can adjust this value upward until the response time is acceptable.


Setting Cache Usage via DSTrace

The syntax for setting eDirectory cache by using DSTrace is very similar to that used in the _NDSDB.INI file:

 SET DSTRACE=!M  option1,option2,option3  ,... 

DSTrace also uses the same list of options as used in _NDSDB.INI , with two exceptions and one addition:

  • !MB cache_in_bytes specifies the upper cache limit for the Hard Limit mode.

  • You cannot adjust CacheAdjustInterval , CacheCleanupInterval , BlockCachePercent , or any of the dirty cache parameters via DSTrace. They need to be specified using the _NDSDB.INI file.

  • By default, cache settings made using DSTrace are automatically written to the _NDSDB.INI file. The NOSAVE option prevents that. When this option is not specified, the settings are saved.

The following example sets a dynamically adjusting cache limit of 60% of available memory and a minimum of 16MB, and it saves the settings to _NDSDB.INI (because NOSAVE is not specified):

 SET DSTRACE=!MDYN,%:60,MIN:16711680 

The following example sets a hard memory limit of 75% of available physical memory and a minimum of 32MB, and it does not save the settings to _NDSDB.INI :

 SET DSTRACE=!MHARD,%:75,MIN:33423360,AVAIL,NOSAVE 

For instance, if the available system cache memory is 100MB, this command will allocate 75MB of that as a hard memory limit.

The following example sets a hard memory limit of 24MB:

 SET DSTRACE=!MB25165824 

NOTE

Configuring cache usage via DSTrace does not require a reset of DS or the server. Changes are effective immediately.


Setting Cache Usage via NDS iMonitor

An easier and more user -friendly, but not necessarily the fastest , method to configure cache settings is to use NDS iMonitor. The appropriate settings are made from the Database Cache Configuration page in NDS iMonitor (see Figure 16.1), which is accessed via the Database Cache link under Agent Configuration.

Figure 16.1. eDirectory database cache settings.
graphics/16fig01.jpg

Monitoring the Cache Statistics

Periodically, you should check the eDirectory cache statistics to ensure that the settings used are effective. You can easily determine the various cache hits and misses (which are generally referred to as cache faults ) by looking at the Database Cache statistics information from the Database Cache link under Agent Configuration in NDS iMonitor (see Figure 16.2).

Figure 16.2. eDirectory database cache information.
graphics/16fig02.jpg

Of particular interest in the cache statistics is the percentage listed next to Requests Serviced from Cache. This number reflects the cache efficiency. The percentage is calculated as (Total Hits) / (Total Hits + Total Faults), where Total Hits is the total for both the block and entry caches. Typically you want to keep this number somewhere in the 90s for best cache efficiency. If this number is below 90%, you might want to look at how much available memory you have in your server and perhaps change the way your cache is allocated (for example, change the percentage of block cache versus entry cache).

By comparing the reported DIB size to the summed maximum size of entry cache and block cache, you can determine how much of the DIB can be cached. The example shown in Figure 16.2 suggests that all of the DIB can be cached because the DIB size is a little over 1MB in size (1,280KB), while the summed maximum cache size is almost 14MB. In this case, you could safely reduce the amount of maximum cache limit and free up the extra RAM for other server processes.

NOTE

Refer to TID #10082323 for a detailed description about the items on the Database Cache page in NDS iMonitor.


TIP

To ensure optimal performance, you should configure your system based on cache hit and cache fault indicators, database size, and memory available. You should not expect to cache the entire database unless you have a small DIB set. You should not expect to see zero cache faults, and you shouldn't expect the faults to be at zero to have optimal performance in eDirectory.




Novell's Guide to Troubleshooting eDirectory
Novells Guide to Troubleshooting eDirectory
ISBN: 0789731460
EAN: 2147483647
Year: 2003
Pages: 173

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net