Creating LDIF for Benchmarks

The MakeLDIF 1.3 program is an LDIF structure and data generation tool for testing the performance of the Sun ONE Directory Server version 5.2. This program provides the ability to generate LDIF files that can be imported into the Sun ONE Directory Server for testing purposes. The information that MakeLDIF uses to generate the entries is specified using a template file that can be customized to produce LDIF files that can be used in a test harness for benchmark performance validation tests.

The MakeLDIF program is available for download. See "Obtaining the Downloadable Files for This Book" on page xxvii.

The MakeLDIF Program

The MakeLDIF program is a Java-based utility for generating LDIF files. It can be used to generate sample data to import into an LDAP directory server, such as the Sun ONE Directory Server software. Although other utilities exist for this purpose (for example, dbgen.pl), MakeLDIF offers a number of powerful features not available in other tools:

• MakeLDIF is highly customizable. LDIF files are generated based on templates defined by the user . This means that it is easy to include custom attributes, model complex DIT structures, and generate realistic data.

• User-defined templates can use a wide range of tags that make it possible to dynamically generate different kinds of data. In addition, an API is available that makes it possible to define custom tags to handle different kinds of processing not offered by the standard tags.

• MakeLDIF is written in Java. This allows it to be used on a wide range of platforms without recompilation like native code, but does not suffer from the lack of large file support in most Perl interpreters.

• MakeLDIF makes it possible to provide additional information while it is in the process of generating the LDIF file. For example, you can write an additional file with the list of the DNs of all the entries created, or you can generate a list of potential search filters that can be used to access the data once it has been imported.

MakeLDIF can easily be used as a standalone utility for generating LDIF data for a number of purposes. However, because it has been designed for use with the SLAMD distributed load generation engine (covered later in this chapter), it offers a number of features that make it especially useful for generating data to use in a directory server for running many of the SLAMD jobs that work with LDAP directory servers.

Installing MakeLDIF

The only requirement for using MakeLDIF is that a Java runtime environment be present on the target system. MakeLDIF has been designed to work with Java versions 1.2 or higher, although Java 1.4 or higher is recommended for best performance.

The following procedure assumes that you have obtained the downloadable MakeLDIF-1.3.tar.gz file (see "Obtaining the Downloadable Files for This Book" on page xxvii).

To Install MakeLDIF

1. Copy the MakeLDIF-1.3.tar.gz file onto your system, and into the location in which you wish to install MakeLDIF .

2. Uncompress the MakeLDIF-1.3.tar.gz file as shown:

    $  gunzip MakeLDIF-1.3.tar.gz  

   Note

   If the gunzip command is not available on the system, the -d option with the gzip command:

     $ gzip -d MakeLDIF-1.3.tar.gz  

3. Extract the files from the MakeLDIF-1.3.tar file as shown:

   This creates a MakeLDIF directory under the current working directory, and all of the associated files are unpacked into that directory.

    $  /usr/bin/tar -xvf MakeLDIF-1.3.tar  

Running MakeLDIF

The following examples show you how to run MakeLDIF in various ways, depending on your environment.

The minimum required command line to run MakeLDIF is:

 $  java -jar MakeLDIF.jar MakeLDIF -t   template   -o   output_filename  

template is the path to the template file that describes the way in which the LDIF file should be generated and output_filename is the path to the output file that is created.

In the following example, the MakeLDIF command uses the template defined in the file example.template and writes the output into the file example.ldif .

 $  java -jar MakeLDIF.jar -t example.template -o example.ldif  

If the Java executable is not currently in your path, you can specify the absolute path to the Java executable you wish to use.

 $  /usr/java/bin/java -jar MakeLDIF.jar -t   template   -o   output_filename  

If for some reason the Java runtime environment does not support the use of the --jar option, you can instead place the MakeLDIF.jar file in the Java classpath and run it like in the following example:

 $  java -cp MakeLDIF.jar MakeLDIF -t   template   -o   output_filename  

Although only the -t and -o options were illustrated , a number of command-line arguments may be used with MakeLDIF to customize its behavior. The full set of supported arguments is shown in TABLE 9-1.

Table 9-1. MakeLDIF Options

Option	Description
-f filename	Specifies the name of the file containing first names to use in the entry generation process. By default, the MakeLDIF program uses a file named first.names in the current working directory. If a custom first name file is used, it must only contain the first name information with one first name per line.
-l filename	Specifies the name of the file containing last names to use in the entry generation process. By default, the MakeLDIF program uses a file named last.names in the current working directory. If a custom last name file is used, it should contain only last name information with one last name per line.
-t filename	Specifies the template file that is used to determine how to create the LDIF file. This is a required parameter. The format of the template file is discussed in a later section.
-o filename	Specifies the name of the output LDIF file to generate.
-d filename	Specifies the name of a file to which the DNs of generated entries are written. This is useful for a number of utilities that can make use of a DN file.
-b filename	Specifies the name of a file to which bind information for the generated entries is written. The format requires one user per line, with the DN followed by a tab and the password for the user with that DN.
-L filename	Used to specify the name of a file to which login information for the generated entries is written. The format is one user per line, with the login ID followed by a tab and the password for that user.
-i attr	Used to specify the name of the attribute that should be used as the login ID. By default, the uid attribute is used as the login ID.
-F filename	Specifies that search filters constructed from the generated entries are written to the specified file. You need to use the -T option to specify which filter types to generate.
-T type	Specifies the filter types to generate. The format of {type} should be the name of the attribute followed by a colon and a comma-separated list of filter types to create for that attribute. Allowable filter types are eq and sub . Example: -T uid:eq -T cn:eq,sub .
-s value	Specifies the seed to use for the random number generator. If no seed is specified, a value is chosen based on the current time. If a positive integer value is given, that is used as the seed to the random number generator. Using the same template file and the same random seed should consistently produce exactly the same LDIF file.
-m value	Specifies the maximum number of entries that should be written to a single LDIF file. After this number of entries has been written, the current LDIF file is closed and a new file created with a .2 extension. This counter increments sequentially for each new file created.
-x value	Specifies the maximum number of entries that should be created for each template below each branch. This can be used to create an example of the LDIF file that would be generated from the provided template to verify that the template syntax is valid.
-w	Specifies that long lines should be wrapped (folded). By default, the entire value of an attribute is written on a single line, which is useful if the LDIF file needs to be processed by another application, but can be difficult to read for long values.
-M	Specifies that a different filter file should be created for each index type for each attribute for which filter information is to be collected.
-S	Specifies that branch entries should be skipped when writing the LDIF information to the requested output file.
-D	Specifies that MakeLDIF should operate in debug mode, which causes it to provide additional debugging information for some errors that may occur while using MakeLDIF .
-H	Displays usage information for the MakeLDIF program.
-V	Displays version information for the MakeLDIF program.

The Template Format

As mentioned earlier, MakeLDIF uses template files to define the way in which the LDIF files should be generated. This is a much more powerful approach than those taken by other LDIF generators, because it allows for a great deal of flexibility without the need to alter any code to produce the desired result.

The template files contain three sections:

• Global replacement variables

• Branch definitions

• Template definitions

The remainder of this section discusses each separately.

Customizing the Template File for MakeLDIF

The MakeLDIF program uses a template file to customize LDIF files. The use of the template file makes it possible to customize the location and kinds of entries that are generated by the MakeLDIF program. Two kinds of entries can be written into a template file: branch entries and template entries . It is also possible to define variables that may be used throughout the branch and template entries.

Global Replacement Variables

Replacement variables define strings of text that are referenced later in the template file and that are automatically replaced as each line is read into memory.

MakeLDIF branch entries example:

 define suffix=dc=example,dc=com 

In this example, define creates a variable called suffix with a value of dc=example,dc=com (the first equal sign is the delimiter between the variable name and the value). Once that definition is made, any occurrence of [suffix] in the template file is automatically replaced with dc=example,dc=com . As noted earlier, this replacement occurs as each line of the token file is read into memory, which means that it is possible to perform these replacements in branch definitions and other places where other tokens are not parsed.

Note

The parsing algorithm used by MakeLDIF uses the [ character to denote the beginning of a replacement variable usage. If it is necessary to have the [ character included in the template file, but not in the context of a variable reference, it should be escaped with the backslash character ( \[ ). This signifies that the bracket is not to be treated as part of a variable reference. The backslash immediately before the opening bracket is removed.

Branch Entries

Branch entries define a hierarchical branch of a directory server DIT to include in the LDIF file and may be associated with zero or more templates that specify the kinds of entries to create beneath that branch in the hierarchy. A branch entry with no subordinate entries generated by a template might look like:

 branch: dc=example,dc=com 

This creates an entry that looks like this:

 dn: dc=example,dc=com objectclass: top objectclass: domain dc: example 

The basic structure of the entry is defined by the RDN attribute of dc specified in the DN of the branch definition. MakeLDIF automatically associates the dc RDN attribute with the domain object class. It has similar default definitions for other common RDN attributes in branch entries:

• o -- Creates an entry with the organization object class

• ou -- Creates an entry with the organizationalUnit object class

• c -- Creates an entry with the country object class

• l -- Creates an entry with the locality object class

In addition, it is possible to use any other kind of RDN attribute for a branch entry. For branch entries with an RDN attribute other than one specified above, the entry is created with the extensibleObject object class.

It is possible to customize the information contained in a branch entry by adding the additional information below the branch definition.

Example:

 branch: dc=example,dc=com description: This is the description. 

The preceding example produces an entry that looks like:

 dn: dc=example,dc=com objectclass: top objectclass: domain dc: example description: This is the description. 

Note

All of the branch entries defined so far have created only a single entry in the directory with limited control over the information contained in that entry. However, this does not have to be the case. If one or more subordinateTemplate values are specified, it is possible to create additional entries below this branch.

Example:

 branch: ou=People,dc=example,dc=com subordinateTemplate: person:1000 

The preceding example causes the ou=People,dc=example,dc=com entry to be created, but also creates 1000 entries below it based on the person template. If you wish to have multiple kinds of entries created below a single branch, you can specify each kind of entry with a different subordinateTemplate definition.

For example:

 branch: ou=People,dc=example,dc=com subordinateTemplate: person:1000 subordinateTemplate: certificatePerson: 100 

Branch entries are not limited to just one subordinateTemplate definition. It is possible to specify multiple subordinateTemplate definitions by simply including them on separate lines of the branch definition. The example above creates 1000 entries based on the person template and an additional 100 entries based on the certificatePerson template:

Template Entries

The heart of the MakeLDIF template file are the actual template definitions. Template definitions define the structure of the entries that will be generated. They specify the set of attributes to include in the entries and the types of values that those attributes should contain. The specification of the values is handled through the use of tags that are parsed by MakeLDIF and replaced with the appropriate values for those tags.

A sample template entry might look like this:

 template: person rdnAttr: uid objectclass: top objectclass: person objectclass: organizationalPerson objectclass: inetOrgPerson givenName:  first  sn:  last  cn: {givenName} {sn} initials: {givenName:1}{sn:1} uid: {givenName}.{sn} mail: {uid}@[maildomain] userPassword: <random:alphanumeric:8> telephoneNumber: <random:telephone> homePhone: <random:telephone> pager: <random:telephone> mobile: <random:telephone> employeeNumber: <sequential:100000> street: <random:numeric:5> <file:streets> Street l: <file:cities> st: <file:states> postalCode: <random:numeric:5> postalAddress: {cn}${street}${l}, {st} {postalCode} description: This is the description for {cn}. 

The actual tags that may be included in a template definition are discussed in a later section. However, this example does illustrate some of the flexibility that MakeLDIF offers when generating LDIF data.

The tags contained in this template are parsed and replaced with information appropriate for the specified tag. For example, an entry created using the preceding template might look like this:

 dn: uid=Neil.Wilson,ou=People,dc=example,dc=com objectclass: top objectclass: person objectclass: organizationalPerson objectclass: inetOrgPerson givenName: Neil sn: Wilson cn: Neil Wilson uid: Neil.Wilson mail: [e-mail] userPassword: d82fk32n telephoneNumber: 823-630-8157 

At the top of the template definition are two lines that provide information about the template itself and are not actually included in entries created from this template. The first line specifies the name of the template. This is the name that is referenced in the subordinateTemplate lines of the branch definition. The second line specifies the name of the attribute that should be used as the RDN attribute for the entry. This attribute must be assigned a value lower in the template definition, and the way in which the value is assigned must ensure that the value is unique.

Note

It is possible to use multivalued RDNs by separating the attribute names with a plus sign, like rdnAttr: uid+employeeNumber .

If multivalued RDNs are used, the combination of RDN attribute values must be unique, but it is possible for one or more of the attributes in the RDN to be nonunique as long as the combination is never duplicated .

In addition to the template and rdnAttr lines, it is also possible to include one or more subordinateTemplate lines. This makes it possible to include dynamically generated entries below other entries that have been dynamically generated (that is, if each user entry has one or more entries below it), and can allow for some rather complex hierarchies. While there is no limit placed on this level of nesting, it is important to ensure that no recursive loops are created by having a subordinateTemplate that either directly or indirectly creates additional entries using the same template.

Template definitions also support the concept of inheritance through the use of the extends keyword. For example, entries generated from the template definition look like this:

 template: certificatePerson rdnAttr: uid extends: person userCertificate;binary:: <random:base64:1000> 

Include all of the attributes defined in the person template as well as userCertificate;binary with the specified format. Multiple inheritance is allowed (by including multiple lines with the extends keyword), but as with the subordinateTemplate keyword it is important not to create a recursive loop in which a template could either directly or indirectly extend itself.

Template File Tags

In order to ensure that MakeLDIF provides the ability to generate LDIF files that can be used to simulate a wide variety of deployments, a large number of tags have been defined for use in template definitions. This section describes the standard set of tags that may be used in a MakeLDIF template file. It is possible to use custom tags in template files as well, and the process for developing custom tags is defined in a later section.

Standard Replacement Tags

The tags that may be used in a template file include those in TABLE 9-2:

Table 9-2. Supported Tags for MakeLDIF Template Entries

Tag	Description
<presence:{percent}>	Indicates how likely the associated attribute value is to be included in any given entry generated from this template. The value specified for { percent } should be an integer between 0 and 100, inclusive. This should only be used with attributes that are not required by the object classes used in the entry. There should also be something included in the value of the attribute that will be present in entries chosen to include this attribute value. The presence tag itself is replaced with an empty string.
<ifpresent:{attribute}>	Indicates that this attribute value is to be included in an entry only if the entry contains one or more values for the attribute { attribute }. If this feature is used, { attribute } must be assigned a value in the template before the line that checks for its presence. The ifpresent tag itself is replaced with an empty string.
<ifpresent:{attribute}:{value}>	Indicates that this attribute value is to be included in an entry only if the entry contains the attribute { attribute } with a value of { value }. If this feature is used, { attribute } must be assigned a value in the template before the line that checks for its presence. The ifpresent tag itself is replaced with an empty string.
<ifabsent:{attribute}>	Indicates that this attribute value is to be included in an entry only if the entry does not contain any values for the attribute { attribute }. If this feature is used, { attribute } must be assigned a value in the template before the line that checks for its presence. The ifabsent tag itself is replaced with an empty string.
<ifabsent:{attribute}:{value}>	Indicates that this attribute value is to be included in an entry only if the entry does not contain attribute { attribute } with a value of { value }. If this feature is used, { attribute } must be assigned a value in the template before the line that checks for its presence. The ifabsent tag itself is replaced with an empty string.
<first>	Replaces the tag with a value from the first name file. If both a first name and a last name are included in an entry, the combination of the first and last name is guaranteed to be unique. That is, no two entries in the same LDIF file have the same combination of first and last name values. To guarantee this, it is necessary to ensure that the first name file does not contain any duplicate values, the last name file does not contain any duplicate values, and the first and last name values are used in their entirety. (You cannot use the substring feature of the attribute value replacements of the form { givenName:5 } discussed below.)
<last>	Replaces the tag with a value from the last name file. If both a first name and a last name are included in an entry, the combination of the first and last name is guaranteed to be unique. That is, no two entries in the same LDIF file have the same combination of first and last name values. To guarantee this, it is necessary to ensure that the first name file does not contain any duplicate values, the last name file does not contain any duplicate values, and the first and last name values are used in their entirety. (You cannot use the substring feature of the attribute value replacements of the form { sn:5 } discussed below.)
<dn>	Replaces the tag with the distinguished name (DN) of the current entry. For this to work properly, the RDN attribute for the entry must be assigned a value on an earlier line of the template.
<parentdn>	Replaced with the DN of the parent entry.
<ancestordn:{depth}>	Replaces the tag with the DN of the entry's ancestor at the specified depth. A depth of 1 returns the DN of the entry's immediate parent, a depth of 2 returns the DN of the entry's grandparent, and so on. If the entry does not have an ancestor at the specified depth, the <ancestordn:{depth}> tag is replaced with an empty string.
< exec :{command}>	Replaces the tag with the information sent to standard output when the command { command } is executed on the system. Because this requires a separate process to be invoked for each entry created using this template, using this tag can make the LDIF generation process proceed much more slowly than if the exec tag is not used.
<exec:{command},{arg1},{arg2},...,{argN}>	Replaces the tag with the information sent to standard output when the command { command } is executed on the system with the provided set of arguments. Because this requires a separate process to be invoked for each entry created using this template, using this tag can make the LDIF generation process proceed much more slowly than if the exec tag is not used.
<random:chars:{ characters }:{length}>	Replaces the tag with { length } characters from the character set { characters }. The character set { characters } can contain any character other than the colon.
<random:chars:{characters}:{minLength}:{maxLength}>	Replaces the tag with between { minLength } and { maxLength } (inclusive) characters from the character set { characters }. The character set { characters } can contain any character other than the colon.
<random:alpha:{length}>	Replaces the tag with a string of { length } randomly chosen alphabetic characters.
<random:alpha:{minLength}:{maxLength}>	Replaces the tag with a string of between { minLength } and { maxLength } (inclusive) randomly chosen alphabetic characters.
<random:numeric:{length}>	Replaces the tag with a string of { length } randomly chosen numeric digits.
<random:numeric:{minValue}:{maxValue}>	Replaces the tag with an integer value between { minValue } and { maxValue } (inclusive). The integer value is not padded with leading zeroes, so if that is desired the <random:numeric:{minValue}:{maxValue}:{minLength}> tag should be used.
<random:numeric:{minValue}:{maxValue}:{minLength}>	Replaces the tag with an integer value between { minValue } and { maxValue } (inclusive). If the integer value chosen contains less than { minLength } digits, it is padded with leading zeroes to the required minimum length.
<random:numeric:{length}>	Replaces the tag with { length } randomly chosen alphanumeric characters.
<random:numeric:{minValue}:{maxValue}>	Replaces the tag with between { minLength } and { maxLength } (inclusive) randomly chosen alphanumeric characters.
<random:hex:{length}>	Replaces the tag with { length } randomly chosen hexadecimal digits.
<random:hex:{minLength}:{maxLength}>	Replaces the tag with between { minLength } and { maxLength } (inclusive) randomly chosen hexadecimal digits.
<random:base64:{length}>	Replaces the tag with { length } randomly chosen characters from the base64 character set. If { length } is not a multiple of 4, the generated value is padded with equal signs so that the total length is a multiple of 4 as per the base64 specification.
<random:base64:{minLength}:{maxLength}>	Replaces the tag with between { minLength } and { maxLength } (inclusive) randomly chosen characters from the base64 character set. If the selected length is not a multiple of 4, the generated value is padded with equal signs so that the total length is a multiple of 4 as per the base64 specification.
<random:telephone>	Replaces the tag with a string of randomly chosen numeric digits in the form 123-456-7890 . This uses a U.S.-style telephone number, but it is possible to generate telephone numbers in other formats by combining other kinds of tags (for example, to generate a telephone number in the UK format, you could use +44 <random:numeric:4> <random:numeric:6> ).
<random:month>	Replaces the tag with the name of a randomly chosen month.
<random:month:{length}>	Replaces the tag with the first { length } characters from the name of a randomly chosen month.
<guid>	Replaces the tag with a GUID (globally unique identifier) value containing hexadecimal digits in the form 12345678-90ab-cdef-1234- 567890abcdef . GUID values should be unique within the same LDIF file.
<sequential>	Replaces the tag with a sequentially increasing numeric value. The first entry generated using this tag has a value of , the second a value of 1 , and so on. Sequential counters are maintained on a per-attribute and per-template basis, so it is possible to use multiple sequential counters in different attributes of the same entry without impacting each other. It is also possible to use sequential counters for the same attribute in different templates without impacting each other. However, it is not possible to use multiple sequential counters for the same attribute in the same template without them impacting each other.
<sequential:{initial}>	Replaces the tag with a sequentially increasing numeric value, starting at specified initial value { initial }. The first entry generated using this tag has a value of { initial }, the second a value of {initial}+1 , and so on. Sequential counters are maintained on a per-attribute and per-template basis, so it is possible to use multiple sequential counters in different attributes of the same entry without impacting each other. It is also possible to use sequential counters for the same attribute in different templates without impacting each other. However, it is not possible to use multiple sequential counters for the same attribute in the same template without them impacting each other.
<list:{value1},{value2},...,{valueN}>	Replaces the tag with a randomly chosen value from the provided comma-delimited list. Each value in the list provided has an equal chance of being selected.
<list:{value1}:{weight1},{value2}:{weight2},...,{valueN}:{weightN}>	Replaces the tag with a randomly chosen value from the provided comma-delimited weighted list. The weight associated with each list item determines how likely that value is to be chosen. A list item with a weight of 2 is twice as likely to be chosen as an item with a weight of 1. The weights specified must be positive integers.
<file:{filename}>	Replaces the tag with a randomly chosen value from the specified file. There should be one value per line of the file. It is not possible to assign weights to the values in the file, but a value can be weighted artificially by including it in the file multiple times. For example, a value that appears in the file three times is three times as likely to be chosen as a value that appears only once.
<base64:{value}>	Replaces the tag with the base64-encoded representation of { value }. The value { value } is converted into a byte array using the UTF-8 character set, and that byte array is base64 encoded.
<base64:{charset}:{value}>	Replaces the tag with the base64-encoded representation of { value }. The value { value } is converted into a byte array using the rules of the Java character set { charset }, and that byte array is base64 encoded. See the Java API documentation to determine the names of the character sets that may be used.
<loop:{lowerBound}:{upperBound}>	Creates ( {upperBound} - {lowerBound} + 1 ) copies of this line with this tag replaced in each copy with a sequentially incrementing number starting at { lowerBound } in the first copy, ( {lowerBound} + 1 ) in the second copy, and so on, so that in the last copy this tag is replaced with { upperBound }. It is possible to include multiple loop tags on the same line (even with different { lowerBound } values), but only the first tag is used to determine the number of copies to create.
<custom:{class}>	Replaces the tag with the value generated by invoking the custom tag whose implementation is provided in the class { class }. The provided class name { class } must be fully qualified (that is, including the package name if applicable ), and that class must exist in the classpath used to run MakeLDIF .
<custom:{class}:{arg1},{arg2},...,{argN}>	Replaces the tag with the value generated by invoking the custom tag whose implementation is provided in the class { class } with the specified argument list. The provided class name { class } must be fully qualified (that is, including the package name if applicable), and that class must exist in the classpath used to run MakeLDIF .

Attribute Value Reference Tags

In addition to the standard replacement tags listed in TABLE 9-2, it is possible to use tags that reference the values of other attributes in the same entry. These tags are called attribute value reference tags and they may be used by simply enclosing the name of the desired attribute in curly braces. When these tags are encountered in the template, they are replaced with the value of the specified attribute. If the specified attribute has not been assigned a value for the current entry, this tag is replaced with an empty string.

For example, consider the following excerpt from a template:

 givenName: <first> sn: <last> uid: {givenName}.{sn} cn: {givenName} {sn} mail: {uid}@example.com 

If the value chosen for the first name is Neil and the last name is Wilson , the following LDIF output would result:

 givenName: Neil sn: Wilson uid: Neil.Wilson cn: Neil Wilson mail: [e-mail] 

Note

In order for this to work properly, it is necessary to assign a value to an attribute before it may be referenced in this manner. If, for example, the definition of the mail attribute had appeared before the definition of the uid attribute, the resulting email address would have been @example.com because the {uid} tag would not have a value and therefore would have been replaced with an empty string.

It is also possible to place a colon after the name of the attribute followed by a positive integer value {length} . This causes at most the first {length} characters of the value from the specified attribute to be used instead of the entire value. For example, the template excerpt:

 givenName: <first> sn: <last> initials: {givenName:1}{sn:1} 

Would produce the following LDIF for a first name of Neil and a last name of Wilson :

 givenName: Neil sn: Wilson initials: NW 

If the specified {length} is longer than the value of the named attribute, the entire value of the attribute is used and no padding is added.

Tag Evaluation Order

The order in which tags are evaluated while parsing a template definition is important because changing the evaluation order can change the way that the output is produced. Although it is not possible for the end user to change this evaluation order, it is important to understand how template definitions are processed so that LDIF files are generated in the appropriate manner.

In most cases, it is not possible to nest tags. That is, the output of one tag cannot be used as input to another. For example, consider the following MakeLDIF tag evaluation order:

 description: <random:alpha:<random:numeric:5:10>> 

One might hope that this would first evaluate the <random:numeric:5:10> tag to create a numeric value between 5 and 10 (for example, 7) and then evaluate the outer tag as <random:alpha:7> . However, this is not the case. Because of the way that MakeLDIF parses template definitions, this fails because it tries to interpret <random> as an integer value. Fortunately, this is not a problem in most cases because enough variations of the standard replacement tags have been provided to deal with this. For example, to achieve the desired result attempted by the above template line, you can instead use:

 description: <random:alpha:5:10> 

For the purposes of this discussion, MakeLDIF parses template files in the following order:

1. All standard replacement tags other than base64 , custom , and loop .

2. All attribute value reference tags.

3. The base64 standard replacement tags ( <base64:{value}> and <base64:{charset}:{value}>) .

4. The custom standard replacement tags ( <custom:{class}> and <custom:{class}: {arg1},{arg2},...,{argN}>) .

5. The loop standard replacement tag ( <loop:{lowerBound}:{upperBound}>) .

Based on this order of operations, any tag at one level may be used as input to any tag at a higher level. For example, the following is legal because base64 standard replacement tags are evaluated after attribute value reference tags:

 description:: <base64:{givenName}> 

Defining Custom Tags

One of the benefits of MakeLDIF is the large number of tags it provides that can be used to customize the process of creating LDIF files. However, even with the large set of tags provided, it may be desirable to extend the functionality even further. This is possible through the use of custom tags.

Custom tags can be easily defined by creating a Java class that extends the abstract CustomTag class. This class defines three methods :

• public void initialize() -- Provides the ability to perform any one-time initialization that may be necessary when this tag is first created. This is an optional step, and by default no initialization is performed.

• public void reinitialize () -- Provides the ability to perform additional initialization every time the template is used to start processing a new branch. This is also optional, and by default no reinitialization is performed.

• public String generateOutput(String[] tagArguments) -- Generates the output that should appear whenever this custom tag is invoked in the template. The provided set of arguments can be used to customize the output as necessary. This method must be implemented in all custom tags.

A Sample Custom Tag Implementation

 # MakeLDIF Custom Tag Usage # /** * This class provides an implementation of a MakeLDIF custom tag that will be * used to calculate the sum of all the integer values provided as arguments to * the tag. */ public class SumCustomTag extends CustomTag /** * Performs any necessary one-time initialization that should be performed * when this custom tag is first created. In this case, no initialization is * performed. */ public void initialize() { // No implementation required. } /** * Performs any initialization that should be performed each time the LDIF * generation starts working on a new branch (e.g., to reset any internal * variables that might have been in use). In this case, no reinitialization * is performed. */ public void reinitialize() { // No implementation required. } { /** * Parses the list of arguments, converts the values to integers, and totals * those values. * * @param tagArguments The arguments containing the numeric values to be * totaled. * * @return The string representation of the total of all the argument values. */ public String generateOutput(String[] tagArguments) { int sum = 0; for (int i=0; i < tagArguments.length; i++) { sum += Integer.parseInt(tagArguments[i]); } return String.valueOf(sum); } } 

Using the Example Custom Tag

Once the custom tag class has been implemented and compiles properly, it may be used by specifying that class in a custom tag in the template file. For example, to use the custom tag we just implemented, something like the following could be placed in the template file:

 cn: <custom:SumCustomTag:1,2> 

When MakeLDIF is run using a template that contains this definition, the generateOutput method of the SumCustomTag class is invoked and produces output containing:

 cn: 3 

Because static values were provided as arguments to the tag, the output is exactly the same every time. This is not all that useful in this case, but it doesn't have to be that way. As indicated earlier, custom tags are parsed after most other kinds of tags. This means that it is possible to use something like:

 cn: <custom:SumCustomTag:<random:numeric:5>,<random:numeric:5>> 

There, the output from other tags is used as arguments to the custom tag. In this case, each <random:numeric:5> tag generates a five-digit integer, and the custom tag adds those values together. Unlike the previous example, the output in this case may be different for each entry because the arguments are randomly chosen values. Of course, even then this particular custom tag is not very useful, but it is a simple example that can be used as the foundation for creating more useful custom tags for real-world purposes.

Using and Automating MakeLDIF

Once you understand how to use the MakeLDIF application, you can automate the process of generating test data for a benchmark.

As a summary, the MakeLDIF application is recommended for use as an LDIF structure and data generation tool for testing the performance of the Sun ONE Directory Server 5.x (or any other directory server for that matter). MakeLDIF provides the ability to generate LDIF files that can be imported into the Sun ONE Directory Server 5.x environment. The information needed to generate the entries is specified using a template file that can be customized to produce LDIF files for use as a test harness for directory server benchmark performance validation tests.

TABLE 9-3 lists examples of the layout and files that can be used in benchmark performance validation tests. Specific examples follow the table.

Table 9-3. BenchMark Performance Validation Tests 250k

Sun ONE Directory Server 5.x Software	Files and LDIF Templates (250 Kbyte Entries)
Makefiles	Makefile
MakeLDIF User Defined Template	Template
Sample LDIF Data	example-data.ldif

Sample Root Makefile

 # Copyright  2003 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # #ident "@(#)Makefile    1.0 08/01/03 SMI" # # Root Makefile for 250k.ldif file # OUTDIR=bp250k-data TARGET=bp250k all: output_dir $(TARGET) clean:   output_dir          @-if [ -d $(OUTDIR) ]; then \          /usr/bin/rm -rf $(OUTDIR); fi bp100k: @echo Building... @-#java -cp MakeLDIF.jar MakeLDIF -t bp250k.template -o $@.ldif @echo Moving LDIF Files... @echo Please wait... @-/usr/bin/mv $@.ldif $(OUTDIR) output_dir:         @-if [ ! -d $(OUTDIR) ]; then mkdir -p $(OUTDIR); fi 

Makefile and Generating the Filter File

 # Copyright  2003 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # #ident "@(#)Makefile    1.0 08/01/03 SMI" # # Root Makefile for 250k.ldif file # OUTDIR=bp250k-data TARGET=bp250k all: output_dir $(TARGET) clean: output_dir        @-if [ -d $(OUTDIR) ]; then \        /usr/bin/rm -rf $(OUTDIR); fi bp100k: # # Generate the filter file with (caution, its VERY slow) # @echo Building... @-#java -cp MakeLDIF.jar MakeLDIF -t bp250k.template -o $@.ldif \ -T uid:eq -T sn:eq -T givenName:eq -T cn:eq -T cn:sub -F $@filter- file.ldif @echo Moving LDIF Files... @echo Please wait... @-/usr/bin/mv $@.ldif $(OUTDIR) @-#/usr/bin/mv $@-filter-file.ldif $(OUTDIR) output_dir:         @-if [ ! -d $(OUTDIR) ]; then mkdir -p $(OUTDIR); fi 

The following code box shows an example of running the Makefile without the filter option and the output:

 Building 250k LDIF File... Processed 250000 entries 250000 entries written to bp250k.ldif Moving LDIF file to bp250k-data location... Done. 

The following code box shows an example of running the Makefile with the filter option and the output:

 Building 250k LDIF File... Processed 250000 entries 250000 entries written to bp250k.ldif Writing filters to bp205k-filter-file.ldif Wrote 250000 equality filters for uid Wrote 250000 equality filters for givenname Wrote 250000 equality filters for sn Wrote 2479 substring filters for cn Moving LDIF files to bp250k-data location... Done. 

The previous examples showed how you can automate the process of generating LDIF data sets. It is also possible to automate the process of building of larger data sets using a global Makefile.

Example:

 # Copyright  2003 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # #ident "@(#)Makefile     1.0 08/01/03 SMI" # # Global Makefile # SUBDIRS= bp250k-src bp500k-src bp1M-src bp5M-src \ bp10M-src bp25M-src bp50M-src all: #all clean:         @for i in $(SUBDIRS) ; \         do \           cd $$i ; \           echo "==================================" ; \           echo " Current directory: $$i" ; \           echo "==================================" ; \           if [ -f makefile ] ; \           then \             $(MAKE) -f makefile $@ ; \           else \             $(MAKE) -f Makefile $@ ; \           fi ; \           cd .. ; \         done 

Below is an example of running the global Makefile and the output:

 ==================================  Current directory: bp250k-src ================================== Building 250k LDIF File... Processed 250000 entries 250000 entries written to bp250k.ldif Moving LDIF file to bp250k-data location... Done. ==================================  Current directory: bp500k-src ================================== Building 500k LDIF File... Processed 500000 entries 250000 entries written to bp500k.ldif Moving LDIF file to bp500k-data location... Done. ==================================  Current directory: bp1M-src ==================================  Building 1M LDIF File... Processed 1000000 entries 250000 entries written to bp1M.ldif Moving LDIF file to bp1M-data location... Done. etc... etc...