Dealing with Attributes with Many Values

LDAP schema allows attributes to be either single valued or multivalued. However, in Active Directory and ADAM, multivalued attributes are really divided into two different categories: standard attributes and link value paired attributes. Link value paired attributes have a DN syntax (DS-DN, DN-With-Binary, DN-With-String) and have what is known as a linkID defined (see Chapter 7 for more details on the schema and linkIDs). DN syntax attributes without linkIDs and all other syntaxes have the standard multivalue behavior. In practice, nearly all DN attributes in the default schema are link valued, but they do not have to be.

The key difference here is the maximum limit on the number of values that are allowed for a multivalued attribute. Standard attribute values have a maximum number of elements, which is around 800 in Windows 2000 Active Directory and 1,300 in Windows 2003 and ADAM. Note that the actual maximum size may vary, but these numbers are as high as we can ever expect. However, link value paired attributes do not have a defined maximum number of values. In Windows 2000 Active Directory, Microsoft recommends against DN link value paired attributes with more than 5,000 values because a variety of interesting problems can result, but the directory did not enforce this limit and many people exceeded it with mixed success. These issues were resolved in Windows 2003 and ADAM, so link value paired attributes are effectively unlimited again.

The problem for LDAP programmers is that Active Directory and ADAM limit the maximum number of attribute values that can be returned from a single attribute in a single search. Like paged searches (discussed in Chapter 4), this is done to improve the performance of the directory and to protect it from potential denial of service attacks. The maximum number of values that can be returned is determined by the MaxValRange LDAP policy, which is set to 1,000 in Windows 2000 and 1,500 in Windows 2003 and ADAM. This essentially means that PropertyValueCollection or ResultPropertyValueCollection will never have more than the MaxValRange number of values, regardless of how many values the attribute actually contains.

Given that the maximum number of allowed attribute values for standard attribute types is lower than these two limits, we see that standard attributes are not subject to this limitation and that only link value paired attributes are affected.

We could probably ignore this little detail altogether in most cases, except for the fact that one very important link value paired attribute, the member attribute, which determines membership in groups, runs into the MaxValRange limit with some frequency. If we are reading a group's membership, we probably would like to know all of the members, not just the first 1,500, so we have to do some extra work.

How to Use Range Retrieval in SDS

To get all of the attribute values, they must be retrieved using a process called range retrieval, where subsets of the attribute values are retrieved in subsequent requests. Instead of referencing the attribute by its normal name, we use a special syntax to specify the range of values we want to retrieve:

mvaAttributeToRead;range={low index}-{high index}

Here, low index and high index are numbers representing the range of results to be returned. Note that if high index - low index exceeds the MaxValRange policy, only a maximum of MaxValRange values will be returned. It is also acceptable to specify * for high index to always get back the maximum allowed number of results.

A typical algorithm will repeat the query using low index = high index + 1 from the previous query. We know that we have retrieved all of the attribute values when the high index returned is * rather than a numeric value.

For example, we might request this:

member;range=0-*

...and receive this:

member;range=0-1499

In this case, we know that the attribute has at least 1,500 members, and probably more. However, we might also receive this in reply to the same request:

member;range=0-*

In this case, we can assume that the attribute has fewer values than the maximum range value, so additional searches are not required. This search contains all of the values. The tricky part about using range retrieval is that the high index of the attribute that we must look for will change to * when we request a range that exceeds the remaining number of values. However, we can use this to our advantage to detect when we have arrived at the last search in the range retrieval. Listing 6.8 demonstrates this technique.

Range retrieval works on only a base-level search. It is possible to use either DirectoryEntry or DirectorySearcher to do range retrieval, but our tests have shown that DirectorySearcher seems to perform a little bit better and is more straightforward to use.

Listing 6.8. Range Retrieval Using DirectorySearcher

public static ArrayList RangeExpansion(
 DirectoryEntry entry,
 string attribute)
{
 ArrayList al = new ArrayList(5000);
 int idx = 0;

 //zero based index, so less 1
 int step = entry.Properties[attribute].Count - 1;

 string range = String.Format(
 "{0};range=-",
 attribute
 );

 string currentRange = String.Format(range, idx, step);

 DirectorySearcher ds = new DirectorySearcher(
 entry,
 String.Format("({0}=*)", attribute),
 new string[] { currentRange },
 System.DirectoryServices.SearchScope.Base
 );

 bool lastSearch = false;
 SearchResult sr = null;

 while (true)
 {
 if (!lastSearch)
 {
 ds.PropertiesToLoad.Clear();
 ds.PropertiesToLoad.Add(currentRange);

 sr = ds.FindOne();
 }

 if (sr != null)
 {
 if (sr.Properties.Contains(currentRange))
 {
 foreach (object dn in sr.Properties[currentRange])
 {
 al.Add(dn);
 idx++;
 }

 //our exit condition
 if (lastSearch)
 break;

 currentRange = String.Format(
 range,
 idx,
 (idx + step)
 );

 }
 else
 {

 //one more search
 lastSearch = true;
 currentRange = String.Format(range, idx, "*");
 }
 }
 else
 break;
 }
 return al;
}

We might call this helper method using some code like this:

ArrayList members = RangeExpansion(group, "member");

An interesting and possibly confusing aspect to range retrieval with DirectorySearcher is that the attribute name in ResultPropertyValueCollection includes the range syntax. Thus, we would have to look for member;range=0-* and not simply member. We should also note that Listing 6.8 works equally well with .NET 1.x and 2.0.

Basics of Writing Attribute Values