Performance Analysis
| So far in this chapter, we've focused on techniques to monitor your directory service and notify you when a problem occurs. This type of reactive monitoring is essential to the success of your directory service, but it's only half of the picture. Your directory service will be more successful if you also monitor the performance of your directory servers, and use the raw performance data to anticipate necessary configuration changes and upgrades to your service. When you initially deploy your directory service, you implicitly or explicitly perform some initial capacity planning. You decide how many servers to deploy; how much memory, disk, and network I/O capacity each server should have; and so on. You make these decisions on the basis of your knowledge of the directory-enabled applications that will initially be deployed, and perhaps build in some extra capacity to accommodate anticipated growth. Once your directory service is deployed, however, you will find that you need to revisit your assumptions about how your service will be used. The only effective way to do this is to observe the service in operation, and this means obtaining and analyzing usage and performance data, and drawing conclusions that help you in capacity planning. In this chapter we focus on
Obtaining Raw Usage DataThere are two sources for raw usage and performance data: directory server access logs and operating system logs. Directory Server Access LogsIf you use Netscape Directory Server, you can configure it to record every client access to a file. This file is usually called access and is contained in the logs directory beneath the server instance directory. If your server is not configured to record activity in the access log, you can use Netscape Server Administrator to enable such recording. Netscape Directory Server records the following events in its access log:
Each of these events is recorded on a separate line in the access log, marked with the date and time it occurred. Access logs are stored on a disk local to the directory server machine. You'll want to periodically move them to a central location where you perform analysis, rather than performing analysis on the server itself. One common method of collecting directory server access logs is to periodically copy all of the access logs from your servers to a central location and delete the original copy on each server. Once the logs have been copied to the central location, you can perform whatever log analysis you wish, then delete the raw data logs. This task is made easier if the directory server software supports automatic log rotation, in which the active log file is closed and renamed when it reaches a predetermined size or age. With log rotation, you don't need to worry about the fact that the server is writing to the access log you're copying; you simply copy the rotated logs. Other server software may require that you stop the server, rename the logs, and restart the server in order to rotate the log files. Consult your server documentation for specifics. Operating System LogsMany operating systems maintain usage counters and provide utilities for recording historical values of those usage counters. For example, Solaris provides the sar utility, which can record many usage parameters of the operating system, and can generate tables and graphs of the data. Microsoft Windows systems provide the perfmon utility, which can perform many of the same tasks . You should arrange to sample operating system parameters on a periodic basis (say, once every five minutes) and record the raw data to a log. Periodically you can produce a summary report of the raw data and delete the raw data log. The summary reports should be saved indefinitely so that you can review the long- term trends. For example, the sar utility on Solaris can sample multiple operating system parameters. Typically, sar is run from the cron facility and samples all relevant parameters, storing the results in a file. The sar utility can later be run to extract information from that file. Digesting and Analyzing Raw Performance DataAfter you have the raw usage logs for your directory server in a central location, you need to make sense of the data. One way to do this is to divide the log file into portions of equal time, and summarize the activity for each period. For example, you might produce a summary of activity for each hour (the sample Perl log analysis utility included in this chapter does exactly this). For looking at longer-term trends, you can always combine hourly summaries to produce daily, weekly, or monthly summaries as you desire . The Perl script ldap_analyzer.pl , shown in Listing 19.4, can read a Netscape Directory Server access log and writes hourly summaries of activity using comma-separated value (CSV) format. CSV files can be read into spreadsheet programs like Microsoft Excel for analysis. Listing 19.4 The ldap_analyzer.pl Script #!/usr/local/bin/perl # # ldap_analyzer.pl # # Analyze Netscape Directory Server access logs # # Usage: perl ldap_analyzer.pl < log-file # # This script reads a Netscape Directory Server access # log and outputs a CSV (comma-separated value) file that can # be imported into a spreadsheet or database program for further # analysis. The output file consists of a header line that gives # a description of each column, then one row of values for every # hour covered by the access log. The columns and their meanings # are: # # Date/Time - The date and time of the period covered. # Total Connections - The total number of connections handled. # SSL Connections - The number of SSL connections handled. # Operations - The total number of LDAP operations handled. # Bind Operations - The total number of bind operations handled. # V2 Binds - The number of LDAP version 2 bind operations handled. # V3 Binds - The number of LDAP version 3 bind operations handled. # Anonymous Binds - The number of anonymous bind operations handled. # This number is always zero for Netscape Directory # Server 4.x and earlier log formats. # Non-Anonymous Binds - The number of nonanonymous bind operations # handled. This number is always zero for Netscape # Directory Server 4.x and earlier log formats. # Unbinds - The number of unbind operations handled. # Search Operations - The total number of search operations handled. # Base-level Searches - The number of base-level searches handled. # Onelevel Searches - The number of one-level searches handled. # Subtree Searches - The number of subtree searches handled. # Unindexed Searches - The number of searches that had to be serviced # without the aid of an index. # Avg Search ET - Average elapsed time for all search operations. # Add Operations - Number of add operations handled. # Avg Add ET - Average elapsed time for all add operations. # Modify Operations - Number of modify operations handled. # Avg Mod ET - Average elapsed time for all modify operations. # Delete Operations - Number of delete operations handled. # Avg Del ET - Average elapsed time for all delete operations. # ModifyDN Operations - Number of modify DN (rename) operations handled. # Avg ModDN ET - Average elapsed time for all modify DN operations. # # Compute an average. sub compute_avg { my($total, $count, $avg); $total = $_[0]; $count = $_[1]; if ($count > 0) { $avg = $total / $count; } else { $avg = 0; } return $avg } # Print summary information for the time period. sub print_summary { my($now); $now = $_[0]; if ($now ne "xxx") { $av_srch_et = &compute_avg($tot_srch_et, $num_srchs); $av_add_et = &compute_avg($tot_add_et, $num_adds); $av_mod_et = &compute_avg($tot_mod_et, $num_mods); $av_del_et = &compute_avg($tot_del_et, $num_dels); $av_moddn_et = &compute_avg($tot_moddn_et, $num_moddns); print "$now,"; print "$num_conns,"; print "$num_ssl_conns,"; print "$num_ops,"; print "$num_binds,"; print "$num_v2_binds,"; print "$num_v3_binds,"; print "$num_anon_binds,"; print "$num_non_anon_binds,"; print "$num_unbinds,"; print "$num_srchs,"; print "$num_base_srchs,"; print "$num_onelevel_srchs,"; print "$num_subtree_srchs,"; print "$num_unindexed_srchs,"; printf "%.4f,", $av_srch_et; print "$num_adds,"; printf "%.4f,", $av_add_et; print "$num_mods,"; printf "%.4f,", $av_mod_et; print "$num_dels,"; printf "%.4f,", $av_del_et; print "$num_moddns,"; printf "%.4f", $av_moddn_et; print "\n"; } &init_interval_counters(); } # Print the header for the CSV file. sub print_header { # Print header print "Date/Time,"; print "Total Connections,"; print "SSL Connections,"; print "Operations,"; print "Bind Operations,"; print "V2 Binds,"; print "V3 Binds,"; print "Anonymous Binds,"; print "Non-anonymous Binds,"; print "Unbinds,"; print "Search Operations,"; print "Base-level Searches,"; print "Onelevel Searches,"; print "Subtree Searches,"; print "Unindexed Searches,"; print "Avg Search ET,"; print "Add Operations,"; print "Avg Add ET,"; print "Modify Operations,"; print "Avg Mod ET,"; print "Delete Operations,"; print "Avg Del ET,"; print "ModifyDN Operations,"; print "Avg ModDN ET"; print "\n"; } # Reinitialize all counters. sub init_interval_counters { $num_conns = 0; # Number of non-SSL connections handled $num_ssl_conns = 0; # Number of SSL connections handled $num_ops = 0; # Number of operations handled $num_binds = 0; # Number of bind operations handled $num_v2_binds = 0; # Number of LDAPv2 binds handled $num_v3_binds = 0; # Number of LDAPv3 binds handled $num_anon_binds = 0; # Number of anonymous binds handled $num_non_anon_binds = 0; # Number of non-anonymous binds handled $num_unbinds = 0; # Number of unbind operations handled $num_srchs = 0; # Number of search operations handled $num_base_srchs = 0; # Number of base-level search operations handled $num_onelevel_srchs = 0; # Number of onelevel search operations handled $num_subtree_srchs = 0; # Number of subtree search operations handled $tot_srch_entries = 0; # Total number of search entries handled $num_unindexed_srchs = 0; # Total number of search entries handled $tot_srch_et = 0; # Total elapsed time spent on searches $num_adds = 0; # Number of add operations handled $tot_add_et = 0; # Total elapsed time spent on adds $num_mods = 0; # Number of modify operations handled $tot_mod_et = 0; # Total elapsed time spent on mods $num_dels = 0; # Number of delete operations handled $tot_del_et = 0; # Total elapsed time spent on dels $num_moddns = 0; # Number of moddn (rename) operations handled $tot_moddn_et = 0; # Total elapsed time spent on moddns $num_entries = 0; # Number of entries sent to clients } &print_header(); $curr_date_time = "xxx"; while (<>) { # Check for date/time in correct format if (/^\[(\d{2})\/(\w{3})\/(\d{4}):(\d{2}):(\d{2}):(\d{2}) [+-](\d{4})\]/) { $day = ; $month = ; $year = ; $hour = ; $minute = ; $second = ; $gmt_offset = ; # Detect when we have rolled over to a new hour $date_time = "${month}/${day}/${year} ${hour}:00"; if ($curr_date_time ne $date_time) { &print_summary(${curr_date_time}); $curr_date_time = $date_time; } # Now look for specific types of log format # BIND operation if (/.* conn=(\d+) op=(\d+) BIND dn="(.*)" method=(\d+) version=(\d+).*$/) { $conn = ; $op = ; $binddn = ; $bindmethod = ; $ldapversion = ; $num_ops++; $num_binds++; if ($ldapversion == "2") { $num_v2_binds++; } elsif ($ldapversion == "3") { $num_v3_binds++; } } # UNBIND operation if (/.* conn=(\d+) op=(\d+) UNBIND$/) { $num_ops++; $num_unbinds++; } # SEARCH operation if (/.* conn=(\d+) op=(\d+) SRCH base="(.*)" scope=(\d+) filter="(.*)"$/) { $conn = ; $op = ; $base=; $scope=; $filter=; $num_ops++; $num_srchs++; # What is the search scope? if ($scope == 0) { $num_base_srchs++; } elsif ($scope == 1) { $num_onelevel_srchs++; } elsif ($scope == 2) { $num_subtree_srchs++; } } # ADD operation if (/.* conn=(\d+) op=(\d+) ADD dn="(.*)"$/) { $conn = ; $op = ; $dn=; $num_ops++; $num_adds++; } # DEL operation if (/.* conn=(\d+) op=(\d+) DEL dn="(.*)"$/) { $conn = ; $op = ; $dn=; $num_ops++; $num_dels++; } # MOD operation if (/.* conn=(\d+) op=(\d+) MOD dn="(.*)"$/) { $conn = ; $op = ; $dn=; $num_ops++; $num_mods++; } # MODRDN operation if (/.* conn=(\d+) op=(\d+) MODRDN dn="(.*)"$/) { $conn = ; $op = ; $dn=; $num_ops++; $num_modrdns++; } # RESULT log entries. There are several variants. if (/.* conn=(\d+) op=(\d+) RESULT err=(\d+) tag=(\d+) nentries=(\d+) etime=(\d+).*$/) { $conn = ; $op = ; $err = ; $tag=; $nentries = ; $etime = ; $num_results++; # BIND result if (/.* conn=(\d+) op=(\d+) RESULT err=(\d+) tag=97 nentries=(\d+) etime=(\d+)( dn=(. Listing 19.5 shows what output from the Perl script looks like. The output is terse and designed to be imported into a spreadsheet for further analysis. For example, the graph in Figure 19.9 was generated with Microsoft Excel. It shows the number of bind operations serviced by the server during a 24-hour period. Listing 19.5 Sample Output from the ldap_analyzer.pl ScriptDate/Time,Total Connections,SSL Connections,Operations,Bind Operations,V2 Binds,V3 Binds, Figure 19.9. Bind Operations Serviced in a 24-Hour Period In Figure 19.9, notice that the number of bind operations increases every four hours. In this example, which is drawn from a real server installation, an automated synchronization process runs every four hours and performs multiple bind operations, resulting in the usage spikes shown in the figure. Graphing the CSV files generated by the ldap_analyzer.pl script can help you visualize the types of load your directory server is handling. Drawing ConclusionsNow that you have obtained the raw usage data and digested it into a format that is easily understood , what conclusions can you draw? Spotting ProblemsIf your directory server is experiencing performance problems, you might try graphing the number of operations serviced by your server, and compare the result to a graph of the average elapsed time for operations. If your server is simply being overwhelmed by client load, you'll see a close correspondence between periods of high usage and poor performance. On the other hand, if your directory performance degrades without a corresponding increase in client load, you may want to examine operating system parameters such as CPU utilization, memory utilization, and disk throughput. You may discover, for example, that another process on the server is competing for machine resources. Spotting TrendsIf you have been collecting usage data for a while, you can graph usage over longer time periods and start to look for long-term trends. For example, if you have hourly usage data for the previous six months, you might sum the usage data on a weekly basis and examine how directory usage is changing over time. If you have an idea of the maximum throughput your server can handle (perhaps you measured this empirically during you directory pilot), you can predict the point in time when your server will become overloaded. Be aware, however, that the deployment of a new application may cause an unanticipated increase in directory usage. |
+))*$/) { $dn = ; $tot_bind_et += $etime; if (length($dn) > 0) { if (${dn} =~ "\"\"") { $num_anon_binds++; } else { $num_non_anon_binds++; } } } # SEARCH result if (/.* conn=(\d+) op=(\d+) RESULT err=(\d+) tag=101 nentries=(\d+) etime=(\d+)( notes=( 
EAN: 2147483647
Pages: 242
- The Effects of an Enterprise Resource Planning System (ERP) Implementation on Job Characteristics – A Study using the Hackman and Oldham Job Characteristics Model
- Data Mining for Business Process Reengineering
- Intrinsic and Contextual Data Quality: The Effect of Media and Personal Involvement
- A Hybrid Clustering Technique to Improve Patient Data Quality
- Relevance and Micro-Relevance for the Professional as Determinants of IT-Diffusion and IT-Use in Healthcare