Section 10.1. Net-SNMP | Essential SNMP, Second Edition

10.1. Net-SNMP

When you install the Net-SNMP package,^[] it creates a sample EXAMPLE.conf in the source directory. This file contains some great examples that demonstrate how to extend your agent. Read through it to see the types of things you can and can't do. We will touch on only a few of Net-SNMP's features: checking for any number of running processes (proc), executing a command that returns a single line or multiple lines of output (exec), and checking disk-space utilization (disk).

^[] Net-SNMP was formerly called UCD-SNMP and as a result, youll see references to the University of California at Davis in the code.

The main Net-SNMP configuration file can be found at $NET_SNMP_HOME/share/snmp/snmpd.conf, where $NET_SNMP_HOME is the directory in which you installed Net-SNMP. Here is the configuration file that we will use for the remainder of this section:

     # Filename: $NET_SNMP_HOME/share/snmp/snmpd.conf     # Check for processes running     # Items in here will appear in the ucdavis.procTable     proc sendmail 10 1     proc httpd     # Return the value from the executed program with a passed parm.     # Items in here will appear in the ucdavis.extTable     exec FileCheck /opt/local/shell_scripts/filecheck.sh /tmp/vxprint.error     # Multiline return from the command     # This needs its own OID     # I have used a subset of my registered enterprise ID (2789) within the OID     exec .1.3.6.1.4.1.2021.2789.51 FancyCheck /opt/local/shell_scripts/fancycheck.     sh \       /core     # Check disks for their mins     disk / 100000

Note that the system-monitoring OIDs presented in this section can be found in the UCD-SNMP-MIB.

Whenever you make changes to the Net-SNMP agent's configuration file, you can have it reread the configuration by sending the process an HUP signal:

     $ ps -ef | grep snmpd         root   12345     1  0   Nov 16 ?        2:35 /usr/local/bin/snmpd     $ kill -HUP 12345

Now let's look at the file itself. The first proc command says to check for the process sendmail. The numbers 10 and 1 define how many sendmail processes we want running at any given time (a maximum of 10 and a minimum of 1). The second proc command says that we want at least one httpd process running. To see what effect these commands have on our agent, let's look at an snmpwalk of the ucdavis.procTable (.1.3.6.1.4.1.2021.2):

     $ snmpwalk sunserver2 public .1.3.6.1.4.1.2021.2     enterprises.ucdavis.procTable.prEntry.prIndex.1 = 1     enterprises.ucdavis.procTable.prEntry.prIndex.2 = 2     enterprises.ucdavis.procTable.prEntry.prNames.1 = "sendmail"     enterprises.ucdavis.procTable.prEntry.prNames.2 = "httpd"     enterprises.ucdavis.procTable.prEntry.prMin.1 = 1     enterprises.ucdavis.procTable.prEntry.prMin.2 = 0     enterprises.ucdavis.procTable.prEntry.prMax.1 = 10     enterprises.ucdavis.procTable.prEntry.prMax.2 = 0     enterprises.ucdavis.procTable.prEntry.prCount.1 = 1     enterprises.ucdavis.procTable.prEntry.prCount.2 = 6     enterprises.ucdavis.procTable.prEntry.prErrorFlag.1 = 0     enterprises.ucdavis.procTable.prEntry.prErrorFlag.2 = 0     enterprises.ucdavis.procTable.prEntry.prErrMessage.1 = ""     enterprises.ucdavis.procTable.prEntry.prErrMessage.2 = ""     enterprises.ucdavis.procTable.prEntry.prErrFix.1 = 0     enterprises.ucdavis.procTable.prEntry.prErrFix.2 = 0

The agent returns the contents of the procTable. In this table, the sendmail and httpd process entries occupy instances 1 and 2. prMin and prMax are the minimum and maximum numbers we set for the sendmail and httpd processes.^[*] The prCount value gives us the number of processes currently running: it looks like we have one sendmail process and six httpd processes. To see what happens when the number of processes falls outside the range we specified, let's kill all six httpd processes and look at the procTable again (instead of listing the whole table, we'll walk only instance 2, which describes the httpd process):

^[*] When prMin and prMax are both 0, it says that we want at least one and a maximum of infinity processes running.

     $ snmpwalk sunserver2 public .1.3.6.1.4.1.2021.2     enterprises.ucdavis.procTable.prEntry.prIndex.1 = 1     enterprises.ucdavis.procTable.prEntry.prNames.1 = "httpd"     enterprises.ucdavis.procTable.prEntry.prMin.1 = 0     enterprises.ucdavis.procTable.prEntry.prMax.1 = 0     enterprises.ucdavis.procTable.prEntry.prCount.1 = 0     enterprises.ucdavis.procTable.prEntry.prErrorFlag.1 = 1     enterprises.ucdavis.procTable.prEntry.prErrMessage.1 = "No httpd     process running."     enterprises.ucdavis.procTable.prEntry.prErrFix.1 = 0

We had six httpd processes running and now, per prCount, we have none. The prErrMessage reports the problem, and the prErrorFlag has changed from 0 to 1, indicating that something is wrong. This flag makes it easy to poll the agent, using the techniques discussed in Chapter 8, and see that the httpd processes have stopped. Let's try a variation on this theme. If we set prMin to indicate that we want more than six httpd processes running, and then restart httpd, our prErrMessage is:

     enterprises.ucdavis.procTable.prEntry.prErrMessage.1 = "Too few     httpd running (# = 0)"

The next command in the configuration file is exec; this command allows us to execute any program and return the program's results and exit value to the agent. This is helpful when you already have a program you would like to use in conjunction with the agent. We've written a simple shell script called filecheck.sh that checks whether the file that's passed to it on the command line exists. If the file exists, it returns a 0 (zero); otherwise, it returns a 1 (one):

     #!/bin/sh     # FileName: /opt/local/shell_scripts/filecheck.sh     if [ -f $1 ]; then        exit 0     fi     exit 1

Our configuration file uses filecheck.sh to check for the existence of the file /tmp/vxprint.error. Once you have the filecheck.sh script in place, you can see the results it returns by walking ucdavis.extTable (.1.3.6.1.4.1.2021.8):

     $ snmpwalk sunserver2 public .1.3.6.1.4.1.2021.8     enterprises.ucdavis.extTable.extEntry.extIndex.1 = 1     enterprises.ucdavis.extTable.extEntry.extNames.1 = "FileCheck"     enterprises.ucdavis.extTable.extEntry.extCommand.1 =     "/opt/local/shell_scripts/filecheck.sh /tmp/vxprint.error"     enterprises.ucdavis.extTable.extEntry.extResult.1 = 0     enterprises.ucdavis.extTable.extEntry.extOutput.1 = ""     enterprises.ucdavis.extTable.extEntry.extErrFix.1 = 0

The first argument to the exec command^[*] in the configuration file is a label that identifies the command so that we can easily recognize it in the extTable. In our case, we used FileCheckthat's not a particularly good name, because we might want to check the existence of several files, but we could have named it anything we deemed useful. Whatever name you choose is returned as the value of the extTable.extEntry.extNames.1 object. Because the file /tmp/vxprint.error exists, filecheck.sh returns a 0, which appears in the table as the value of extTable.extEntry.extResult.1. You can also have the agent return a line of output from the program. Change filecheck.sh to perform an ls -la on the file if it exists:

^[*] See the EXAMPLE.conf configuration file introduced at the beginning of this chapter.

     #!/bin/sh     # FileName: /opt/local/shell_scripts/filecheck.sh     if [ -f $1 ]; then        ls -la $1        exit 0     fi     exit 1

When we poll the agent, we see the output from the script in the extOutput value the agent returns:

     enterprises.ucdavis.extTable.extEntry.extOutput.1 = \     "  16 -rw-rr--   1 root     other       2476 Feb 3 17:13 /tmp/vxprint.error."

This simple trick works only if the script returns a single line of output. If your script returns more than one line of output, insert an OID in front of the string name in the exec command.

Here's the next command from our snmpd.conf file:

     exec .1.3.6.1.4.1.2021.2789.51 FancyCheck /opt/local/shell_scripts/fancycheck.     sh \     /core

This command runs the program fancycheck.sh, with the identifying string FancyCheck. We won't bother to list fancycheck.sh; it's just like filecheck.sh, except that it adds a check to determine the file type. The OID identifies where in the MIB tree the agent will place the result of running the command. It needs to be in the ucdavis enterprise (.1.3.6.1.4.1.2021). We recommend that you follow the ucdavis enterprise ID with your own enterprise number, to prevent collisions with objects defined by other sources and avoid overwriting one of ucdavis's subtrees. Follow your enterprise number with another number to identify this particular command. In this case, our enterprise ID is 2789 and we assign the arbitrary number 51 to this command. Thus, the complete OID is .1.3.6.1.4.1.2021.2789.51.

Here are the results from walking the .1.3.6.1.4.1.2021.2789.51 subtree:

     $ snmpwalk sunserver2 public .1.3.6.1.4.1.2021.2789.51     enterprises.ucdavis.2789.51.1.1 = 1     enterprises.ucdavis.2789.51.2.1 = "FancyCheck"     enterprises.ucdavis.2789.51.3.1 =     "/opt/local/shell_scripts/fancycheck.sh /core"     ucdavis.2789.51.100.1 = 0     ucdavis.2789.51.101.1 = "-rw-rr--   1 root     other     346708 Feb 14 16:30 /core."     ucdavis.2789.51.101.2 = "/core:..ELF 32-bit MSB core file SPARC     Version 1, from 'httpd'."     ucdavis.2789.51.102.1 = 0

Notice that we have a few additional lines in our output. 2789.51.100.1 is the exit number, 2789.51.101.1 and 2789.51.101.2 are the output from the command, and 2789.51.102.1 is the errorFix value. These values can be useful when you are trying to debug your new extension. (Unfortunately, snmpwalk can give you only the numeric OID, not the human-readable name, because snmpwalk doesn't know what 2789.51.x is.)

The last task for Net-SNMP's extensible agent is to perform some disk-space monitoring. This is a great option that lets you check the availability of disk space and return multiple (useful) values. The disk option takes a filesystem mount point followed by a number. Here is what our entry looks like in snmpd.conf:

     # Check disks for their mins     disk / 100000

The definition of the disk option from UCD-SNMP-MIB.txt is "Minimum space required on the disk (in kBytes) before the errors are triggered." Let's first take a look on sunserver2 to see what the common df program returns:

     $ df -k /     Filesystem            kbytes    used   avail capacity  Mounted on     /dev/dsk/c0t0d0s0     432839   93449  296110    24%    /

To see what SNMP has to say about the disk space on our server, run snmpwalk against the ucdavis.diskTable object (.1.3.6.1.4.1.2021.9). This returns virtually the same information as the df command:

     $ snmpwalk sunserver2 public .1.3.6.1.4.1.2021.9     enterprises.ucdavis.diskTable.dskEntry.dskIndex.1 = 1     enterprises.ucdavis.diskTable.dskEntry.dskPath.1 = "/" Hex: 2F     enterprises.ucdavis.diskTable.dskEntry.dskDevice.1 =     "/dev/dsk/c0t0d0s0"     enterprises.ucdavis.diskTable.dskEntry.dskMinimum.1 = 100000     enterprises.ucdavis.diskTable.dskEntry.dskMinPercent.1 = -1     enterprises.ucdavis.diskTable.dskEntry.dskTotal.1 = 432839     enterprises.ucdavis.diskTable.dskEntry.dskAvail.1 = 296110     enterprises.ucdavis.diskTable.dskEntry.dskUsed.1 = 93449     enterprises.ucdavis.diskTable.dskEntry.dskPercent.1 = 24     enterprises.ucdavis.diskTable.dskEntry.dskErrorFlag.1 = 0     enterprises.ucdavis.diskTable.dskEntry.dskErrorMsg.1 = ""

As you can see, the Net-SNMP agent has many customizable features that allow you to tailor your monitoring without having to write your own object definitions. Be sure to review $NET_SNMP_HOME/share/snmp/mibs/UCD-SNMP-MIB.txt for complete definitions of all of Net-SNMP's variables. While we touched on only a few customizable options here, you will find many other useful options in the EXAMPLE.conf file that comes with the Net-SNMP package.