8.7 Using EVM

This section will not attempt to cover every possible function within the EVM subsystem; instead, it will discuss some of the interesting uses we have discovered and how they relate to the TruCluster Server product. For more in depth analyses of EVM, we suggest the Tru64 UNIX System Administration Guide, or the following reference pages:

evmget(1)
evmpost(1)
evmsort(1)
evm(5)
EvmFilter(5)
evmstart(8)
evmviewer(8)
evminfo(1)
evmshow(1)
evmwatch(1)
EvmEvent(5)
evmreload(8)
evmstop(8)

A complete list of reference pages can be obtained by using the "man -k" command, or our section-based sman script (see Appendix B for where to get it).

8.7.1 How Can I Find Out Which Events are Registered with EVM?

You can retrieve the registered events by using the evmwatch command with the "-i" switch.

Note

Prior to Tru64 UNIX version 5.1, the "-A" command option did not exist for the evmwatch and evmget commands. The "-A" switch added to the evmwatch command automatically invokes evmshow, whereas the evmget command automatically invokes the evmsort command.
If you are using Tru64 UNIX version 5.0A, use the following command in place of the "-A" command option to evmwatch:

 # evmwatch –i | evmshow

 # evmwatch –A –i

Since that command gives us a lot of events, we may want to limit our search to just network-related events.

With V5.0A, use "evmwatch –i –f" "[name *.net]" "| evmshow".

With V5.1 (and newer), use "evmwatch -A -i -f" "[name *.net]".

 # evmwatch -A -i -f "[name *.net]" NIFF: node sheridan detected a failed network connection on network via interface NIFF: node sheridan has detected an available network connection on network via interface NIFF: node sheridan has declared a connectivity alert with network via interface NetRAIN: node sheridan detected a failed network connection on network via interface NetRAIN: node sheridan has detected an available network connection on network via interface NetRAIN: node sheridan has declared a connectivity alert with network via interface

Okay, so we narrowed it down, but what if we want to see the name of the event instead of the format text? The EvmEvent(5) reference page defines the "Standard Data Items" that can be used to create a "show-template". A "show-template" can be created with the "-t" switch to the evmget, evmshow, evmsort, and evmwatch commands (or if you use the same show-template, you can set the EVM_SHOW_TEMPLATE environment variable with your favorite format).

With V5.0A, use: "evmwatch –i –f" "[name *.net]" | evmshow –t "@name"

With V5.1 (or newer), use: "evmwatch -A -i -f" "[name *.net]" -t "@name"

 # evmwatch -A -i -f "[name *.net]" -t "@name" sys.unix.hw.net.niff.down sys.unix.hw.net.niff.up sys.unix.hw.net.niff.alert sys.unix.hw.net.netrain.down sys.unix.hw.net.netrain.up sys.unix.hw.net.netrain.alert

8.7.2 How Can I Wait for an Event to Happen?

Waiting for events to occur on the system or cluster is easily accomplished using the evmwatch command without the "-i" switch.

With V5.0A (or newer):

 # evmwatch | evmshow

With V5.1 (or newer):

 # evmwatch -A

Note that until an event occurs, no output will appear. Using the evmwatch command works similarly to using the "tail –f" command to display the output from the /var/adm/messages file. The difference, however, is that with the evmwatch command you have the power of filtering and formatting the events and seeing the events as they occur from a variety of sources on not just one system but on every member in a cluster!

8.7.3 How Can I Retrieve Events that Have Occurred?

If the event was logged, it can be retrieved with the evmget command.

With V5.0A (or newer):

 # evmget | evmshow

With V5.1 (or newer):

 # evmget -A

Remember that you can filter the events as well.

 # evmget -A -f "[name *.hw] AND [prio > 400]" Correctable error reporting state changed Correctable error reporting state changed

By default, only the event text is reported. To see additional information, you can use a "show-template".

 # evmget –A -f "[name *.hw] AND [prio > 400]" -t "@timestamp [@priority] @name @@" 22-Mar-2001 09:22:56 [700] sys.unix.binlog.hw.correctable_rpt_switch Correctable error reporting state changed 22-Mar-2001 09:30:16 [700] sys.unix.binlog.hw.correctable_rpt_switch Correctable error reporting state changed

8.7.4 How Can I See which Channels are Configured?

 # evminfo –lc syslog binlog misclog evmlog

8.7.5 How Can I Get Events from One Particular Channel?

You can limit the scope of your search to a particular channel using the "-C" flag to evmget. This was added to EVM in Tru64 UNIX version 5.1.

 # evmget -A -C binlog -f "[age < 1d]" -t "@timestamp @name @@" 28-Mar-2001 07:36:13 sys.unix.binlog.op.timestamp System timestamp 28-Mar-2001 07:37:41 sys.unix.binlog.hw.scsi SCSI event 28-Mar-2001 07:37:41 sys.unix.binlog.hw.scsi SCSI event 28-Mar-2001 07:37:41 sys.unix.binlog.hw.scsi SCSI event 28-Mar-2001 16:17:42 sys.unix.binlog.op.timestamp System timestamp

To retrieve information from more than one channel, use multiple "-C" switches.

 # evmget -A -C binlog –C syslog -f "[age < 1d]" -t "@timestamp @name @@"

8.7.6 How Can I Format the "timestamp" Received from the Event?

As you have seen, you can retrieve the time and date using the "@timestamp" filter, but this gives you both the time and date. If you only want a specific portion to the timestamp, you can use evmshow and specify the time format with the "-T" switch. The format you use is the same format used for the date(1) command.

 # evmget -f "[prio >= 700]" | evmshow -T "%A (%D) %H:%M" Thursday (03/22/01) 09:22 Correctable error reporting state changed Thursday (03/22/01) 09:30 Correctable error reporting state changed

8.7.7 How Can I Send an Event?

You can send (or post) an event using the evmpost command. The evmpost command receives its input from an input stream (or file), converts the input to EVM's binary format, and then sends the event to the EVM daemon for distribution to subscribers.

 # echo "event { name tcrhb.timeEvent.overDue }" | evmpost

This simply posts the "tcrhb.timeEvent.overdue" event to EVM using the default values defined in the event template file (see section 8.4 for more information).

You can also pass variables to the event as defined in the template for the event. For example, in the "tcrhb.timeEvent.overdue" event, we defined the variable "dueDate" which was defined as a string with a default value of "Today".

 event {     name         tcrhb.timeEvent.overDue     format       "TCR Handbook: this book is overdue. $dueDate is the due date."     priority     400     var          {name dueDate type STRING value "Today"} }

In order to post the event with an alternate value to the "dueDate" variable, we could do the following:

 # echo "event { name tcrhb.timeEvent.overDue var {name dueDate value \"'date'\"}}" \ | evmpost

If you do not want to type the event on the command line to post it, you can place it in a file and have evmpost read the file instead.

 # evmpost myEvent

You can also post simple messages to EVM with the "-a" (administrator) and "-u" (user) switches.

 # evmpost –a "Starting the upgrade for the C++ compiler" –p 201 # evmpost –u "I am here!"

Note

If you are using the csh shell, you will need to escape the "!" character like "\!". This is necessary to be able to print the "!" character. Omitting the "\" will result in an ": Event not found." error. Note further that the "Event" in the error message does not refer to an EVM event but is the C shell's way of saying that the command you were trying to recall was not found. The "!" is how you recall a command from your history list.

8.7.8 How Can I Send an Event from a Script?

You can post an event from a script by using the evmpost command from within your script. There is also a small excerpt of a script in the evmpost(1) reference page.

8.7.9 How Can I Format the Event Information I Receive?

Formatting event information can be done using the "-t" switch to the evmwatch, evmget, or evmshow commands. The "-t" enables you to create a "show-template" for the command. The format can use any valid EVM data item as defined in the EvmEvent(5) reference page. When using a data item in the "show-template", prefix it with "@". The data items are listed in uppercase in the EvmEvent(5) reference page, but you can use lowercase. Note that the "@@" is the actual event text with any variable values, referred to in the text, substituted in place of their references.

 # evmget -A -t "@pid [@priority] @name: @@" -f "[prio > 600]" - [700] sys.unix.binlog.hw.correctable_rpt_switch: Correctable error reporting state changed - [700] sys.unix.binlog.hw.correctable_rpt_switch: Correctable error reporting state changed 302 [700] sys.unix.hw.net.niff.down: NIFF: node sheridan detected a failed network connection on network 192.168.0.69 via interface tu0

If you find that you are constantly using the same format, you may wish to define the format in the EVM_SHOW_TEMPLATE environment variable.

 # export EVM_SHOW_TEMPLATE="@pid [@priority] @name: @@" # evmget -A -f "[name *.hw]" 21667 [300] sys.unix.hw.dev_base_name_changed.tape._hwid.123: Device base name changed from unknown to tape0 (HWID=123) 21667 [300] sys.unix.hw.dev_base_name_changed.tape._hwid.124: Device base name changed from unknown to tape1 (HWID=124)

As of V5.1A, newline (\n) and tab (\t) characters can be used to increase the readability of your event output. For example, you can insert a newline into the event output by using a "\n".

 # evmget -A -f "[age < 1d]" \ > -t "\n@timestamp - @event_id [@priority]\n@name\n@@" | more 26-Sep-2002 03:43:59 - 344680 [200] sys.unix.clu.drd.server_leave._hwid.52 DRD: Removed (unmapped) DRD server molari

Using the "%" character next to a filter keyword allows you set a minimum field width. For example, to set the minimum field width of a cluster_event field to five you can use the following command.

 # evmget -A -f "[age < 1d]" \ > -t "@cluster_event%5 - @name" | more True  - sys.unix.hw.net.niff.alert False - sys.unix.evm.logger.log_closed False - sys.unix.evm.logger.log_started False - sys.unix.evm.mark False - sys.unix.binlog_chan.binlog_size False - sys.unix.binlog.hw.scsi True  - sys.unix.clu.drd.server_leave._hwid.52 False - sys.unix.binlog.hw.scsi True  - sys.unix.clu.drd.server_add._hwid.52 True  - sys.unix.clu.drd.new_accessnode._hwid.52 True  - sys.unix.clu.drd.new_accessnode._hwid.52 ...

8.7.10 How Can I Find Out Which Filters are Available?

Filter keywords are defined in the EvmFilters(5) reference page. You can find filters defined in the filter files located in the /usr/share/evm/filters (system) and /var/evm/adm/filters (site defined) directories.

 # ls /var/evm/adm/filters /usr/share/evm/filters /usr/share/evm/filters: binlog.evf     evm.evf     sys.evf    syslog.evf /var/evm/adm/filters: <no output>

Also, the TruCluster Server Handbook web site (see Appendix B for the URL) contains a Korn Shell script called evf(1) that can be used to find out which filters are available in a filter file.

 evf [-v] [-d FilterFileDir] [file1 file2...filen]

The evf script will use the current working directory, /var/evm/adm/filters, and /usr/share/evm/filters as the default search path if either the "-d" switch or the EVF_DIR_PATH environment variable is not used.

 # evf binlog /   [no event filter files found] /var/evm/adm/filters   [no event filter files found] /usr/share/evm/filters    binlog.evf        @binlog:binlog        @binlog:scsi        @binlog:lsm        @binlog:info        @binlog:msg        @binlog:startup        @binlog:shutdown        @binlog:panic        @binlog:timestamp        @binlog:unknown

In this example, the "-d" switch is used to search only the /usr/share/evm/filters directory; the "-v" switch displays the details of the filter by performing an "evmshow -f filter_exp –F".

 # evf -v -d /usr/share/evm/filters binlog /usr/share/evm/filters    binlog.evf        @binlog:binlog       -> [name sys.unix.binlog]        @binlog:scsi         -> [name sys.unix.binlog.hw.scsi]        @binlog:lsm          -> [name sys.unix.binlog.sw.lsm]        @binlog:info         -> [name sys.unix.binlog.op.info]        @binlog:msg          -> [name sys.unix.binlog.op.info.ascii_msg]        @binlog:startup      -> [name sys.unix.binlog.op.startup]        @binlog:shutdown     -> [name sys.unix.binlog.op.shutdown]        @binlog:panic        -> [name sys.unix.binlog.op.panic]        @binlog:timestamp    -> [name sys.unix.binlog.op.timestamp]        @binlog:unknown      -> [name sys.unix.binlog.unknown_class]

You can see all the filters in every file in the specified directory with the following command:

 # evf –d /usr/share/evm/filters

If you just want to see a couple of files, separate them by white space.

 # evf syslog binlog

The evf command also accepts standard shell wildcards for file specs, provided that file spec is enclosed in double quotes (""). The following example will search the EVF_DIR_PATH for all filter files that end in "log", all filter files that start with a "c" followed by either an "u" or an "a" followed by either a "u" or an "a", and any four-letter filter files.

 # export EVF_DIR_PATH="/var/evm/adm/filters /var/share/evm/filters" # evf –v "*log" "c[la][ua]" "????"

For more information on the evf command, see the evf (1) reference page included in the evf.tar.gz kit.

8.7.11 How Can I See the Format of a Filter in a Filter File?

If you are using a filter from a filter file and would like to see what it is actually filtering, you can look at the filter file (if you know where it is), or you can ask evmshow to show you.

 # evmshow -f @binlog:scsi -F  ( [name sys.unix.binlog.hw.scsi] )

8.7.12 How Can I Check the Syntax of the Template I Created?

If you are an administrator, you can use the "evmreload –n" command.

 # evmreload -n

If you are a user, you can use evmpost.

 # cat New_Events.evt | evmpost -r -M | evmshow -D

8.7.13 How Can I Sort the Events?

Since events can come from a variety of channels, the order in which they are displayed is not always the order in which you may expect to see them. Furthermore, you may wish to see the events sorted by different data items. The evmsort command was designed to accomplish this task.

Note

As of Tru64 UNIX version 5.1, the evmget command automatically invokes the "evmsort –A" command when the "-A" switch is used.

To sort events, you use the "-s" switch followed by a "Sort-Spec". The "Sort-Spec" is defined as a colon (:) delimited string of EVM data types. The sort order is determined by placing either a "+" or "−" after an EVM data type in the "Sort Spec" (the "+" indicates an ascending sort order while a "−" indicates a descending sort order). If you do not specify the sort order it will default to ascending order.

For example:

@timestamp+:@name:@priority-

This sorts each event by the timestamp (in ascending order), then by the name (in ascending order), and then by priority (in descending order).

The following examples will deliver the same results.

 # evmget | evmsort -s "@timestamp+:@name:@priority-" | \ > evmshow -t "@timestamp [@priority] @name"

Or:

 # evmget | \ > evmsort -A -s "@timestamp+:@name:@priority-" -t "@timestamp [@priority] @name"

Or:

 # evmget -A -s "@timestamp+:@name:@priority-" -t "@timestamp [@priority] @name"

Returns:

 ... 27-Mar-2001 02:00:01 [200] sys.unix.evm.chmgr.cleanup_done 27-Mar-2001 02:00:39 [500] sys.unix.syslog.mail 28-Mar-2001 07:37:41 [200] sys.unix.sysman.station.update_object.HOST.elroy 28-Mar-2001 07:37:43 [300] sys.unix.hw.dev_base_name_changed.tape._hwid.123 28-Mar-2001 07:37:43 [300] sys.unix.hw.dev_base_name_changed.tape._hwid.124 29-Mar-2001 18:02:41 [200] sys.unix.evm.mark 29-Mar-2001 18:15:41 [500] sys.unix.syslog.mail 29-Mar-2001 18:17:44 [200] sys.unix.binlog.op.timestamp

If you find that you are constantly using the same format, you may wish to define the sort order in the EVM_SORT_SPEC environment variable. If, for instance, EVM_SHOW_TEMPLATE and EVM_SORT_SPEC were both defined, then your evmget command could be shortened to:

 # export EVM_SHOW_TEMPLATE="@timestamp [@priority] @name" # export EVM_SORT_SPEC="@timestamp+:@name:@priority-" # evmget -A

8.7.14 How Can I Find Out More Information about the Event(s) I Retrieved?

There are three options to the evmshow command that will return additional information (if available) about the event:

"-d" returns the details of the event from the "fn_details" routine of the channel.
"-x" returns an explanation of the event from the "fn_explain" routine of the channel.
"-D" returns a formatted dump of the event's full contents.

Say you retrieve a series of events from the system. One of them looks interesting, and you decide you want more information. Here is an example using each switch:

 # evmget -f "[id = 4817]" | evmshow 29-Mar-2001 12:44:36 [200] sys.unix.syslog.auth

 # evmget -f "[id = 4817]" | evmshow -x     Syslog authorization events are posted by Security Integration     Architecture (SIA) components to report actions such as granting of     superuser privileges to a user through the "su" command, or refusal     to grant access.     Authorization events should be monitored by the System     Administrator.

 # evmget -f "[id = 4817]" | evmshow -D Formatted Message:      login: ROOT login on /dev/pts/1 Event Data Items:      Event Name          : sys.unix.syslog.auth      Priority            : 200      PID                 : 302      PPID                : 1      Event Id            : 4817      Timestamp           : 29-Mar-2001 12:44:36      Host IP address     : 192.168.0.69      Host Name           : sheridan      User Name           : root      Format              : login: ROOT login on /dev/pts/1      Reference           : cat:evmexp.cat:200 Variable Items:      None

 # evmget -f "[id = 4817]" | evmshow -d ============================ Syslog event ============================ EVM event name: sys.unix.syslog.auth       Syslog authorization events are posted by Security Integration       Architecture (SIA) components to report actions such as granting of       superuser privileges to a user through the "su" command, or refusal       to grant access.       Authorization events should be monitored by the System       Administrator. ====================================================================== Formatted Message:       login: ROOT login on /dev/pts/1 Event Data Items:       Event Name           : sys.unix.syslog.auth       Priority             : 200       PID                  : 302       PPID                 : 1       Event Id             : 4817       Timestamp            : 29-Mar-2001 12:44:36       Host IP address      : 192.168.0.69       Host Name            : sheridan       User Name            : root       Format               : login: ROOT login on /dev/pts/1       Reference            : cat:evmexp.cat:200 Variable Items:       None ======================================================================

As you probably noticed, the "-d" switch is a combination of both the "-x" and "-D" switches. The fn_details scripts actually call the evmshow command with the "-x" and "-D" switches.

Here is the details script from the evmlog channel (as defined in /etc/evmchannel.conf):

 # grep evmshow evmlog/evmlog_details /usr/bin/evmshow -n 1 -t "EVM event name: @name" < $TEMPFILE 2>&1 /usr/bin/evmshow -n 1 -x < $TEMPFILE 2>&1 /usr/bin/evmshow -D < $TEMPFILE 2>&1