Section 5.1. Alerts

5.1. Alerts

The first four settings relate to alerts and responses to alerts. The Notification Command Format , the Alert Resolution State, and the Custom Alert Field global settings work in concert with the event, performance, and alert rules in the rule groups. These settings set the default behavior of an alert, or modify the default information that is included in an alert at the time of its creation. All data that is included in an alert can be used in the Operator console to build filters for custom views.

5.1.1. Notification Command Format

When you configure responses in alert rules, you have the option to send a message to a notification group (see Figure 5-2). Here, the Network Administrators notification group has been selected.

Figure 5-2. Alert rule notification response options

One way to communicate with a notification group is to run an executable and pass application- and alert-specific fields to it. This is done on the Command Format tab (see Figure 5-3).

You can either define a custom command format for any particular alert rule or use the default command format. The default command format is set at the global level. When you select the Notification Command Format tab in the Global Settings container, you see the configuration page shown in Figure 5-4.

The intended use for this feature is to call third-party paging applications. This feature would be used, for instance, if your MOM 2005 installation does not have access to an SMTP-based paging service or SMTP server for email, but you still need to send alerts to operators in addition to displaying them in the Operator console.

Figure 5-3. Alert rule notification response options Command Format tab

Figure 5-4. Global Settings Notification Command Format

One way to do this is to install a third-party paging program on the management server. When an alert is raised and the alert rule is configured to send out a message to a notification group, this option will launch the paging application to create the message and send it out via a direct attached modem or shared modem pool. When sending notifications out via modem, be aware that in the event of a notification flood, the modem may not be able to keep up with the number of notifications from MOM. In this case, make sure that only the most critical notifications are going out to reduce the load.

The best way to use this feature is to explicitly call an executable that is designated in the Full Path to Application text box. That means that you can only pass options to the executable that it understands. In addition to the executable parameters, event and alert fields can also be passed. Note that you will only receive the number of characters that the paging program supports.

Sending a page is not the only use, however, for the Notification Command Format. You can call any executable from here through the Windows command interpreter, as shown in Figure 5-4.

For example, you might want to send a network notification to an operator who is logged onto the network, via the net send command. To do this, you need to check the "Use Windows command interpreter" box. This will pass the command line that you enter to cmd.exe. This is the exact same thing as typing the command-line syntax at a command prompt with the addition of MOM-specific variables.

Constructing the parameters string in the Command Line text box is simple. It consists of a combination of plain text for every notification and variables that are replaced with alert- or event-specific data at the time the notification is generated. The variable fields are delineated with a dollar sign ($) bracketing them. The plain text is simply written out.

You don't have to know all of the possible event and alert fields. The right-facing arrow to the right of the Command Line box (see Figure 5-5) branches into a list of alert fields and event fields. Select the variable you want to insert into the Command Line string and it is placed at the insertion point.

For example, if you want to use net send, which depends on the Windows messenger service and Windows Internet Naming Service (WINS) running in your environment, you would select the Windows command interpreter and construct a command line that looks like this:

 Send $Operator ID$ "Operations Manager Alert on $Source Domain$\$Source Computer$: $Description$ (view with $Alert URL$)" 

net send is used as an illustration here. There are known exploits that make use of the messenger services so be careful about using this in production. See Microsoft article 330904 for more information (http://support.microsoft.com/default.aspx?scid=kb;en-us;330904).

Figure 5-5. Alert fields that are available for inclusion in the command-line parameters

When the alert fires off, it will generate a pop-up message like the one shown in Figure 5-6.

Figure 5-6. Messenger pop-up notification of an Agent heartbeat failure alert

It is a good idea to confirm what will be included in a notification message, no matter what transport you have chosen. The following example uses the MOM Agent heartbeat failure alert by following these steps:

1. Configure the Global Settings Notification Command Format (see Figure 5-5).

2. Create a notification operator and add it to the Network Administrators notification group. This operator must have the Command option enabled (Figure 5-7).

   Figure 5-7. Enable the operator to receive notifications via command

   

3. Navigate to the Microsoft Operations Manager Operations Manager 2005 Server Alert Rules group. Create an Alert rule (right-click, create rule) with Alert Criteria that only matches alerts generated by the Server rule group (see Figure 5-8).

   Figure 5-8. Configuring the Alert Criteria to match all alerts generated in the Server rule group

   

4. Click Next and accept the default schedule, which is to always process data. Click Next again to proceed to the Responses configuration page.

5. On the Alert Rule Properties Responses page (see Figure 5-9), click Add and select the "Send a notification to a Notification Group" option. Then select the Network Administrators notification group from the drop-down list shown in Figure 5-2.

Figure 5-9. Choosing to send an alert to a notification group

Click Next to bring up the Alert Rule Properties - General page, where you give the new alert rule a name (Figure 5-10). In this example, it is named NotificationTestRule. Click Finish.

To complete testing, log on as the operator account, which was added to the Network Administrators notification group, on any machine on the network (the messenger service must be enabled on both the management server and the computer you are logged onto). Then, on any of the agent-managed servers on the network, stop the MOM service and wait for the pop-up to appear.

5.1.2. Email Server

The Email Server tab in the Global Settings is where you indicate the SMTP server that all of the management servers in the management group communicate with when sending notifications. This tab consists of five fields:

Transport

This is read-only and is set to SMTP.

Server Name

This can be either the FQDN or the NetBIOS name of the SMTP server.

Figure 5-10. Naming a newly created alert rule

Return Address

All SMTP communication sessions require that a properly formatted sender address be passed from the sender to the SMTP server. This is used as the From address on the email.

Character Set

This defines which character set to use for the email notifications. Keep the default of Unicode Transformation Format 8-bit (UTF-8).

SMTP port

The SMTP protocol defaults to TCP port 25. Unless you have a very specific reason to change this, I recommend accepting the default.

5.1.3. Alert Resolution States

Alert resolution states are covered in the "The Life of a MOM 2005 Alert" section in Chapter 1. But to refresh, all alerts exist in one of several resolution states:

• New

• Acknowledged

• Level 1: Assigned to help desk

• Level 2: Assigned to subject matter expert

• Level 3: Requires scheduled maintenance

• Level 4: Assigned to external group or vendor

• Resolved

By default, when an alert first appears in the Operator console, its resolution state is always set to New.

MOM times how long an alert is in any resolution state. When that time exceeds the value configured in the service-level agreement for that resolution state, the alert will then appear in the service-level Exceptions Alert view in the Operator console. This allows a company to track their ability to meet service-level agreements.

You can modify the default settings, shown in Figure 5-11, and even delete them (except for the New and Resolved states). The most common modification made is to change the service-level agreement time period to suit your environment. The resolution states themselves are useful only if your support structure breaks down into these four levels. If your environment alerts are only sent to the Windows administrators, who then open a support case with an external vendor, then there is no need for the alert resolution states. You can delete the unnecessary alerts, or leave them and not use them, at your discretion. In addition, you can create new alert resolution states that suit your environment.

Figure 5-11. Default values for alert resolution states

When you select an alert state from the list and click Modify, you'll see the Edit Alert State dialog box, shown in Figure 5-12. An alert resolution state contains five fields :

Name

The purpose of the Name field is obvious, but keep in mind that the more descriptive it is, the more useful it is.

Figure 5-12. Attributes of an alert resolution state

ID

Internally to MOM, all alert resolution state objects are actually identified by their ID number. This number must be between 0 and 255. The 0 ID is reserved for the New resolution state, and ID 255 is reserved for the Resolved resolution state. All other values (1 to 254) are available for use.

Service level agreement

This field can be set to any minute, hour, or day.

Shortcut keystroke

To help ease administration of an alert in the Operator console, you can assign a keystroke combination to an alert resolution state. For example, you can assign Ctrl-Alt-I to the Acknowledged state. After restarting the Operator console you can select an alert, press Ctrl-Alt-I, and its resolution state will be changed to Acknowledged.

Display in console menus

You can control whether operators use an alert resolution state in the Web or Operator consoles. This is useful for restricting access to custom created states. If this checkbox is cleared, this particular resolution state will not be available in the Operator or Web consoles. You can still configure event and performance rules to create alerts with the unavailable state as the default state. The alerts will then have this state when they appear in the Operator console, but it would not appear as a resolution state option for that alert. However, you can still access and make use of this alert resolution state programmatically.

For example, to build a custom alert view in the My Views node in the Operator console, you could build a filter using this non-visible alert resolution state value to populate the view. This is particularly useful for alerts that you want to see in the Operator console, but do not require action. For example, the alert generated by the Test End to End Monitoring task in Chapter 3 confirmed that the round-trip communication from the management server to the agent was working. The following is how a Custom Alert view is built:

1. Create a new alert resolution state by clicking the Add button, as shown in Figure 5-11.

   The Edit Resolution State page will appear, as shown in Figure 5-13.

   Figure 5-13. Creating a new alert resolution state

   

2. Create a new resolution state, for example "Test alert - Ignore" with an ID value of 86.

3. Clear the "Users may set this alert resolution state in the Operator Console and Web Console" checkbox.

4. Set this custom resolution state as the default for the MOM End to End Monitoring event rule.

5. Navigate to the Rule Groups folder in the Administrator console and from there to Microsoft Operations Manager Operations Manager 2005 Agents on All MOM Roles rule group.

6. Assign this new alert resolution state as the default state on the Alert tab, as shown in Figure 5-14.

   Whenever the Test End to End Monitoring task is fired off, if a success alert is generated it will have "Test alert - Ignore" as its starting resolution state.

   Figure 5-14. Set the event rule to generate an alert with the newly created resolution state

   

7. Launch the Test End to End Monitoring task against an agent-managed computer.

   The alert is generated and surfaces in the default Alerts view, as shown in Figure 5-15.

8. Create a custom view in the My Views node that will filter all alerts that have the "Test alert - Ignore" state.

9. Navigate to the All My Views node in the Operator console, open the context menu, and select Create a New Alerts View (see Figure 5-16).

   Figure 5-15. An alert with the custom "Test alert - ignore" state

   

   Figure 5-16. Creating a new Alerts view in the Operator console

   

   This opens the "Which type of Alert view do you want to create?" page.

10. Select the "Alerts that satisfy specified criteria" option (see Figure 5-17).

    Figure 5-17. Step 1 of creating a custom alert view

    

11. Proceed through the filter creation process. Click Next and select the "with specified resolution state" checkbox (see Figure 5-18).

    Figure 5-18. Selecting to filter on a specified resolution state

    

12. Click on the underlined value to select the "Test alert - Ignore" value for the resolution state in the "View description" pane.

13. Select the "Test alert - Ignore" resolution state, as shown in Figure 5-19.

    Figure 5-19. Choosing the custom resolution state for the filter

    

14. Click OK, name the view, and finish the process.

    The view is then built in the Operator console and is populated with the alert that that meets the criteria, as you can see in Figure 5-20.

Figure 5-20. Custom view filtering for the "Test alert - ignore" resolution state

5.1.4. Custom Alert Fields

Each alert has five custom fields that enable you to modify the information embedded in an alert when it is created. At the global level, you can provide names for these fields. At the event and performance rule levels, you can populate the content of the named field much like the Notification Command Format command-line string is populated with text and variables.

Out-of-the-box, a generated alert only shows about half of the available fields in an alert object. One of the fields that is not present by default is the Event Number. Ordinarily, to find out which event is associated with an alert, you need to navigate to the Events tab in the Details pane. To find the details of that event, such as the event number, open the associated event. An easier way to include this data in a generated alert is to name one of the custom alert fields with an appropriate name, like Event Number (see Figure 5-21), at the global level. Apply the change and then commit the configuration change on the management packs object in the Administrator console.

Figure 5-21. Renaming Custom field 1

To have this value included as a field in an alert, open the event or performance rule and click the Custom Fields button as shown in point 1 in Figure 5-22.

Figure 5-22. Adding values to the custom fields of an event

This brings up the Custom Fields page, where the value of that custom field is set to the $Event Number$, as shown in Figure 5-23. The $Event Number$ value is one of many values available, including Full Event Number, Event type, Message DLL, Source Name, Provider Name, Provider type, Description, and so on. For a complete listing of fields that are available for alerts, refer to the MOM SDK. The full list is available when you click on the right-facing arrow button pictured in Figure 5-23 to the right of the $Event Number$ text box.

Once the configuration changes are set, all future alerts generated by that rule will include the $Event Number$ value. You can see this in the Operator console in the Alert Details pane for the alert.

One last note about custom fields: you are not restricted to using the MOM 2005 alert variables, you can add plain text as well. In Figure 5-23, the value could have been constructed to read "The event number is $Event Number$." When the MOM Agent heartbeat alert is generated, the text reads "The event number is 21284," as shown in point 1 in Figure 5-24.

Figure 5-23. Adding the event number parameter to CustomField1, renamed Event Number

Figure 5-24. Concatenating plain text and alert variables into a custom field