6.2 Concrete Monitors
6.2 Concrete MonitorsEvery JMX implementation provides three concrete subclasses of Monitor: StringMonitor , CounterMonitor , and GaugeMonitor . Each of them is designed for a specific monitoring task that requires attributes and notification types beyond those provided by the Monitor base class. However, because they are all extensions of a common base class, their usage follows a common pattern. This section describes that usage pattern and then explores the additional attributes and behavior specific to each of the concrete monitors. 6.2.1 Monitoring Apache's StatusIn order to demonstrate monitors, we need something to monitor. Rather than making up a toy example, we'll use the Apache MBean we developed in Chapter 3. Not only does it have a rich set of attributes to monitor, but given the ubiquity of Apache-based Web sites, our code could have immediate application in the real world of systems management. 6.2.1.1 Apache MBean RefresherRecall that the Apache MBean leverages Apache's mod_status module to retrieve diagnostic information about an Apache instance. When we do an HTTP GET on URLs of the form http:// <hostname> /server-status?auto an Apache instance with the mod_status module enabled will return a machine-readable page of diagnostic information. Here is the ApacheMBean interface from Chapter 3: package net.jnjmx.ch3; public interface ApacheMBean { public String DOWN = "OWN"; public String NOT_RESPONDING = "NOT_RESPONDING"; public String RUNNING = "RUNNING"; public String UNKNOWN = "UNKNOWN"; public int getBusyWorkers() throws ApacheMBeanException; public int getBytesPerSec()throws ApacheMBeanException; public float getBytesPerReq()throws ApacheMBeanException; public long getCacheLifetime(); public float getCpuLoad()throws ApacheMBeanException; public int getIdleWorkers()throws ApacheMBeanException; public float getReqPerSec()throws ApacheMBeanException; public String getScoreboard()throws ApacheMBeanException; public String getServer(); public String getState(); public int getTotalAccesses()throws ApacheMBeanException; public long getTotalKBytes()throws ApacheMBeanException; public long getUptime(throws ApacheMBeanException); public boolean isCaching(); public void setCacheLifetime(long lifetime); public void setCaching(boolean caching); public void setServer(String url) throws MalformedURLException,IllegalArgumentException; public int start()throws ApacheMBeanException; public int stop()throws ApacheMBeanException; } After we describe the common aspects of JMX monitor programming, we'll use the attributes defined in this interface to explore the use of the various JMX monitors. 6.2.2 Canonical Usage PatternUsing a monitor follows a general pattern regardless of the monitor's type. Once the MBean we're interested in has been created, we instantiate the monitor, configure it, set up the appropriate notification listeners, and start the monitor. 6.2.2.1 Creating and Configuring the MonitorBecause each of the concrete monitors provides a no-args constructor, it's simple to create instances via the MBeanServer's createMBean() method. Once the monitor has been created, we have to set its various attributes; the most efficient way to do that is with the MBeanServer's setAttributes() method, which lets us set multiple attributes in a single invocation: package net.jnjmx.ch6; import javax.management.*; import javax.management.monitor.*; import net.jnjmx.ch3.Apache; import net.jnjmx.ch3.ApacheMBean; public class ApacheMonitor { ... public ObjectName setupStateMonitor(MBeanServer mbs,ObjectName server) throws Exception The setupStateMonitor() method illustrates this process. The State attribute values are strings, so we need a StringMonitor instance. The caller passes in the object name of the ApacheStatus MBean that is to be monitored ; we use the url key property of that ObjectName instance to create a unique object name for our StringMonitor object. After we create the StringMonitor instance via createMBean() , we populate an AttributeList instance with the attributes we want to set on StringMonitor . The call to setAttributes() sets the specified attributes; after checking to make sure that they were in fact set ”that is, that the instance of AttributeList returned by setAttributes() contains an entry for each attribute successfully set ”we return the StringMonitor object name. Note that this example illustrated the pattern with StringMonitor , but the pattern is the same regardless of the monitor's concrete type. The only difference is in the attributes we use to configure the monitor. 6.2.2.2 Listening for NotificationsOnce a monitor is set up, a management application can listen for and react to the notifications it sends: package net.jnjmx.ch6; import javax.management.*; import javax.management.monitor.*; import net.jnjmx.ch3.Apache; import net.jnjmx.ch3.ApacheMBean; public class ApacheMonitor { public static class StateNotificationListener implements NotificationListener { public void handleNotification(Notification notification,Object handback) { // Take some action when a notification is received from the // State attribute monitor } public static void main(String[] args) throws Exception { // Create the MBean to be monitored MBeanServer mbs = MBeanServerFactory.createMBeanServer(); ObjectName server = new ObjectName("resources.http:type=ApacheStatus,url=www. Once the State attribute monitor is set up, we add a notification listener to that monitor. That listener is responsible for taking appropriate action when it receives a notification from the monitor. For example, if the server is not responding, there is no point in attempting to monitor its other attributes; so when the value of the State attribute is ApacheStatusMBean.NOT_RESPONDING , we could stop any other monitors configured for that server. Of course, if we stop the monitors when the server is not responding, we should start them again when it is responding. In the next section we'll see how to use StringMonitor 's attributes to arrange for that behavior. 6.2.3 StringMonitorStringMonitor observes an attribute of type String and can detect two conditions:
The StringMonitorMBean interface adds five attributes to the management interface defined by MonitorMBean . These attributes are described in Table 6.4. There are also two MonitorNotification types specific to StringMonitor (see Table 6.5). Table 6.4. StringMonitor -Specific Attributes
Table 6.5. StringMonitor -Specific Notification Types
As a convenience, the MonitorNotification class defines static final fields representing both of these types. The field identifiers are MonitorNotification.STRING_T0_COMPARE_VALUE_MATCHED // for jmx.monitor.string.matches MonitorNotification.STRING_TO_COMPARE_VALUE_DIFFERED // for jmx.monitor.string.differs At the beginning of each granularity period, a StringMonitor instance retrieves the value of the observed attribute from the MBean it is observing. That value is saved in the monitor's DerivedGauge attribute, and the time of the observation is saved in the DerivedGaugeTimeStamp attribute. The new value of the DerivedGauge attribute is then compared to the value of the StringToCompare attribute. This comparison has two possible outcomes : match, no match. If the outcome is the same as that of the comparison that occurred at the beginning of the previous granularity period ”that is, there was a match before and there's still a match, or the two values didn't match before and they still don't match ”then nothing happens. Otherwise, if the values used to be different and now they match and the value of the monitor's NotifyMatch attribute is true , a MonitorNotification instance of type jmx.monitor.string.matches is sent. Similarly, if the values used to match and now they don't and the value of the monitor's NotifyDiffer attribute is true , a MonitorNotification instance of type jmx.monitor.string.differs is sent. Note that it is perfectly legal to make the monitor's NotifyMatch and NotifyDiffer attributes true simultaneously . Doing so will result in a notification whenever the relationship between the StringToCompare attribute and the observed attribute changes. 6.2.3.1 Using StringMonitorNow we have enough information to complete ApacheMonitor 's setupStateMonitor() method we sketched in Section 6.2.2.1. Let's say we want to monitor ApacheStatusMBean 's State attribute so that we can start monitoring the server whenever it is running and stop monitoring it when it stops responding or goes down. The brute-force approach would be to use three StringMonitor instances ”one for each of the possible values of ApacheStatusMBean 's State attribute. That approach would work and have the advantage of being simple to code and easy to understand for any future developer maintaining the code. The downside is that each of those monitors has an associated overhead. Although it is generally a bad idea to indulge in gratuitous optimizations, in this case we can use the technique mentioned in the previous section to meet the management application's requirements with a single instance of StringMonitor . All we want to know is when the value of ApacheStatusMBean 's State attribute goes from ApacheStatusMBean.RUNNING to another value and back again. Setting the monitor's StringToCompare attribute to ApacheStatusMBean.RUNNING and both its NotifyMatch and NotifyDiffer attributes to true provides the desired behavior. The first time the State attribute and the value of the StringToCompare attribute ( ApacheStatusMBean.RUNNING ) match, a MonitorNotification instance of type jmx.monitor.string.matches is sent. No further notifications are sent until the State attribute no longer matches ApacheStatusMBean.RUNNING ; then a MonitorNotification instance of type jmx.monitor.string.differs is sent. Monitor notifications of the appropriate type continue to be sent as the value of the State attribute changes. It's now up to the management application listening for those notifications to react so as to provide the desired behavior. The expanded version of ApacheMonitor that follows demonstrates how to configure the State monitor and handle the subsequent notifications: package net.jnjmx.ch6; import javax.management.*; import javax.management.monitor.*; import net.jnjmx.ch3.Apache; import net.jnjmx.ch3.ApacheMBean; public ApacheMonitor { public static class StateNotificationListener implements NotificationListener { public void handleNotification(Notification notification, Object handback) { String type = notification.getType(); if (type.compareTo(MonitorNotification.STRING_TO_COMPARE_VALUE_MATCHED) == 0) { Iterator i = monitors.iterator(); while (i.hasNext()) { ObjectName on = (ObjectName) i.next(); ((MBeanServer) handback).invoke(on, "start", new Object[] {}, new String[] {}); } }else if (type.compareTo(MonitorNotification.STRING_TO_COMPARE_VALUE_DIFFERED) == We add three StringMonitor -specific attributes to the attribute list in setupStateMonitor . When the main() method creates the remaining monitors, it adds them to a list that NotificationListener 's handleNotification() method uses. Finally, note that the application need not start the other monitors; NotificationListener will take care of that as soon as it notices that the server is running. 6.2.4 CounterMonitorCounters are attributes whose numeric values increase monotonically over time. The number of requests a Web server has handled, the number of packets a router has forwarded, the number of orders a system has processed , and so on are all example of counters. Management applications often need to initiate an action when counters reach a threshold value ”taking the system offline for scheduled maintenance, paging the administrator, and so on. JMX-based management applications use instances of CounterMonitor to observe counter attributes and send notifications when threshold values are reached. Table 6.6 describes the attributes that the CounterMonitorMBean interface adds to the common MonitorMBean interface. Table 6.6. CounterMonitor -Specific Attributes
CounterMonitor adds a single new MonitorNotification type. A MonitorNotification instance of type jmx.monitor.counter.threshold is sent whenever the monitor's threshold is exceeded and the value of the monitor's Notify attribute is true . The MonitorNotification field that corresponds to this type is MonitorNotification.THRESHOLD_VALUE_EXCEEDED // for jmx.monitor.counter.threshold CounterMonitor enforces the following constraints on the values of its attributes:
Violating either of these constraints will cause a MonitorNotification instance of type jmx.monitor.error.threshold to be sent when CounterMonitor is started. The logical model for CounterMonitor is simple: The monitor makes periodic observations of the counter attribute's value, and the first time that value exceeds the value of the monitor's Threshold attribute, a MonitorNotification instance of type jmx.monitor.counter.threshold is sent. In practice, several issues arise that make the actual implementation more complex:
Whereas the management application could take responsibility for increasing CounterMonitor 's Threshold attribute when it receives a jmx.monitor.counter.threshold notification, CounterMonitor provides its own mechanism for addressing this issue. Whenever the value of the observed attribute exceeds the value of CounterMonitor 's Threshold attribute, the monitor sends the appropriate notification and then computes a new value for its Threshold attribute by adding the value of its Offset attribute to the current Threshold attribute value: Threshold new = Threshold current + Offset Now we will be notified the first time the value of the observed counter attribute exceeds this new threshold. This behavior allows management applications to receive periodic notifications based on the value of a counter attribute ”for example, every hundred requests. CounterMonitor 's Modulus attribute addresses the second issue. When the monitor is configured, its Modulus attribute should be set to the observed attribute's maximum value. The Modulus value is then used in the computation of the new Threshold value. The pseudocode that follows illustrates this computation: if (Threshold current + Offset > Modulus) then Threshold new = Threshold original else Threshold new = Threshold current + Offset For example, assume that the maximum value of the observed attribute is 64 . Suppose we configure a CounterMonitor instance for this attribute with an initial Threshold value of 16 and an Offset value of 16 . This configuration will work fine until the observed attribute rolls over at 64 while the Threshold value is increased to 80. Setting the Modulus value to 64 allows us to avoid this unfortunate situation. Finally, CounterMonitor has two modes of operation. The first, raw mode, is the default; in raw mode the monitor's derived gauge is the raw value of the observed attribute. In difference mode, which we enable by setting the value of the monitor's DifferenceMode attribute to true , the derived gauge is the difference between the current value of the observed attribute and its previous value. If the difference is negative, because of a rollover between observations, Modulus is added to the difference. In DifferenceMode , CounterMonitor measures a rate; so, for example, if the observed attribute is TotalRequests and the monitor's GranularityPeriod is one minute, by using DifferenceMode we can monitor the number of requests per minute without adding any additional instrumentation to the observed MBean. 6.2.4.1 Using CounterMonitorLet's use the DifferenceMode technique to monitor an Apache server for traffic spikes. ApacheStatusMBean has two counter attributes: TotalAccesses and TotalKBytes ; we are interested in the latter. The setupTrafficRateMonitor() method creates and configures an instance of CounterMonitor that will do the job. package net.jnjmx.ch6; import javax.management.*; import javax.management.monitor.*; import net.jnjmx.ch3.Apache; import net.jnjmx.ch3.ApacheMBean; public class ApacheMonitor { ... public ObjectName setupTraffidRateMonitor(MBeanServer mbs, ObjectName server) throws Exception { /* Assume ApacheStatus object names are of the form: * <domain>:type=ApacheStatus,url=<URL of the server we're * interested in> */ String url = server.getKeyProperty("url"); ObjectName mon = new ObjectName("monitors:type=String,attr=State,url=" + url); mbs.createMBean("javax.management.monitor.CounterMonitor", mon); AttributeList al = new AttributeList(); al.add(new Attribute("ObservedObject", server)); al.add(new Attribute("ObservedAttribute", "TotalKBytes"); al.add(new Attribute("GranularityPeriod", 60000); al.add(new Attribute("Modulus", new Long(4294967295)); al.add(new Attribute("DifferenceMode", new Boolean(true)); al.add(new Attribute("Threshold", new Long(7500)); AttributeList sl = mbs.setAttributes(mon, al); // Check to make sure all of the attributes were set return mon; } ... } The only mysterious part of the configuration is the Modulus setting. Where did we get the number 4294967295? A quick look at the source to Apache's mod_status module reveals that an unsigned long value is used to hold the TotalKBytes value. In C, the largest number an unsigned long value can represent is 4294967295 ”hence the Modulus setting. Once the traffic rate monitor is configured, the management application can add a notification listener that takes the appropriate action when a traffic spike is detected by the monitor. 6.2.5 GaugeMonitorA gauge, like a counter, is a numeric attribute. Unlike a counter, the value of a gauge may increase and decrease over time. Transfer rate, hit rate, average wait time, and so forth are all examples of gauges. In the management interface presented by ApacheStatus , the ReqPerSec attribute is a gauge. Management applications that monitor gauge attributes usually do so to see that their values fall into a range of "acceptable" values ” neither too high nor too low. When a gauge's value falls outside the range of acceptable values, the management application needs to be notified so that it can take the appropriate action. Management applications may also want to be notified when the gauge's value falls back into the acceptable range so that corrective actions ”for example, blocking new connection requests ”can be terminated . The GaugeMonitorMBean interface adds seven attributes to the management interface defined by MonitorMBean . These attributes are listed in Table 6.7. Unlike StringMonitor and CounterMonitor , GaugeMonitor also defines an operation: public void setThresholds(Number high, Number low) Table 6.7. GaugeMonitor -Specific Attributes
The somewhat unfortunately named setThresholds() method (the set name makes it easy to mistake it for an attribute) establishes values for GaugeMonitor 's HighThreshold and LowThreshold attributes in a single call. Why not make these two attributes read/write ”that is, provide independent setters for each of them ”and avoid the confusion? At first blush that seems like a reasonable approach; it turns out, however, that GaugeMonitor 's behavior depends on having meaningful values for both of these attributes. If their values were established independently, there would be no way to guarantee that both values got set, and it's not possible to come up with an appropriate default value for one threshold based on the value of the other. Two additional MonitorNotification types, listed in Table 6.8, are associated with GaugeMonitor . As a convenience, the MonitorNotification class defines static final fields representing both of these types. The field identifiers are MonitorNotification.THRESHOLD_HIGH_VALUE_EXCEEDED // for jmx.monitor.gauge.high MonitorNotification.THRESHOLD_LOW_VALUE_EXCEEDED // for jmx.monitor.gauge.low GaugeMonitor enforces the following constraints on its attributes:
Violating either of these constraints will cause a MonitorNotification instance of type jmx.monitor.error.threshold to be sent when GaugeMonitor is started. When the value of the observed attribute is increasing and exceeds the value of GaugeMonitor 's HighThreshold attribute and GaugeMonitor 's NotifyHigh attribute is true , a MonitorNotification instance of type jmx.monitor.gauge.high will be sent. Likewise, when the value of the observed attribute is decreasing and it becomes less than the value of GaugeMonitor 's LowThreshold attribute and GaugeMonitor 's NotifyLow attribute is true , a MonitorNotification instance of type jmx.monitor.gauge.low will be sent. In both of these cases, after a notification has been sent, no further notifications will be sent until the observed attribute's value crosses the opposite threshold ”for example, after the observed attribute's value exceeds the monitor's HighThreshold value, which generates a jmx.monitor.gauge.high notification ”no further jmx.monitor.gauge.high notifications will be sent until the observed attribute's value goes below the monitor's LowThreshold . Table 6.8. GaugeMonitor -Specific Notification Types
There are two things to note about the behavior just described. The first is that if the observed attribute's value oscillates across either of the threshold values, only the first threshold crossing will generate a notification. The second is that when an observed attribute's value exceeds HighThreshold , a notification is sent only if the value is increasing; similarly, when an observed attribute's value is less than LowThreshold , a notification is sent only if the value is decreasing. This behavior has two implications. First, at least two observations are required before a notification can be sent from GaugeMonitor . Second, when GaugeMonitor is started, the monitor may see a sequence of values that are all above HighThreshold , or below LowThreshold , and not send a notification because the values are always decreasing, or increasing, respectively. Finally, like CounterMonitor , GaugeMonitor operates in one of two modes: raw mode, the default, which uses the actual values of the observed attribute in its comparisons; or difference mode, which uses the difference between the current observation and the previous one as the basis for comparison. We enable difference mode by setting the monitor's DifferenceMode attribute to true . 6.2.5.1 Using GaugeMonitorManagement applications usually create an instance of GaugeMonitor for each "boundary" in an attribute's range of acceptable values. So in the general case where there was an upper and a lower bound on the range of acceptable values, a management application would create a GaugeMonitor instance for the upper bound and another for the lower bound. Why two? Why not a single GaugeMonitor instance with the HighThreshold value set to the upper bound and the LowThreshold value set to the lower bound? An administrator or a management application needs to know when normal operations have been restored, not just when something has gone wrong. If we use a single monitor, we'll get a notification when the attribute's value goes below the lower bound; that's fine. However, we won't get any more notifications until the attribute's value exceeds the upper bound; that's not fine. If we take some action to bring the attribute's value up, we want to know as soon as the value is acceptable again, not when it becomes unacceptable on the other side of the range. How does all this apply to our ApacheMonitor management application? The Apache MBean's management interface includes several gauge attributes. Let's pick one and monitor it. The ReqPerSec attribute is one of the Apache MBean gauges; how we monitor it depends on our site. For example, if this is a product support site, we are probably concerned only about the site being inundated with requests; lack of traffic isn't an issue (if the site is not getting any requests, our product must be so wonderful that customers don't need support). On the other hand, if we're running a commercial site, we're still concerned about too many requests but we also want to know if we're not at least receiving some minimum level of traffic. For the sake of simplicity, let's assume that this is a product support site and all we're concerned about is receiving more requests than we can handle ”say, 100 requests per second. In that case we'll create a single instance of GaugeMonitor with a HighThreshold value of 100 and a LowThreshold value of 95. The code in the setupRPSMonitor() method presented here does the job: package net.jnjmx.ch6; import javax.management.*; import javax.management.monitor.*; import net.jnjmx.ch3.Apache; import net.jnjmx.ch3.ApacheMBean; public class ApacheMonitor { ... public ObjectName setupRPSMonitor(MBeanServer mbs, ObjectName server) throws Exception { /* Assume ApacheStatus object names are of the form: * <domain>:type=ApacheStatus,url=<URL of the server we're * interested in> */ String url = server.getKeyProperty("url"); ObjectName mon = new ObjectName("monitors:type=String,attr=State,url=" + url); mbs.createMBean("javax.management.monitor.GaugeMonitor", mon); AttributeList al = new AttributeList(); al.add(new Attribute("ObservedObject", server)); al.add(new Attribute("ObservedAttribute", "ReqPerSec"); al.add(new Attribute("GranularityPeriod", new Long(60000)); al.add(new Attribute("NotifyHigh", new Boolean(true)); al.add(new Attribute("NotifyLow", new Boolean(true)); AttributeList sl = mbs.setAttributes(mon, al); // Check to make sure all of the attributes were set mbs.invoke(mon, "setThresholds", new Object[] {new Float(100.0), new Float(95.0) }, new String[] {"float", "float" }); return mon; } ... } Now we can add a notification listener to this instance of GaugeMonitor and perhaps react to MonitorNotification instances of type jmx.monitor.gauge.high by instructing the server to return a "busy" page when new requests arrive and then restore service when we get a subsequent MonitorNotification instance of type jmx.monitor.gauge.low . |
{ /* Assume ApacheStatus object names are of the form: * <domain>:type=ApacheStatus,url=<URL of the server we're * interested in> */ String url = server.getKeyProperty("url"); ObjectName mon = new ObjectName("monitors:type=String,attr=State,url=" + url); mbs.createMBean("javax.management.monitor.StringMonitor", mon); AttributeList al = new AttributeList(); al.add(new Attribute("ObservedObject", server)); al.add(new Attribute("ObservedAttribute", "State"); al.add(new Attribute("GranularityPeriod", 60000); // Add additional StringMonitor-specific attributes AttributeList sl = mbs.setAttributes(mon, al); // Check to make sure all of the attributes were set return mon; } ... } 
EAN: 2147483647
Pages: 115