4.6. Monitoring System Health

 < Day Day Up > 

Performance monitoring concerns might seem out of place in a book about system security, but system availability is a vital part of system security. After all, denial of service attacks are considered security concerns even though they merely make systems unavailable. Keeping a keen eye on things like disk usage, load averages, or the existence or absence of specific daemons will ensure that you are immediately aware of your systems behaving unusually.

Network monitoring can be a bit of a double-edged sword. Keeping track of what your systems are doing will definitely help you know when they misbehave. Yet to do this, you must invariably allow connections to your system from your management station; and most monitoring suites offer little in the way of authentication. Moreover, these suites often have a long history of vulnerabilities. Ironically you could increase your exposure to risks by installing software that helps monitor for risks. As with any other software you install, you should remain vigilant.

Moreover, a carefully deployed and administered monitoring suite will be a 24/7 guardian over your network. It will have a comprehensive view of every server, service, and vital application. It is imperative that any monitoring solution you deploy is very well protected against prying eyes. Employ the principal of least privilege here and allow very few shell accounts to the monitoring station. Access to any web interface should be tightly restricted and require authentication over an encrypted channel from known hosts. There are few better reconnaissance tools than a monitoring suite carefully configured by a conscientious system administrator.

There are several open source monitoring tools that can be deployed on FreeBSD and OpenBSD systems like Big Brother, which is free under certain conditions, Big Sister, and OpenNMS. There are also a variety of tools to monitor both hosts and network devices using SNMP. All of these are contenders in the network monitoring space, but we will be looking closely at one of the most flexible and widely deployed network monitoring packages, Nagios (formerly NetSaint).

4.6.1. Nagios

Nagios is available in FreeBSD's ports tree or from the Nagios web site, available at: http://www.nagios.org/. Nagios implements host and service monitoring for systems across a network. The nagios daemon runs on a single server and uses various plugins to perform periodic checks.

The plug-ins distributed with Nagios (in the nagios-plugins package) are capable of performing local checks, which monitor load average, disk space usage, memory and swap utilization, number of logged in users, and so on. Of course there is more to system monitoring than only examining the monitoring host. Therefore, plug-ins are also included to monitor the availability of network services on remote systems, as long as a TCP or UDP port is open for probing. When problems are noticed, Nagios can be figured to send out notifications in a variety of ways, commonly via email.

Plug-ins for Nagios are constantly updated, and new plug-ins appear on a regular basis, as needed. Given a particular need in your environment, writing your own plug-in is very simple using just about any programming or scripting language you choose. In addition to plug-ins are add-ons. Add-ons extend Nagios' functionality by providing a breadth of functionality too extensive to mention. One of these add-ons is the Nagios Remote Plugin Executor (NRPE). This daemon is configured on client systems with a variety of local checks allowing the monitoring host to check local statistics on remote systems.

4.6.1.1 Installation

Installation on FreeBSD starts and ends with a make install from the net-mgmt/nagios subdirectory of the ports hierarchy. This installs both Nagios and the nagios-plugins collection automatically. It will also create a Nagios user and group for the execution of the daemon. OpenBSD administrators will need to fetch the compressed tarball and install the software in the traditional way per the documentation on the Nagios web site. The rest of this overview will assume that you have either installed Nagios from ports or have installed it manually in compliance with hier(7).

4.6.1.2 Configuration

Default configuration files for Nagios are installed to /usr/local/etc/nagios. Many of these configuration files can be left as they are, after the sample suffix has been removed. There are three main configuration files to look at when first configuring Nagios. These are the main Nagios configuration file, the CGI configuration file, and the object configuration files.


nagios.cfg

The main Nagios configuration file, nagios.cfg by default, controls the way Nagios finds its configuration and subsequently operates. The sample file is well documented, but you should also consult the documentation on the nagios.org web site as you work through this file. There are sets of options in this file worth discussing.

The first is the check_external_commands option, which enables the submission of commands through the web interface. If you feel your Nagios web interface is sufficiently protected (for instance, by using digest, Kerberos, or certificate-based authentication over SSL), you may wish to change this value to 1 to enable external commands. This will allow you to schedule service/host downtime, enable and disable various checks and notifications, delay checks, force checks, and issue other commands through the web interface. These web-submitted commands go into the file specified by command_file. Only the web server, users should have access to the directory in which this file is stored.

The second set of options are those that point to the object configuration files cfg_file and cfg_dir. Each can be specified multiple times and may point to specific files or specific directories. When directories are specified, all files ending in .cfg within the directory will be processed. Directory specification allows for a little more flexibility and easier delegation of responsibility.

You will want to peruse the rest of the settings in this file, but most will have reasonable defaults.


cgi.cfg

The CGI configuration file, cgi.cfg by default, controls the behavior of the Nagios web interface. This includes how Nagios should build URLs within the interface, and which users have access to various aspects of the Nagios system. These users must be authenticated by the web server and gain access by username.

It is possible to both disable authentication and allow commands to be submitted through the web interface by specifying a default_user_name. If you do this, make sure your Nagios web interface is protected in some other way so that only trusted administrators can access it.


Object configuration files

The object configuration files are the heart of Nagios's configuration. These files describe the hosts Nagios will monitor, the services to monitor on these hosts, who will be contacted when problems are detected, when checks are performed, and so on. In order to get Nagios operational, your best bet is to look over all the sample configuration files and move them into a configuration subdirectory after you have modified them to suit your environment. Start with the most basic configuration files like hosts.cfg-sample, services.cfg-sample, timeperiods.cfg-sample, and contacts.cfg-sample. You will probably be able to simply rename the checkcommands.cfg-sample and miscocmmands.cfg-sample configuration files until you have a better idea of additional commands you need to run. Start by configuring Nagios to monitor local statistics. When you feel comfortable with the way that works, start monitoring remotely accessible services on other systems. Once this is done, you will be ready to tackle NRPE.

The information here is a cursory overview of Nagios configuration. For detailed explanations of all the available options, make sure to read the sample configuration files thoroughly and peruse the documentation available at http://www.nagios.org/.


4.6.1.3 Installing NRPE

The Nagios Remote Plugin Executor makes local checks on remote systems possible. NRPE is available in the FreeBSD ports tree in ports/net-mgmt/nrpe2. OpenBSD administrators must fetch the port from the addons page at nagios.org. Once retrieved make sure you include support for OpenSSL. This may be done on FreeBSD systems by running make WITH_SSL=yes from the port directory. OpenBSD administrators will need to pass the --enable-ssl argument to the configure script.

Ensuring that NRPE is built with OpenSSL support means that all communications between the check_nrpe program on the monitoring host and nrpe daemon on client systems will be encrypted. Remember that this is just encryption, not authentication.

Beware of enabling command-line arguments for nrpe. TRaditionally, nrpe is configured on client systems with a known set of named commands. The paths to these commands and associated arguments are hard coded on the client systems. Enabling command-line arguments allows the check_nrpe plug-in to not only tell client systems to run a particular check, but also provides the specific command-line arguments. While this allows you to manage your configuration of client checks from the Nagios monitoring host, it has a variety of unpleasant security ramifications. If you have developed a means to securely distribute configuration files as described earlier in this chapter, managing nrpe configuration centrally should be trivial.

On the Nagios monitoring host, once the NRPE package has been compiled, the check_nrpe binary must be copied into /usr/local/libexec/nagios with the rest of the Nagios plug-ins. On all client systems, copy the nrpe binary to /usr/local/sbin instead.

4.6.1.4 Configuring Nagios with NRPE

On the monitoring host, you will need to tell Nagios to use NRPE to run local checks on remote systems. To do this, add the following check_nrpe command to your checkcommands.cfg file as follows.

define command {   command_name    check_nrpe   command_line    /usr/local/libexec/nagios/check_nrpe -H $HOSTADDRESS$ -c $ARG1$ }

You will then need to add commands to one of your configuration files or create a new configuration file on the monitoring host that specifies which NRPE checks should be run on which remote systems. This procedure is fully documented in the README distributed with NRPE. No other configuration is required on the monitoring host.

NRPE on client systems may then be configured to run out of inetd(8) (or xinetd) to make use of tcpwrappers support and rate limiting. Alternately it may be run directly as a service using the startup script provided in the port. nrpe can be configured with a list of IP addresses from which to accept commands directly.

On all client systems, you will need to install the nagios-plugins port and configure NRPE by creating an nrpe.cfg configuration file usually located in /usr/local/etc. This file should contain a list of local commands whose output the nrpe daemon will send back to the Nagios process on the monitoring host.

4.6.1.5 Fine-tuning

A complete description of configuring Nagios for the variety of environments out there would consume far more pages than we are able to spare, but rest assured, documentation exists. As a newcomer to Nagios, do not expect to get the system operational in a day, or even a few days. With the extensive documentation on the Nagios web site, the FAQ, the forums, and the mailing lists, however, you will not be short on help.

4.6.1.6 Wrapping up

With Nagios and nrpe operational, you have a 24/7 observer of all the systems under your jurisdiction. Configure your thresholds appropriately and you will become immediately aware when unusual activity is detected. For instance, on an ftp download-only server, it may be especially important to detect even a small increase in disk usage. This might indicate that the system is misconfigured and allowing people to add content. Watching the number of smtpd processes on a mail relay may provide an early warning allowing you to investigate before the system goes down.

What to watch, and what thresholds to set, are questions you will have to answer for yourself. After you have Nagios set up, configure it to warn you earlier rather than later when it detects a problem. If you find that your thresholds are set too low, you can always raise them. Your goal is to know as soon as something unusual is happening on your system, but you don't want to be badgered by useless alerts.

     < Day Day Up > 


    Mastering FreeBSD and OpenBSD Security
    Practical Guide to Software Quality Management (Artech House Computing Library)
    ISBN: 596006268
    EAN: 2147483647
    Year: 2003
    Pages: 142
    Authors: John W. Horch

    flylib.com © 2008-2017.
    If you may any questions please contact us: flylib@qtcs.net