Chapter 16. Monitoring

Accepting the fact that a network will be attacked is an important step for network administrators. Attacks will be launched against almost all networks eventually, and some will be successful. A good monitoring infrastructure can help detect attacks as they occur and often stop them before there is a problem.

Monitoring and logging are often used interchangeably. The truth is they serve very different purposes. Monitoring systems are in place to track and fix problems as they occur. Logging provides administrators with historical data about the network, while monitoring provides an instant snapshot of the network.

Network monitoring should be performed 24x7. Even if the administrative staff is not onsite 24x7, notifications should be sent to pagers ”or cell phones ” as incidents occur. To create accountability, one or more of the administrative staff should be on-call at all times to deal with potential security breaches.

This bears repeating yet again: The quicker a security incident can be detected , the quicker it can be contained and removed. An effective monitoring system is the best way to ensure that network changes are quickly caught and dealt with in a time-sensitive manner.

In order for a monitoring program to be effective it has to have two characteristics: It has to be secure and intelligent. An attacker should have a difficult time gaining access to the monitoring servers and preventing alerts from being generated. If alerts are not properly managed, it will be difficult for the monitoring staff to get to the heart of a problem. Intelligent monitoring means that alerts are generated only when there is a good chance that a problem exists. Rather than generating hundreds of useless alerts each day the monitoring system should only notify administrators when there is a real potential for a problem. Although, ultimately, the administrators will have to determine whether a problem is real or a false alarm. No software program can monitor a network device better than a human administrator.

Secure, intelligent monitoring starts with the positioning of the monitoring servers. An enterprise organization should have at least two monitoring servers, one located within the network, and another located outside of the network (Figure 16.1. If an organization has multiple locations one of the monitoring servers can be located in a remote location; otherwise , the remote server can be co-located with a hosting provider.

Figure 16.1. Multiple monitoring servers increase redundancy and provide different views of the network

There are two reasons for using multiple monitoring servers located on diverse networks. The first is for redundancy. If the primary server fails, the secondary server is still monitoring and continues to maintain information about the network. The second reason is that servers on diverse networks can provide network administrators with different information about the networks being monitored , providing a more complete picture.

The server within the network should be located within the firewall DMZ and be connected to the management network. The monitoring server will need to be able to monitor both the primary and the management IP addresses of the network devices, which means it will have a complete network topology. Hence, it is critical that the machine be protected from attackers . As with DNS and logging servers, a layer of security, over and above the steps taken as part of standard server security precautions , should be used when provisioning a monitoring server. On a Unix server this usually involves using Netfilter, or their equivalent.

The remote server, or servers, will only monitor the public IP addresses and should be allowed through the firewall to monitor any server or network device within the network. As indicated, the remote monitoring server should be on a separate network. This will help administrators determine how devices are performing both internally and externally. Remote monitoring can also give administrators advance warning of WAN connectivity problems.

There are many applications designed to monitor different aspects of the network. In general, it is best to limit the number of different applications in use for monitoring. Programs like OpenView and Netcool can meet the monitoring needs of just about any organization and are sufficiently customizable to be tailored to the particular needs of a network. Running multiple monitoring applications on a network can cause more confusion for administrators trying to sort through all the different error messages. Each additional application also poses a potential security risk and is another program that administrators have to track for security holes and bug fixes.