Working in Stillness

In information security, we must always be "listening" for our enemies. When someone steps across a security boundary, a log should be generated and an alarm should go off, spurring us into action. Such logs and alerts are vital elements to our overall security practices. Just like in the physical world, however, it is impossible to "hear" the enemy if there is excessive surrounding noise to confuse us. We will not notice the hacker tripping over a security checkpoint when our logs are full of millions of unimportant activities.

I have seen many organizations make a considerable investment in devices that alert them about anything and everything going on within their environment. Sadly, such security measures create excessive "noise," resulting in thousands of logs and alerts that need to be searched. Meanwhile, the hacker quietly slips in and out of the treasure room, his or her footsteps hidden in the chaos.

In security we listen; but for there to be noise, there must first be stillness. Security strategists from all ages have used silence as a key tool of defense. So much of security relies on the ability to detect enemies as they attempt to enter the castle. Thus, before we learn to combat our foes, we must first learn how to create silence within the environment.

Creating Stillness

Creating stillness requires a level of silence at every location where a security check takes place. This involves the art of alert filtering, making sure we only hear that which is suspicious. When a new security device is put in place, a new operating system is installed, or a new application is created, it must go through a "noise-tuning process." The tuning process should include some post-installation tweaks and adjustments, one of which should be log reduction to provide filtering. Here are some simple guidelines for tuning an object:

Five Steps for Tuning Silence

1. Study the logging and alerting features of an object. Every device and application works a little differently and weighs security events in its own unique way. It is important to understand the classifications and severities of different events, as well as the actions that trigger them.

2. Initially set the logging and alerting mechanisms to be sensitive, thus generating a substantial number of logs and alerts. Let this run for a few days and note the types of alarms and logs that are naturally generated within the environment by "normal activities." You will most likely see many types of traffic that the security devices will consider "suspicious," even though they are harmless.

3. Adjust the settings and apply filters to remove common acceptable events. It is important to be as specific as possible and to not filter out so much that you miss a hacker performing malicious events with similar qualities.

4. Once "normal" events are operating silently, other abnormal events should begin to show up. These events may not be malicious, but they may not be authorized. Commonly logged activities include devices with misconfigured network settings, pointing them outside the local network, and broadcast services like print servers. In cases where traffic is unauthorized but not malicious, it is important to stop the event from happening at the source, rather than filtering the logs and allowing the action to continue. When possible, go to each device and make changes to stop unauthorized activities.

5. Once all "normal" activities are filtered and abnormal activities are stopped, we are ready to start monitoring. It is important to document any filters that were put in place to ignore "normal activities." Others should know what is not being monitored in case there is ever a security issue related to such an activity.

Once all the common events are removed from the environment, we will be much more likely to notice strange events that may indicate an attack. Having finished the tuning process, it will become a part of our daily effort, applying proper filters as new log-generating activities occur.

Tiered Silence

While it is important to maintain silence at security checkpoints, it is also important to maintain a thorough and accurate record of the events taking place. Many of the events filtered out of the logs could potentially play an important role in a future investigation; do we really want to erase them? When filtering events from logs, it is best to not simply discard them, but rather to store them in an unfiltered archive, separate from normal viewable logs. In this case, logging is set to record many types of events that could potentially be of importance in the future. Only the logs that are the most significant, however, are held in the monitoring system; the rest are sent to an archive server. This way, we can filter out all the extra noise, but still have events on record in case we ever need to go back and do a detailed investigation. At a minimum, this should be performed with critical applications and servers, IDSs, and firewalls.

Figure 5.9. Basic log filtering.

Figure 5.10. Filtering with log preservation.

Striking a Balance

There is a delicate balance that must be struck when creating silence within an environment. If enough of the noise is not filtered out, an attacker's steps could be lost in a sea of extraneous logs and false alarms. If too much is filtered, the alerts will never show a hacker's presence. Here are some good higher practices to follow:

1. Watch for common events in logging and alerting systems and determine the cause of them. Who is doing what, where, when, and why?

2. Design a filter for such events, being as specific as is practical. Often, including extra conditions like the time of day for "normal activities" is important in your filters.

3. Before applying them, filters should be presented to management or others within the security group. Follow the Rule of Change (see Chapter 4) since configuring devices to intentionally ignore security events is a very important action. No one should perform such actions alone.