Monitoring and Maintaining the System

---

Even after you've installed your firewall, you can't just forget about it. There's a certain amount of maintenance that you have to perform, and you should always keep an eye on the system, watching for potential attacks and even compromises. This section discusses some common techniques that you might wish to make use of.

Configuring the System Logger

Perhaps the most important thing you can do to keep your firewall system (or any system) secure and running is to check its log files periodically. However, since you probably aren't going to be logging into the firewall computer a great deal, it would be nice to be able to view the firewall's logs from another computer you use more regularly. This can be done by configuring the firewall to use a remote syslog.

The basic idea is that you will configure a server computer to accept log messages, and then configure the firewall to send its log messages to the internal server, rather than log them to its local disk. The changes in both cases are quite easy to make.

To make these changes, though, you have to know the IP address of the internal server. In the example in this book, the IP address of the server is 192.168.0.3. (In the forthcoming section "Configuring a DHCP Server", this address was fixed, and does not change.) If your server is located at a different address, be sure you account for it.

Configuring the Internal Logging Server

Red Hat Linux includes an RPM package called "sysklogd" that implements a system logger. This package includes a typical implementation of the Unix syslog facility. To configure the internal server to act as a network logger and accept remote messages from the firewall, you simply have to enable that behavior via a runtime flag passed to the syslog daemon when it starts up.

Note 

This section obviously assumes that your internal server is also running Red Hat Linux. If this is not the case, these instructions may not apply, and you may have to configure syslog some other way.

You can probably guess which file you're going to have to modify: /etc/sysconfig/syslog, on the server to which the firewall will be sending its log messages. (Is this becoming old hat yet? Good!) In that file, locate the following line:

 SYSLOGD_OPTIONS="-m 0" 

You need to change this line by adding the -r flag to the variable containing flags that get passed to syslogd when it is started from the script in /etc/rc.d/init.d/syslog. (This flag instructs the syslog server to accept remote connections.) The line should now look like this:

 SYSLOGD_OPTIONS="-m 0 -r" 

You should now restart syslog on that server using the command service syslog restart.

Punching Through the Firewall

At this point, syslog on your logging server will happily accept connections from the firewall. However, if you have the local firewall enabled on the logging server itself (which you should), then the firewall will still be preventing the logging server from ever actually seeing the packets. There are two steps required to correct this.

First, edit the file /etc/sysconfig/ipchains on the logging server, which is the configuration file containing actual rules (in the older ipchains syntax) for the local firewall. You need to add this line to the file:

 -A input -s 192.168.0.1 syslog -p udp -i eth0 -j ACCEPT 

This line adds a rule to the firewall that permits syslog packets into the system. (As you can see, this ipchains syntax looks very similar to iptables.)

The second change you'll need to make to the logging server is to correct an idiosyncracy in Red Hat's /etc/sysconfig/network-scripts/ifup-post file. Among other things, this file creates iptables rules that explicitly allow DNS packets from the DNS servers specified by the DHCP server. This has to be done dynamically, rather than stored with all the other local firewall rules in /etc/sysconfig/ipchains, because the DNS servers could potentially change every time the DHCP server renews the lease. The ifup-post has code to check to see if a rule for the nameserver already exists, and if one does, then ifup-post takes no action. Unfortunately, in this case study, the firewall is also the nameserver, and so the rule you just added to the /etc/sysconfig/ipchains file refers to the nameserver. This will prevent ifup-post from correctly allowing access to the DNS. In other words, you'll have no nameserver support!

Fortunately, the fix is simple and requires a simple change to the /etc/sysconfig/network-scripts/ifup-post file. To make the change, find this line:

 if ! ipchains -L input -n | grep -q $nameserver ; then 

Change the line to this:

 if ! ipchains -L input -n | grep "\<53\>" | grep -q $nameserver ; then 

(You're adding the command grep <\53\> to the line.) Once you've made these two changes, the local firewall on your server will allow access to both DNS and syslog. Red Hat has been alerted to this issue, and may—or may not—fix it in a later release.

Now that you've configured the relevant files and corrected the ifup-post script, you can restart everything that needs to be restarted with the service command. Specifically, you need to restart the ipchains local firewall, the syslog service (so that it listens to connections from the network firewall server), and the network. That is, run these commands:

 $ service ipchains restart $ service syslog restart $ ifdown eth0; ifup eth0 

After that, your internal server computer should accept syslog connections from the firewall server, and record its log for you! In the next section, you'll configure the network firewall server to start sending its log messages to your newly prepared logging server.

Configuring syslog on the Firewall

Once you have syslog on the internal server set to accept log messages from the network, you have to configure the firewall to actually send its log messages to the server. You can accomplish this by editing the file /etc/syslog.conf. This file contains rules and patterns that the syslog server uses to determine what messages get logged where. These rules are stored directly in /etc/syslog.conf instead of a file like /etc/sysconfig/syslog because they are generally not volatile; it's unusual to change the logging rules from the defaults, though you are in this case. For the purposes of your firewall, you really just want to send all messages to the internal server.

It's very simple to configure syslog to send its messages over the network to a separate logging server. First, comment out all lines in /etc/syslog.conf that aren't already commented, by adding a "#" at the start of these lines. This disables all the rules that would normally cause log messages to be placed in files such as /var/log/messages and /var/log/secure.

Once you've commented out the default rules, just add this rule; don't forget to change the IP address if your server is not at 192.168.0.3:

 *.*       @192.168.0.3 

This rule unconditionally sends all system log messages to the host 192.168.0.3 over the network, which means that syslog will not log any messages to disk. If you prefer to only have some messages sent to the internal logging server, or if you wish to have it log to both the network and disk, and so on, you can add additional rules to /etc/syslog.conf as appropriate. The format is not complicated, and by now you've seen enough different examples of configurations that you shouldn't have any trouble adding these rules yourself. Just check the manual page for syslog.conf!

Once you've made these changes, all the log messages for your firewall will appear in the log file of your server (or whatever host you instructed the firewall to log to). You can then view these log messages along with your server's log messages, and you won't have to log in to the firewall to check on its status.

However, you should keep a couple things in mind if you choose to use this approach. First, if your internal server is down, any log messages generated by your firewall will be lost! For this reason, you may wish to leave the firewall configured to store its log files on its local disk. Second, all the log messages from the firewall will be mixed in with the messages on the server, which can be a bit of a jumble; keep that in mind when viewing the logs.

Monitoring Firewall Traffic

The firewall rules discussed earlier in the chapter in the section "Presenting an iptables Script" include rules that log incoming packets to the syslog system logging facility. Meanwhile, the previous section just showed you how to configure the firewall's system log to be transmitted to a second server, which is where you'll view the log files. Shortly after you firewall is up and running, you'll undoubtedly start seeing many log messages reported by the kernel about packets that are dropped. It's a good idea to know how to interpret those lines.

The goal in logging any packets that get dropped by the firewall is to keep you advised of the goings-on that affect your firewall. Since the firewall rules were constructed to allow only legitimate traffic and nothing more, anything that gets dropped must not have been legitimate. This is usually for one of two reasons: Either a computer or network somewhere on the Internet is misconfigured and sending you erroneous packets, or you're being probed or attacked by a hostile host.

Unfortunately, there's no good way to tell the difference. Various vendors produce software that alleges to identify port scans—finding the needle in the haystack, as it were—with varying degrees of success. In the end, the most reliable tool is human judgment. So, when you look at your logs at dropped packets, it's up to you to decide whether it's innocuous or dangerous.

A typical log entry generated in response to a dropped packet looks something like this (except all on one line):

 Mar 25 23:06:04 192.168.0.1 kernel: IN=eth0 OUT= MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx SRC=xxx.xxx.xxx.xxxDST=xxx.xxx.xxx.xxx LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=31194 DF PROTO=TCP SPT=3780 DPT=500 WINDOW=65535 RES=0x00 SYN URGP=0 

This line contains all relevant files for the packet. The ones you're most likely to focus on are the "src="/books/4/36/1/html/2/ value, which is the IP address of the host that sent the packet; the "DST=" value, which is the packet's destination; and the "SPT=" and "DPT=" values, which are the source and destination ports, respectively. Most of the other fields indicate which TCP flags were set and other low-level information like the time to live (TTL) and packet size settings. This information can be used to identify attacks; for example, one attack might have the TCP/IP SYN and RST flags set, so any packets that meet that description are probably hostile packets.

This example is of a simple TCP port scan of port 500. Since you probably don't have anything running on port 500, you can probably assume it was a person or program scanning your machine for vulnerabilities. The fact that it got logged, however, means that it also got dropped, so the attacker got completely stonewalled. From his perspective, it will appear as though your computer doesn't even exist on the network!

Dropping vs. Rejecting Packets

The firewall in this chapter is configured to drop all packets that aren't explicitly accepted. This means that the packet is simply ignored, and no response of any type is sent. Because of this, the host sending the packet has no way of knowing that the firewall system even exists (unless, of course, it sends a packet to a port that is accepted). This is typical behavior for a firewall, and provides the greatest security.

However, there is another alternative. Rather than drop the packets, you can "reject" them. Rejected packets are not "silent" as are dropped packets; rejected packets actually send a response in the form of an Internet Control Message Protocol (ICMP) packet to the host that sent the packet, whereas dropped packets are simply discarded without notifying the remote computer (that is, dropped packets are silent). This may (or may not) reduce security a little, but it is sometimes required behavior for more sophisticated firewalls. For the SOHO application discussed in this book, it's adequate to simply drop packets instead of rejecting them, but it's a good thing to keep in mind.

Packets are rejected by using -j REJECT instead of -j DROP in the iptables commands. For more information, see Ziegler's book, Linux Firewalls, Second Edition, referenced in the bibliography at the end of this chapter.

Classifying Packets

Identifying attacks from the mess of data you get is not easy. As an end user, you may not even really care; your objective in watching the logs is just to keep an eye out for anomalies. Identifying hostile attack patterns from the normal flow is a huge topic, and a matter of big business in some cases. Unfortunately, there really isn't room to address it properly here; the best you can do is to watch your logs for a while to get a "feel" for what your normal traffic looks like, and then watch for deviations from that pattern. For more information on this topic, your best bet is again probably Ziegler's text, or the various online resources such as those mentioned in the bibliography at the end of this chapter..

What is the appropriate way to respond to these cases? Well, that's a difficult question, and is a matter of personal preference. If it bothers you, you can chase down the probe or attack and try to get action taken against the perpetrator by her Internet service provider. Or, you can just adopt a low profile and only worry about the big attacks. This is a very gray area, and it's a decision you have to make for yourself. For more information, you should check out the sites and documents listed in the bibliography at the end of this chapter.

Care and Feeding of Your Firewall

In many ways managing a firewall (or really any computer, even your desktop PC) is a lot like maintaining a car; periodically you have to change the oil and rotate the tires. This section lists some of the things you should do periodically to keep your firewall system secure.

The automotive maintenance analogy really only goes so far. The reason you need to maintain a car is because over time normal wear and tear will reduce its performance—to dangerous levels, if left unaddressed. However, your firewall will never "wear out" through normal usage; the computer will just keep doing what you told it to do, until you tell it to stop or its hardware fails!

The main reason you need to maintain a computer isn't because it wears out, it's because if you leave it alone, the world will pass you by. Exploits that were not known when you installed your firewall may become widespread later. For example, there have been several vulnerabilities identified in the Linux kernel's netfilter code since it's release. Once these weaknesses are known, they can be used against your machine. So, it's important to know as much as you can about what your enemies (that is, hostile crackers) know, and defend yourself.

In other words, knowledge is power. The remainder of this section describes some useful sources of knowledge. You should consider availing yourself of these sources, in order to practice safe computing.

The Bugtraq Mailing List

The Bugtraq mailing list, well, tracks bugs. The list is run by the SecurityFocus web site at http://www.securityfocus.com, and is a forum for individuals and organizations to report security and other major issues in popular software. The list includes information on most Unix-like systems as well as the Microsoft operating systems. The Bugtraq lists are highly regarded in the industry, and you should seriously consider subscribing to the relevant lists for each operating system you have installed on your network.

The Bugtraq mailing lists can be found at online.securityfocus.com/cgi-bin/subscribe.pl.

Vendors’ Lists

Another valuable resource can be services provided by your operating system vendor. For example, Red Hat, Inc. tracks errata in its distribution, releasing critical updates for bug fixes and security. (Red Hat's up2date service makes it easy to download and install these errata. If you happen to be modifying this case study for use on a Debian system, you might use apt-get instead.)

Every operating system or distribution will provide updates and bug fixes periodically. It always pays to keep your system current unless you have an explicit reason not to. URLs to the vendors' support pages for Red Hat Linux, Slackware Linux, and Debian GNU/Linux are provided in Table 16-3.

Note 

The URL for Slackware Linux is a link to Slackware's mailing lists page; you'll have to subscribe to the "slackware-security" list for security information.

Table 16-3: Linux Distribution Update Pages

DISTRIBUTION	URL FOR UPDATE INFORMATION
Red Hat Linux	http://www.redhat.com/apps/support/errata/index.html
Slackware Linux	http://www.slackware.com/lists/
Debian GNU/Linux	http://www.debian.org/security/

Other Sources

As you follow these sites and lists and delve into the community a bit, you'll find that there are many more similar sites out there. Most sites have a certain slant on security-related issues, so it's definitely a good idea to look around and find the sites you like best. Generally, they all have the same information, so don't try and subscribe to them all—eventually any given bit of information gets around.

It doesn't take a great deal of time to stay on top of your system's security. Taking a few minutes out of each day to make sure that you're not vulnerable to any well-known exploits is a responsibility of any Internet user. After all, if your system is compromised, it can be used to attack mine, and that's not the way I like to be introduced to my readers!

Security in Depth

Your firewall is not Athena's Aegis, unfortunately; you're not invincible just because you have a firewall. If you install the firewall as described in this chapter, you'll have gone a long way toward shutting out casual attackers—the so-called "script kiddies." However, almost no system is guaranteed to keep out a determined attacker, and it's possible that a later exploit might be uncovered that even makes your firewall vulnerable. For these reasons, don't just install the firewall and stick your head in the sand. There is a principle known as security in depth, or security in layers, that you should follow.

Simply put, security in depth means that you don't rely on any single source for your security. For example, if you rely solely on a firewall and your firewall is compromised, your entire network is laid bare and unprotected. To avoid this, you must secure each system, not just the firewall.

For example, if you're running a Red Hat Linux system inside the firewall, don't disable the local firewall on it; there's really no need to, and if in the unlikely event that the external firewall is compromised, you'll still have the internal system's built-in protection. Later versions of Microsoft Windows also include a basic internal firewall; don't disable these, for the same reasons.

You should also keep internal systems up to date on security patches, as well. Again, if that external firewall is somehow compromised, an attacker still can't do anything to your internal systems if they aren't vulnerable to any exploits.

It all boils down to that old saw, "Don't put all your eggs in one basket." It may be easy to dismiss this as paranoia, or as a "managed risk" you're willing to take. However, systems are cracked all the time, and when it happens to you, you'll wish you had been more careful. Don't learn that the hard way.

Once you've got basic security in hand and are keeping up with the latest information relevant to your system, you can start to think about some more advanced issues. The next section describes some of these.

---