Tools Discussed in this Chapter | Troubleshooting Linux Firewalls

DNSMasq (http://www.thekelleys.org.uk/dnsmasq/doc.html). The following is from the DNSMasq documentation:
DNSMasq is lightweight, easy to configure DNS forwarder and DHCP server. It is designed to provide DNS and, optionally, DHCP, to a small network. It can serve the names of local machines which are not in the global DNS. The DHCP server integrates with the DNS server and allows machines with DHCP-allocated addresses to appear in the DNS with names configured either in each host or in a central configuration file. DNSMasq supports static and dynamic DHCP leases and BOOTP for network booting of diskless machines.
DNSMasq is useful if you are in an environment where you need to provide an application layer DNS forwarding service but do not have the time or inclination to use a full-featured DNS server such as BIND.
DHCRelay (http://www.isc.org) is part of the larger DHCP package from the Internet Software Consortium (ISC). We use it to provide an application layer proxy server for DHCP requests in a DMZ environment.

Forwarding DNS Queries to an Upstream/Remote DNS Server

In this configuration, you have a firewall, called Host-A (10.10.10.1), and an internal non-routable network (RFC1918) at 10.10.10.0/24. Currently this network either has no internal DNS servers or DNS servers serving queries for internal hosts. To provide DNS services for Internet addresses, these queries will have to be forwarded to an external DNS server that can do this. There are several methods to accomplish this...

The first method involves using application layer proxies, such as BIND or DNSMasq. Our first example involves using BIND, the Berkeley Internet Name Domain, from http://www.isc.org. BIND is the most commonly deployed nameserver in existence and comes with most major Linux distributions. Install BIND on your firewall.
For Redhat systems, a shortcut is to use the up2date (yum or redcarpet) service.
```
     up2date -i bind     yum install bind     rug in bind 
```
For other systems, consult your vendor's documentation or download and build your own nameserver from http://www.isc.org.
Configure your nameserver as a forwarder. For most systems, such as Redhat/Fedora, this will be from the /etc/named.conf file; otherwise, this file will be located in /usr/local/etc/named.conf. Open this file in your preferred editor and change the following lines:
```
        forwarders {                  11.22.33.44;                  11.22.33.45;          }; 
```
Set these numerical IP addresses to correspond with your upstream Internet responding nameservers.
Configure your nameserver to listen only on your internal interface(s). By default, when you start the nameserver, it will bind to all your interfaces, including the external ones. Open the named.conf file with your preferred editor and modify/create the following lines in the Options section:
```
        listen-on {                10.10.10.1;         }; 
```
Set these to numerical IP addresses that correspond to your firewall's internal interfaces.
Allow DNS requests to your firewall from hosts on your internal network. In our example, the internal network is 10.10.10.0/24 with the following rule:
```
 $IPTABLES -A INPUT -p tcp --dport 53 -i eth1 -j ACCEPT $IPTABLES -A INPUT -p udp --dport 53 -i eth1 -j ACCEPT 
```

Optional Step: Redhat distributions also include the "caching-nameserver" rpm. This rpm will make your firewall a caching nameserver. This can accomplished with the following commands:

     up2date -i caching-nameserver     yum install caching-nameserver     rug up caching-nameserver

The second method involves using DNSMasq, available from http://www.thekelleys.org.uk/dnsmasq/doc.html. DNSMasq is a much lighter weight DNS Forwarder/DHCP server suited for this type of task specifically.

Install DNSMasq (consult the DNSMasq documentation if you are not familiar with how to build or install this package). RPM shortcut: rpmbuild -ta dnsmasq-2.6.tar.gz.
Configure your firewall's /etc/resolv.conf. DNSMasq pulls the servers it will forward DNS queries to from the system wide resolv.conf file. If you want to configure DNSMasq to use a different list of nameservers, load up the DNSMasq config file, /etc/dnsmasq.conf in your favorite editor and change the following line:

     resolv-file=/etc/somefile

where /etc/somefile is your file containing a list of nameservers, one per line, that DNSMasq will forward DNS queries to.

DNS Lookups Fail: Internal Hosts Communicating to an External Nameserver

In this configuration, hosts on the RFC1918 network, 10.10.10.0/24, behind the firewall (Host-A) cannot successfully make DNS lookups.

Before delving into deeper diagnostics, first verify that systems on the internal network can make connections through the firewall based on the IP address. Second, verify that you are using the correct IP address for the external DNS server(s)and that they are actually active.
Assuming that both IP-based connections are successful and that the external DNS servers are active, this typically means two things: the NAT rules are not using connection tracking or that there are filters on TCP/UDP port 53. Step through your firewall rules and verify that these ports are not being blocked.

DNS Lookups Fail: Short DNS Name Lookups WorkLong Name Lookups Do Not

This problem might also manifest itself as Unix hosts being able to perform short lookups only and Windows hosts not being able to perform any lookups at all. In general, DNS queries that are 512 bytes and below will use UDP port 53 and above that, TCP port 53. Windows systems will use both the TCP and UDP ports, regardless of size. (We've yet to fathom why it will use one over the other.)

The solution here is to ensure that your firewall rules are not blocking TCP port 53.

DNS Lookups Fail: Nameserver Running on the Firewall

In this first configuration, the firewall Host-A is running a local DNS server, either handling requests for itself or for internal users on the network 10.10.10.0/24. Local zones contained on the server function correctly; lookups on Internet records fail.

Regardless of whether this server is acting as a forwarder or an authoritative nameserveror even as a DNS proxy, all lookups to external zones will fail if the firewall does not a) have rules allowing it to make external connections and b) have connection tracking enabled.

In both cases, we can solve this issue with these rules:

 $IPTABLES -A OUTPUT -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT $IPTABLES -A INPUT  -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT

Note that this uses an OUTPUT rule. This is because the connection is originating on the firewall itself. This is a more general purpose rule that is going to allow all connections originating on the firewall outbound, which might not be what you want. An example of a more restrictive rule follows:

 # where eth0 is the external interface on our firewall $IPTABLES -A OUTPUT -o eth0 -p udp dport 53 -m state state NEW -j ACCEPT $IPTABLES -A OUTPUT -o eth0 -p tcp dport 53 -m state state NEW -j ACCEPT $IPTABLES -A INPUT -i eth0 -p udp dport 53 -m state state NEW,ESTABLISHED,RELATED -j ACCEPT $IPTABLES -A INPUT -i eth0 -p tcp dport 53 -m state state  NEW,ESTABLISHED,RELATED - j ACCEPT

DNS Lookups Fail: Nameserver Running on the Internal and/or DMZ Network

In this configuration, the internal/DMZ nameserver Host-B (10.10.10.53) cannot make DNS lookups through firewall Host-A to root servers. Much like the example here, this is most likely being caused by two scenarios: a) there are no rules allowing this service to connect outbound or b) assuming this service is allowed to connect outbound, connection tracking is not enabled.

First, assuming that this firewall has a restrictive policy on what connections are allowed outbound, we ensure that the service is allowed through the firewall with these rules:

 # where eth1 is the internal interface on the firewall $IPTABLES -A FORWARD -p tcp dport 53 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT $IPTABLES -A FORWARD -p udp dport 53 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT $IPTABLES -A FORWARD -p udp -i eth1 -s 10.10.10.53 -d 0/0 dport 53 -j ACCEPT $IPTABLES -A FORWARD -p tcp -i eth1 -s 10.10.10.53 -d 0/0 dport 53 -j ACCEPT

Misleading rDNS Issue: New Mail, or FTP Connections to Remote Systems Take 30 Seconds or More to Start

This is not actually a DNS issue at all, but enough people associate it with reverse DNS lookups that we decided to put it in this section. The extended description of this problem is that in general, other services such as web browsing or instant messaging are very responsive. Only connections that appear to involve reverse DNS lookups on your firewall, such as an FTP connection for example, take 30 seconds to start. What is probably going on here is that the server on the other side is making an identd (Auth) request to your server, and it's timing out waiting for a reply.

On a host-only firewall, there are three fixes for this. The first is the least favorableto run an identd server on your host. This would both add a service that could be exploited and leak information about your system, which is not desirable. The second method of fixing this is to add in a rule to REJECT any identd requests to your system.

Following is an example:

 # this will send an icmp response indicating that the identd/auth service is not available $IPTABLES -A INPUT -p tcp dport 113 -j REJECT

The third option is to add in a connection tracking rule, which also works for firewalls handling NAT/MASQUERADED traffic:

 $IPTABLES -A INPUT -m state state ESTABLISHED,RELATED

DHCP: Dynamically Updating Firewall Rules with the IP Changes

This is a fairly common scenarioyou have a firewall with a dynamically assigned IP address, and when that IP address changes, you need to update your firewall rules accordingly. Most DHCP clients already have the hooks built in to call an external script when an IP address is renewed. We will cover two of the more popular clients, DHClient, and DHCPcd in the following examples:

DHClient is the default DHCP client for Redhat and Fedora systems (this is generally already configured by default):

Create the file /etc/dhclient-exit-hooks.
Load this up in your favorite editor and add in the reference to your firewall script, example:

      sh /etc/rc.d/rc.firewall

DHCPcd is another popular DHCP client, which you'll find on distributions such as Gentoo:

Load up /etc/dhcpc/dhcpcd.exe into your favorite editor
Locate the switch case at the end. In this you'll note several conditions on the state of the interface (Just activated/New IP, renewal with the same IP, and so on)
In the New IP case, insert a call to your firewall script. For example,

      sh /etc/rc.d/rc.firewall

Note

Ensure that your firewall script(s) always flush their rules prior to reloading.

Ensure something like the following is at the top of your firewall scripts.

 $IPTABLES -F $IPTABLES -X

The first line will flush your tables; the second will delete the user-defined chains.

Blocking Outbound DHCP

This question has been asked enough times that it warranted a response in this book. If you have read the Layer 2 chapter of this book already, you might already know a bit about the issues with blocking outbound DHCP in a bridging environment (or blocking DHCP responses by MAC address for example). This section assumes that you are posed with the problem of keeping DHCP requests confined to a physical segment, separated by a non-bridging firewall. Here's the good newsDHCP requests are not going to cross that firewall. If that's what you were worried about, you're done; worry no longer. If DHCP is running on the server itself, then all that is required is a simple reconfiguration of the DHCP daemon.

In the following example, our firewall, Host-A, has two interfaces eth0 (external), and etH1 (internal, 10.10.10.0/24). This configuration is using the ISC DHCP daemon (http://www.isc.org), which is commonly the default DHCP server in most Linux distributions. To restrict DHCP requests to the internal network, in this case interface eth1, you would start dhcpd with the interface name to which you want it to listen.

For example: /usr/sbin/dhcpd eth1

On some distributions, such as Redhat systems, you can also set this as a configuration file:

Load up /etc/sysconfig/dhcpd in your favorite editor and set the following
```
 DHCPDARGS="eth1" 
```
where etH1 responds to the physical interface you want DHCP to listen on. If other interfaces were defined in this file, your job is finished at this point.

This is just an example of what the primary DHCP config file looks like:

 subnet 10.10.10.0 netmask 255.255.255.0 {      option routers               10.10.10.1;      option subnet-mask           255.255.255.0;      range dynamic-bootp 10.10.10.100 10.10.10.200;      option time-offset           -18000; # Eastern Standard Time      option domain-name-servers    10.10.10.1;      default-lease-time 21600;      max-lease-time 43200; }

The example here configures a DHCP range of 10.10.10.100-10.10.10.200 with a lease time of 12 hours. This also sets the default route to our firewall's internal interface, 10.10.10.1, and the default nameserver, 10.10.10.1.

DHCP: Two Addresses on One External Interface

This is a rather unique question. It was posted to the netfilter mailing list, and we felt that it was bizarre enough to require an honorable mention in the book. If this section is actually useful to you, and you actually did this, please drop us an email about it and why.

This being said, the topic of the post was about getting an additional DHCP address assigned to the external interface on his firewall to get this assigned to his gaming console. This device presumably needed a "clear shot" to the Internet to host games, and port forwarding these ports from the regular IP would have broken other services. Unfortunately not knowing what these ports were, it's hard to say if that was really as insurmountable a port forwarding problem or not, but that's neither here nor there. The initial attempt to lease a second IP by using an IP alias won't work because the IP aliased device (eth0:1) doesn't have a MAC address, which is what the DHCP server on the other end is looking for. According to the author, the following was the wrong way to do it:

The following works:

 /sbin/dhcpcd -t 60 -h CO -d eth0

The following, however, does not work because there's no MAC address for it:

 /sbin/dhcpcd -t 60 -h CO -d eth0:1

At this point, we're in the realm of "there's more than one way to do it." We could use ebtables (Chapter 15) and do some MAC layer NAT-ing of the device we'd like to get a DHCP address for. However, it turns out that the user space client itself can send the MAC address along with it.

 /sbin/dhcpcd -R -N -t 60 -h CO -I 00:11:22:33:44:55 eth0:1

00:11:22:33:44:55 is the MAC address you'd like to send out. The -I flag is the "Client Identifier," which is what tells the DHCP server on the other end the unique ID, typically the MAC address.

At this point you would need to customize the firewall rules based on IP address, using the IP alias, eth0:1, which is not going to work because iptables/netfilter uses the physical, rather than logical, device to match its rules against.

In general, this configuration is probably a bad idea. A safer setup would be to add a third interface and establish a DMZ for this device.

DHCP: Redirect DHCP Requests to DMZ

In this configuration we have a firewall (Host-A), a DHCP server (Host-B, 192.168.1.67), a DMZ segment (192.168.1.0/24), and an internal network 10.10.10.0/24. The strategy is to locate the DHCP server in the DMZ segment, potentially to provide some added security for this device or for some other infrastructure reason and to secure it from the hosts on the internal network. In this scenario, it is the "internal" hosts that are not to be trusted. This is not a safe configuration for an Internet-reachable DMZ. This is a DMZ against the internal network only.

There are numerous reasons for this sort of configurationgenerally they stem from providing DHCP services for an untrusted network or perhaps to create some sort of fault tolerant DHCP cluster for a large network. Regardless, when we say DMZ, we aren't talking about something Internet hosts can reach. A DHCP server on such a DMZ, communicating to internal hosts, would make a fantastic vector for attack on internal systems.

Figure 18.1 should serve to illustrate the utility of this configuration.

Figure 18.1. Redirect DHCP requests to DMZ.

While it is possible to use something such as ebtables (Chapter 15) to directly bridge in DHCP requests to our 192.168.1.0/24 network, it also would more or less open up the internal DMZ network to several vectors of attack. Instead, in this configuration we will use an application layer proxy server called dhcrelay to relay our client requests to the real DHCP server.

First, the proxy server, dhcrelay, is included in the DHCP package from the Internet Software Consortium (ISC).

From the ISC documentation on dhcrelay:

The DHCP Relay Agent listens for DHCP and BOOTP queries and responses. When a query is received from a client, dhcrelay forwards it to the list of DHCP servers specified on the command line. When a reply is received from a server, it is broadcast or unicast (according to the relay agent's ability or the client's request) on the network from which the original request came.

This is the standard DHCP server that comes with most Linux distributions, including Redhat/Fedora, SuSE, Mandrake, and so on. The first step in this configuration is to ensure that this is installed on the server:

Redhat/Mandrake/SuSE users will find this in the dhcp rpm (yum, up2date, and redcarpet shortcuts apply).
```
 up2date -i dhcp yum install dhcp rug in dhcp 
```
For Redhat users, load the configuration file, /etc/sysconfig/dhcrelay, into your preferred editor and modify the following:
```
 INTERFACES="eth1" DHCPSERVERS="" 
```
Where eth1 is the interface dhcrelay should listen for DHCP requests.
For other operating systems, consult your documentation on where this is set, or start dhcrelay with the following example:
```
 dhcrelay -i eth1 192.168.1.67 
```

Ensure that the firewall rules allow connections on the DHCP ports, 67 and 68.

 # where eth1 is your 10.10.10.0/24 network $IPTABLES -A INPUT -m state state ESTABLISHED,RELATED -i eth1 $IPTABLES -A INPUT -p tcp dport 67:68 -i eth1 -j ACCEPT $IPTABLES -A INPUT -p udp dport 67:68 -i eth1 -j ACCEPT