Common QuestionsProblems | Troubleshooting Linux Firewalls

Common Questions/Problems

When enabling a VPN, whether it's IPSEC or something in Layer 4, sometimes the VPN software will change your default route to the VPN. Keep an eye on your routes when starting to perform diagnostics.

Sometimes ISPs will implement no-VPN policies, so if your VPN stops working one day and you have literally changed nothing, check with your ISP to see if it has changed its VPN policy. A sniffer on both ends can help you to diagnose if the VPN packets are going out and if the destination is receiving them.

PPPoE, DSL, and other environments involving encapsulation suffer from issues with MTU path discovery (mentioned in previous chapters). This is a very common problem with VPNs and will manifest itself in all sorts of different ways: poor performance, large transfers of data failing where small ones succeed, a VPN "stalling" (working one minute and stopping suddenly) or authentication issues with VPN applications. The fix in nearly every occasion is to use PMTU and iptables/netfilter to manipulate the Maximum Segment Size (MSS) setting on your firewall with the following rule:

 $IPTABLES -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

In our testing of IPSEC implementations over the years traversing firewalls, we've generally had a relatively small number of specific problems that occur across multiple implementations. When dealing with IPSEC clients or servers, we've found a number of them have specific settings for traversing NAT environments. Always double-check your IPSEC documentation to see if you need to enable different settings for VPN environments.

The following is a sniffer trace of an IPSEC session not responding:

 [root@minimoose root]# tethereal -i eth0 host 66.167.232.50 Capturing on eth0   0.000000 68.98.150.6 -> 66.167.232.50 ISAKMP Identity Protection (Main Mode)   9.994140 68.98.150.6 -> 66.167.232.50 ISAKMP Identity Protection (Main Mode)  29.995130 68.98.150.6 -> 66.167.232.50 ISAKMP Identity Protection (Main Mode)

This is telling us that the originating system is sending out its authentication information, but the server on the other end is not responding for some reason. In this case, we are getting no response at all from the remote end. This could be because of any number of things, from the authentication information not being recognized, to filter rules, to IPSEC just not running on the remote system.

This is what it should look like:

 [root@minimoose root]# tethereal -i eth0 host 68.100.73.75 274.959573 68.100.73.75 -> 68.98.150.6  ISAKMP Identity Protection (Main Mode) 274.960127  68.98.150.6 -> 68.100.73.75 ISAKMP Identity Protection (Main Mode) 275.006148 68.100.73.75 -> 68.98.150.6  ISAKMP Identity Protection (Main Mode) 275.045370  68.98.150.6 -> 68.100.73.75 ISAKMP Identity Protection (Main Mode) 275.196205 68.100.73.75 -> 68.98.150.6  ISAKMP Identity Protection (Main Mode) 275.287978  68.98.150.6 -> 68.100.73.75 ISAKMP Identity Protection (Main Mode) 275.430761 68.100.73.75 -> 68.98.150.6  ISAKMP Quick Mode 275.475328  68.98.150.6 -> 68.100.73.75 ISAKMP Quick Mode 275.581497 68.100.73.75 -> 68.98.150.6  ISAKMP Quick Mode

As you can see, both systems are exchanging information successfully. In this scenario, the problem was that IPSEC was not running on the remote server. It could have just as easily been filter rules in the ISP or on the remote system itself.