Recipe10.4.Enabling IPsec on an Exchange Server 2003 Cluster

Recipe 10.4. Enabling IPsec on an Exchange Server 2003 Cluster

Problem

You have one or more front-end servers communicating with a clustered back-end server, and you want to protect IMAP, POP, or HTTP traffic passing between them.

Solution

Using a graphical user interface

Open the Registry Editor (regedit.exe).
In the left pane, navigate to:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PolicyAgent\Oakley
Click on the Oakley subkey.
In the right pane, right-click the NLBSFlags entry and select Modify.
In the Value data field, type 1 and click OK.
Close the Registry Editor.

Using the command line

On the server where you want to enable IPsec, run the following command:

 > reg add HKLM\System\CurrentControlSet\Services\PolicyAgent\Oakley  /t REG_DWORD /v "NLBSFlags" /d "1" /f

Discussion

You can use IPsec as described in Recipe 10.3 to protect IMAP, POP, and HTTP communications between front- and back-end servers. However, if the back-end server is a cluster, the ordinary setup method doesn't work well. That's because the security association (SA) established between the two servers has to be renegotiated when failover occurs. The default interval for SA renegotiation is five minutes, which means that until that interval elapses, the FE and BE will be unable to communicate. This can take up to six minutes: five minutes for the timer to elapse, plus one minute for the IKE protocol to decide that it needs to establish a new SA. In Exchange 2000, there was no way to fix this, meaning that Microsoft didn't support the use of IPsec in this configuration. However, in Windows 2003, you can adjust the renegotiation interval down so that reestablishment takes a maximum of two minutes: one minute for the idle timer to expire, plus one minute for IKE renegotiation.