16.4 Proxy ARP
16.4 Proxy ARP
In order to avoid the problem of overloading the member on which the device using the MAC address physically resides, we deem the selected cluster member as the "proxy ARP master" for the cluster. The proxy ARP master cluster member will be responsible for handling incoming requests for access to the cluster alias and routing the requests to other cluster members where appropriate (see Figure 16-1). Only one cluster member at a time will function as the proxy ARP master, although any member of the cluster may take over this responsibility if the current master fails or is removed from the cluster.
Figure 16-1: Cluster Alias IP to MAC
When a new proxy ARP master is selected, the MAC address that will be sent back in the case of an ARP request from a network client will be different than it was prior to the proxy master change. The clients will not send out an ARP request if the IP address to MAC translation is already present in the client's ARP cache. The ARP cache on a client can potentially be holding the wrong MAC address for the ~20 minutes it takes for the ARP cache to decay. Therefore, a client could be sending datagrams to a non-existent MAC address for a relatively long time before it figures out that the MAC information in its cache is incorrect (see Figure 16-2). Not a good thing.
Figure 16-2: Gratuitous ARP Ignored or Denied
16.4.1 Gratuitous ARP
The cluster software tries to compensate for this by sending out a gratuitous ARP broadcast to inform all network nodes that they need to replace an entry in their ARP cache. This eliminates the potential havoc caused by the client ARP cache being behind the times.
The following example illustrates gratuitous ARP. All output is generated from a client node. Note how the hardware address associated with babylon5 changes as the current proxy ARP server in the cluster is shutdown.
Note that cluster member sheridan's tu0 interface MAC address ends in "3a:29", while cluster member molari's tu0 interface MAC address ends in "12:31".
# rsh sheridan /usr/sbin/netstat -I tu0 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll tu0 1500 <Link>0:10:64:30:3a:29 452512 0 267120 2 0 tu0 1500 DLI none 452512 0 267120 2 0 tu0 1500 192.168.0 sheridan 452512 0 267120 2 0
# rsh molari /usr/sbin/netstat -I tu0 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll tu0 1500 <Link> 0:10:64:30:12:31 1244810 0 595031 3 0 tu0 1500 DLI none 1244810 0 595031 3 0 tu0 1500 192.168.0 molari 1244810 0 595031 3 0
# ping clualias PING clualias.dec.com (192.168.18.23): 56 data bytes 64 bytes from 192.168.18.23: icmp_seq=0 ttl=64 time=1 ms 64 bytes from 192.168.18.23: icmp_seq=1 ttl=64 time=0 ms ----clualias.dec.com PING Statistics---- 2 packets transmitted, 2 packets received, 0% packet loss round-trip (ms) min/avg/max = 0/0/1 ms
Before shutting down sheridan (the current proxy ARP master), the client has the "3a:29" address in its ARP cache.
# arp clualias clualias (192.168.18.23) at 00-10-64-30-3a-29
Shutdown sheridan!
After shutting down sheridan, the client sees the "12:31" address in its ARP cache.
# arp clualias clualias (192.168.18.23) at 00-10-64-30-12-31
Client access still works seamlessly.
# ping clualias PING clualias.dec.com (192.168.18.23): 56 data bytes 64 bytes from 192.168.18.23: icmp_seq=0 ttl=64 time=0 ms 64 bytes from 192.168.18.23: icmp_seq=1 ttl=64 time=0 ms ----clualias.dec.com PING Statistics---- 2 packets transmitted, 2 packets received, 0% packet loss round-trip (ms) min/avg/max = 0/0/0 ms
Can't get to sheridan anymore (member is down).
# rsh sheridan /usr/sbin/netstat -I tu0 sheridan: Connection timed out
Figure 16-3: Client ARP Cache Refreshed through Gratuitous ARP
Confirming molari's accessibility and MAC address.
# rsh molari /usr/sbin/netstat -I tu0 Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll tu0 1500 <Link> 0:10:64:30:12:31 1245087 0 595328 3 0 tu0 1500 DLI none 1245087 0 595328 3 0 tu0 1500 192.168.0 molari 1245087 0 595328 3 0
However, it is possible that the client node will be running an IP software package that ignores gratuitous ARP broadcasts. If this is the situation at your site, you will need to set up a phony MAC address that does not change if the proxy ARP master changes. This is, in effect, a virtual (or fake) MAC address. Thus this very important option is referred to as Virtual MAC or vMAC.
16.4.2 Virtual MAC (vMAC)
The goal of vMAC is to allow client systems to reference an IP address that resolves to a MAC address that does not change as the proxy ARP mastering responsibilities move from one cluster member to another over the life of the cluster (see Figure 16-4). This allows clients whose IP software is designed to ignore gratuitous ARP requests to interact with the cluster using the standard cluster alias mechanisms. This requires no special efforts from the client. But the cluster members must have certain vMAC variables set in /etc/rc.config.common file.
Figure 16-4: vMAC in Action
Before we take a look at how to set up vMAC, it's important to realize that your Ethernet device is not as simple as you may have thought. Earlier we described the device as having a "burned on" MAC address that ends up in a client system's ARP cache associated with the IP address assigned to the interface. While this is essentially true, the capabilities go beyond that. You may be aware that you can set up an interface with more than one IP address (called an IP alias – not to be confused with a cluster alias, mentioned earlier in this chapter). The interface can also be assigned an additional MAC address using the "physaddr" or "vphysaddr" options to the ifconfig(8) command (see the ifconfig(8) reference page for more information).
You can probably see where we are taking you at this point. In order to support vMAC, the current proxy ARP master will have its network interface configured with a MAC alias that will be associated with the cluster alias IP address. If the proxy ARP master cluster member fails, a surviving member will automatically be selected as the new proxy ARP master for the cluster alias using the same vMAC address. This means that the clients will not need to change anything in their ARP caches. The MAC address they currently have will still be correct. Completely transparently to the clients, the proxy ARP master on the cluster has changed.
All cluster members will be configured identically to support vMAC by setting VMAC_ENABLED to "yes" in their /etc/rc.config.common file.
# rcmgr –c set VMAC_ENABLED yes
By default, the vMAC address will be formed as follows: Take the cluster alias IP address and convert it to hex. (192.168.18.23 becomes C0:A8:12:17). Then prepend the default vMAC prefix, which is AA:01, yielding the entire vMAC address of AA:01:C0:A8:12:17. If desired, the vMAC prefix value (AA:01) can be altered by setting VMAC_PREFIX to some other value in the /etc/rc.config.common file.
# rcmgr –c set VMAC_PREFIX "BB:02"
When a client tries to ping 192.168.18.23, and there is no current translation for this address in the client's ARP cache, the client sends out an ARP request broadcast asking "who has" 192.168.18.23. The current proxy ARP master on the cluster will respond with the vMAC address. The client will stash that information in its ARP cache for future use. The client forms a datagram with the destination MAC address of AA:01:C0:A8:12:17, which is picked up by the current proxy ARP master in the cluster. The proxy ARP master may service the request itself, or may route the request to another cluster member.
The routing decisions will be discussed in the next section.
The following example shows vMAC being enabled in a two-member cluster using the "cluamgr –r vmac" command. The proxy serving member is then shut down. Note that the client sees no change in its ARP information.
The following rcmgr command will ensure that vMAC is enabled as the system boots. The "-c" option causes an entry to be placed in cluster-wide /etc/rc.config.common file.
[molari] # rcmgr –c set VMAC_ENABLED yes
[molari] # rcmgr –c get VMAC_ENABLED VMAC_ENABLED="yes"
The clu_alias script (available at the BRUDEN or Digital Press website – see Appendix B) enables vMAC on all cluster members by running the "/usr/sbin/cluamgr –r vmac" command on each system in the cluster.
# clu_alias -r vmac member1: (molari) "/usr/sbin/cluamgr -r vmac" member2: (sheridan) "/usr/sbin/cluamgr -r vmac" - accessing via sheridan-ics0"
In the following sequence, the client successfully pings the cluster alias (babylon5) and caches the vMAC address.
# ping babylon5 PING babylon5.dec.com (192.168.0.70): 56 data bytes 64 bytes from 192.168.0.70: icmp_seq=0 ttl=64 time=1 ms 64 bytes from 192.168.0.70: icmp_seq=1 ttl=64 time=0 ms
# arp babylon5 babylon5 (192.168.0.70) at aa-01-c0-a8-00-46
The following sequence shows that the cluster member molari is the current proxy ARP master.
[molari] # /sbin/arp babylon5 babylon5 (192.168.0.70) at aa-01-c0-a8-00-46 permanent published
[sheridan] # /sbin/arp babylon5 babylon5 (19.168.0.70) -- no entry
Shut Down molari (the current proxy ARP server)! As you would expect, cluster member sheridan becomes the new proxy ARP server. However, this time there is no change in the client's ARP cache since the aa-01-c0-a8-00-46 address is a vMAC address.
# ./clu_arp babylon5 ================================================================================ member1: (molari) "/sbin/arp babylon5" - accessing via molari-ics0 -------------------------------------------------------------------------------- babylon5 (192.168.0.70) at aa-01-c0-a8-00-46 permanent published ================================================================================ member2: (sheridan) "/sbin/arp babylon5" -------------------------------------------------------------------------------- babylon5 (192.168.0.70) -- no entry
Now, back on the client, note that it has no knowledge of the upheaval that has just occurred in the cluster. The ARP information is the same as it was when molari was the proxy ARP master.
# arp babylon5 babylon5 (192.168.0.70) at aa-01-c0-a8-00-46
All is well from the client's perspective.
# ping babylon5 PING babylon5.dec.com (192.168.0.70): 56 data bytes 64 bytes from 192.168.0.70: icmp_seq=0 ttl=64 time=0 ms 64 bytes from 192.168.0.70: icmp_seq=1 ttl=64 time=0 ms ----babylon5.dec.com PING Statistics---- 2 packets transmitted, 2 packets received, 0% packet loss round-trip (ms) min/avg/max = 0/0/0 ms
EAN: 2147483647
Pages: 273