Secondary IP Addresses and IP Aliases
Secondary IP addresses and IP aliases are two different methods for adding multiple IP addresses to the same physical network card. The first IP address (also called the primary address) is added to the NIC at boot time. Additional IP addresses are then added by Heartbeat based on entries in the haresources configuration file (we'll discuss this file in detail in Chapter 8)—additional IP addresses are either IP aliases or secondary IP addresses. As of this writing, the Linux kernel supports both IP aliases and secondary IP addresses, though IP aliases are deprecated in favor of secondary IP addresses.
| Note | IP aliasing (sometimes called network interface aliasing) is a standard feature of the Linux kernel when standard IPv4 networking is configured in the kernel. Older versions of the kernel required you to configure IP alias support as an option in the kernel. |
 |
Whether you use secondary IP addresses or IP aliases, the source address used in packets that originate from the server (connections created from the server) will use the primary IP address on the NIC. You can force the kernel to make the source address appear to be one of the secondary IP addresses or IP aliases associated with the NIC by using the ip command and the undocumented src option. For example:
#ip route add default via 209.100.100.254 src 209.100.100.3 dev eth0
In this example, 209.100.100.254 is the gateway, and 209.100.100.3 is the secondary IP address (or IP alias) you want to use as a source address for outgoing packets. This method is used by the ipsrcaddr resource script included with the Heartbeat package. (You can also use the command ip route get 209.100.100.10 to see which interface and source address is returned by the kernel's routing table for a particular IP address.)
| Note | Forcing the kernel to use a secondary IP address or IP alias for the return address in the packet header does not work if both sides of an active/active Heartbeat pair of servers use the ipsrcaddr resource, or if different resources need to use different source addresses. |
|
By using secondary IP addresses or IP aliases you can offer a service such as sendmail on one IP address and offer another service, like HTTP, on another IP address, even though these two IP addresses are really owned by the same computer (one physical network card at one MAC address).
When you use Heartbeat to offer services on a secondary IP address (or IP alias) the service is owned by the server (meaning the server is the active, or primary, node) and the server also owns the IP addresses used to access the service. The backup node must not be using this secondary IP address (or IP alias). When the primary node fails, and the service should be offered by the backup server, the backup server will not only need to start the service or daemon, it will also need to add the proper secondary IP address or IP alias to one of its network cards.
A diagram of this two-node Heartbeat cluster configuration using an Ethernet network to carry the heartbeat packets is shown in Figure 6-2.
Figure 6-2: A basic Heartbeat configuration
In Figure 6-2 the IP address 209.100.100.2 is the primary IP address of the primary server, and it never needs to move to the backup server. The backup server's primary IP address is 209.100.100.5, and this IP address will likewise never need to move to another network card. However, the IP addresses 209.100.100.3 and 209.100.100.4 are each associated with a particular service running on the primary server. If the primary server goes down, these IP addresses need to move to the backup server, as shown in Figure 6-3.
Figure 6-3: The same basic Heartbeat configuration after failure of the primary server
Ethernet NIC Device Names
The Linux kernel assigns the physical Ethernet interfaces names such as eth0, eth1, eth2, and so forth. These names are assigned either at boot time or when the Ethernet driver for the interface is loaded (if you are using a modular kernel), and they are based on the configuration built during the system installation in the file /etc/modules.conf (or /etc/conf.modules on older versions of Red Hat Linux). You can use the following command to see the NIC driver, interrupt (IRQ) address, and I/O (base) address assigned to each PCI network interface card (assuming it is a PCI card) that was recognized during the boot process:
#lspci -v | less
You can then check this information against the interface configuration using the ifconfig command:
#ifconfig -a | less
This should help you determine which eth number is assigned to a particular physical network card by the kernel eth numbering scheme. (See Appendix C for more information about NICs.)
Secondary IP Address Names
Secondary IP addresses are added after a primary IP address has already been configured for a NIC. The primary IP address can be assigned to a NIC using the ifconfig command (this is normally how Linux distributions assign IP addresses at boot time) or by using the ip command. Secondary IP addresses, however, can only be added using the ip command. When Heartbeat needs to add a secondary IP address to a NIC it uses the script IPaddr2 (included with the Heartbeat distribution) to run the proper ip command.
Both the primary and the secondary IP addresses can be viewed with this command:
#ip addr sh
| Note | Secondary IP addresses are not shown by the ifconfig command. |
Creating and Deleting Secondary IP Addresses with the ip Command
Creating and deleting secondary IP addresses in Linux is easy. To create (add) a secondary IP address for the eth0 NIC, use this command:
#ip addr add 209.100.100.3/24 broadcast 209.100.100.255 dev eth0
In this example, we are assuming that the eth0 NIC already has an IP address on the 209.100.100.0 network, and we are adding 209.100.100.3 as an additional (secondary) IP address associated with this same NIC. To view the IP addresses configured for the eth0 NIC with the previous command, enter this command:
#ip addr sh dev eth0
To remove (delete) this secondary IP address, enter this command:
#ip addr del 209.100.100.3/24 broadcast 209.100.100.255 dev eth0
| Note | The ip command is provided as part of the IProute2 package. (The RPM package name is iproute.) |
Fortunately, you won't need to enter these commands to configure your secondary IP addresses—Heartbeat does this for you automatically when it starts up and at failover time (as needed) using the IPaddr2 script.
IP Aliases
As previously mentioned, you only need to use one method for assigning IP addresses under Heartbeat's control: secondary IP addresses or IP aliases. If you are new to Linux, you should use secondary IP addresses as described in the previous sections and skip this discussion of IP aliases.
You add IP aliases to a physical Ethernet interface (that already has an IP address associated with it) by running the ifconfig command. The alias is specified by adding a colon and a number to the interface name. The first IP alias associated with the eth0 interface is called eth0:0, the second is called eth0:1, and so forth. Heartbeat uses the IPaddr script to create IP aliases.
Creating and Deleting IP Aliases with the ifconfig Command
In our previous example, we were using IP address 209.100.100.2 with a network mask of 255.255.255.0 on the eth0 Ethernet interface. To manually add IP alias 209.100.100.3 to the same interface, use this command:
#ifconfig eth0:0 209.100.100.3 netmask 255.255.255.0 up
You can then view the list of IP addresses and IP aliases by typing the following:
#ifconfig -a
or simply
#ifconfig
This command will produce a listing that looks like the following:
eth0 Link encap:Ethernet su HWaddr 00:99:5F:0E:99:AB inet addr:209.100.100.2 Bcast:209.100.100.255 Mask:255.255.255.0 UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:976 errors:0 dropped:0 overruns:0 frame:0 TX packets:730 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 Interrupt:11 Base address:0x1400 eth0:0 Link encap:Ethernet HWaddr 00:99:5F:0E:99:AB inet addr:209.100.100.3 Bcast:209.100.100.255 Mask:255.255.255.0 UP BROADCAST RUNNING MTU:1500 Metric:1 Interrupt:11 Base address:0x1400
From this report you can see that the MAC addresses (called HWaddr in this report) for eth0 and eth0:0 are the same. (The interrupt and base addresses also show that these IP addresses are associated with the same physical network card.) Client computers locally connected to the computer can now use an ARP broadcast to ask "Who owns IP address 209.100.100.3?" and the primary server will respond with "I own IP 209.100.100.3 on MAC address 00:99:5F:0E:99:AB."
| Note | The ifconfig command does not display the secondary IP addresses added with the ip command. |
IP aliases can be removed with this command:
#ifconfig eth0:0 down
This command should not affect the primary IP address associated with eth0 or any additional IP aliases (they should remain up).
| Note | Use the preceding command to see whether your kernel can properly support IP aliases. If this command causes the primary IP address associated with this network card to stop working, you need to upgrade to a newer version of the Linux kernel. Also note that this command will not work properly if you attempt to use IP aliases on a different subnet.[10] |
Offering Services
Once you are sure a secondary IP address or IP alias can be added to and removed from a network card on your Linux server without affecting the primary IP address, you are ready to tell Heartbeat which services it should offer, and which secondary IP address or IP alias it should use to offer the services.
| Note | Secondary IP addresses and IP aliases used by highly available services should always be controlled by Heartbeat (in the /etc/ha.d/haresouces file as described later in this chapter and in the next two chapters). Never use your operating system's ability to add IP aliases as part of the normal boot process (or a script that runs automatically at boot time) on a Heartbeat server. If you do, your server will incorrectly claim ownership of an IP address when it boots. The backup node should always be able to take control of a resource along with its IP address and then reset the power to the primary node without worrying that the primary node will try to use the secondary IP address as part of its normal boot procedure. |
Gratuitous ARP (GARP) Broadcasts
As mentioned previously, client computers normally use the Address Resolution Protocol (ARP) to figure out which hardware address owns a particular IP address, and then they store this address in an ARP table. The Heartbeat program uses a little trick, called Gratuitous ARP (GARP) broadcasts, to forcibly update these client computer ARP tables with a new hardware (MAC) addresses when the primary server fails, effectively convincing the client computers to talk to the backup server.[11]
GARP broadcasts[12] are just sneaky ARP broadcasts (broadcasts, remember, are only seen by locally connected nodes). The GARP broadcast asks every node connected to the network, "Who owns this IP address?" when, in fact, the ARP request packet header has a source (or reply) IP address equal to the requested IP address. This forces all nodes connected to the network to update their ARP tables with the new source address.
| Note | As of Heartbeat version 0.4.9.1 the send_arp program included with Heartbeat uses both ARP request and ARP reply packets when sending GARPs (send_arp version 1.6). If you experience problems with IP address failover on older versions of Heartbeat, try upgrading to the latest version of Heartbeat. |
Heartbeat uses the /usr/lib/heartbeat/send_arp program (formerly /etc/ha.d/resource.d/send_arp) to send these specially crafted GARP broadcasts. You can use this same program to build a script that will send GARPs. The following is an example script (called iptakeover) that uses send_arp to do this.
#!/bin/bash # # iptakeover script # # Simple script to take over an IP address. # # Usage is "iptakeover {start|stop|status}" # # SENDARP is the program included with the Heartbeat program that # sends out an ARP request. Send_arp usage is: # # SENDARP="/usr/lib/heartbeat/send_arp" # # REALIP is the IP address for this NIC on your LAN. # REALIP="299.100.100.2" # # ROUTERIP is the IP address for your router. # ROUTER_IP="299.100.100.1" # # SECONDARYIP is the first IP alias for a service/resource. # SECONDARYIP1="299.100.100.3" # or #IPALIAS1="299.100.100.3" # # NETMASK is the netmask of this card. # NETMASK="24" # or # NETMASK="255.255.255.0" # BROADCAST="299.100.100.0" # # MACADDR is the hardware address for the NIC card. # (You'll find it using the command "/sbin/ifconfig") # MACADDR="091230910990" case $1 in start) # Make sure our primary IP is up /sbin/ifconfig eth0 $REALIP up # Associate the virtual IP address with this NIC /sbin/ip addr add $SECONDARYIP1/$NETMASK broadcast $BROADCAST dev eth0 # Or, to create an IP alias instead of secondary IP address, use the command: # /sbin/ifconfig eth0:0 $IPALIAS1 netmask $NETMASK up # Create a new default route directly to the router /sbin/route add default gw $ROUTER_IP # Now send out 5 Gratuitous ARP broadcasts (ffffffffffff) # at two second intervals to tell the local computers to update # their ARP tables. $SENDARP -i 2000 -r 5 eth0 $SECONDARYIP1 $MACADDR $SECONDARYIP1 ffffffffffff ;; stop) # Take down the secondary IP address for the service/resource. /sbin/ip addr del $SECONDARYIP1/$NETMASK broadcast $BROADCAST dev eth0 # or /sbin/ifconfig eth0:0 down ;; status) # We check to see if we own the IPALIAS. OWN_ALIAS=`ifconfig | grep $SECONDARYIP1` if [ "$OWN_ALIAS" != "" ]; then echo "OK" else echo "DOWN" fi ;; # End of the case statement. esac | Note | You do not need to use this script. It is included here (and on the CD-ROM) to demonstrate exactly how Heartbeat performs GARPs. |
The important line in the above code listing is marked in boldface.
This command runs /usr/lib/heartbeat/send_arp and sends an ARP broadcast (to hexadecimal IP address ffffffffffff) with a source and destination address equal to the secondary IP address to be added to the backup server.
You can use this script to find out whether your network equipment supports IP address failover using GARP broadcasts. With the script installed on two Linux computers (and with the MAC address in the script changed to the appropriate address for each computer), you can move an IP address back and forth between the systems to find out whether Heartbeat will be able to do the same thing when it needs to failover a resource.
| Note | When using Cisco equipment, you should be able to log on to the router or switch and enter the command show arp to watch the MAC address change as the IP address moves between the two computers. Most routers have similar capabilities. |
[10]The "secondary" flag will not be set for the IP alias if it is on a different subnet. See the output of the command ip addr to find out whether this flag is set. (It should be set for Heartbeat to work properly.)
[11]To use the Heartbeat IP failover mechanism with minimal downtime and minimal disruption of the highly available services you should set the ARP cache timeout values on your switches and routers to the shortest time possible. This will increase your ARP broadcast traffic, however, so you will have to find the best trade-off between ARP timeout and increased network traffic for your situation. (Gratuitous ARP broadcasts should update the ARP table entries even before they expire, but if they are missed by your network devices for some reason, it is best to have regular ARP cache refreshes.)
[12]Gratuitous ARPs are briefly described in RFC 2002, "IP Mobility Support." Also see RFC 826, "Ethernet Address Resolution Protocol."
EAN: 2147483647
Pages: 219