9.2 Redundant Array of Independent Network Interfaces (NetRAIN)

Tru64 UNIX provides a method of taking multiple network interface cards (NICs) and grouping them together to function as one highly available network, more commonly known as a redundant array of independent network adapters or NetRAIN.

NetRAIN is a virtual interface that works by grouping two or more NICs into an array seen by the operating system as one interface. Each NetRAIN interface uses only one IP address, and there is only one active NIC at any time (for multiple active NICs with one IP address, see "Link Aggregation" in the following section). If the active NIC fails, an idle NIC comes online. The coolness factor in this is that if you happen to be connected to the system via the NetRAIN interface

when a NIC fails, you may experience a brief pause while the new NIC comes online, but you will not be disconnected!

The number of arrays currently supported is nr_maxdev – 1. The nr_maxdev attribute is defined in the netrain subsystem and has a default value of 128.

 # sysconfig -q netrain nr_maxdev netrain: nr_maxdev = 128

For the most up-to-date information on the netrain subsystem for the version of Tru64 UNIX that you are using, see the sys_attrs_netrain (5) reference page.

9.2.1 Network Configuration without NetRAIN

A network configuration without NetRAIN is shown in Figure 9-2. There is no redundancy, so if the network interface fails, the system can no longer communicate on that subnet.

click to expand
Figure 9-2: Traditional Network Interface Configuration

When configuring a network interface without NetRAIN, you (or your favorite network configuration program) modifies the following parameters in /etc/rc.config:

MAX_NETDEVS

The MAX_NETDEVS attribute is the maximum network devices that can be configured. This should be greater than or equal to NUM_NETCONFIG.
```
 # rcmgr get MAX_NETDEVS 2 
```
NUM_NETCONFIG

The NUM_NETCONFIG attribute is the number of network devices to configure. This must be equal to the number of NETDEV_#/IFCONFIG_# pairs configured.
```
 # rcmgr get NUM_NETCONFIG 2 
```
NETDEV_# (where "#" is the instance of the network configured)

Get the interface to configure.
```
 # rcmgr get NETDEV_0 tu0 
```
IFCONFIG_# (where "#" is the instance of the network)

The configuration information to pass to the ifconfig(8) program for the NETDEV_#interface.
```
 # rcmgr get IFCONFIG_0 192.168.0.69 netmask 255.255.255.0 
```

There are several programs that can be used to configure a network interface in Tru64 UNIX: "netconfig", "sysman net_wizard", "sysman interface", "setup", and "checklist". Configuring NetRAIN, however, does not yet have a program to walk you through the task.

9.2.2 Configuring NetRAIN

Caution

Configuring NetRAIN for a LAN Cluster Interconnect is quite a bit different than the method we will be discussing in this section. We will cover configuring a LAN Cluster Interconnect in Chapter 10.

NetRAIN is fairly easy to configure, but there are some rules that need to be followed to ensure success.

All NICs in the array must be physically connected to the same subnet but should have separate paths to the subnet for redundancy. In other words, you probably do not want two NICs connected to the same hub because if the hub breaks, your connection to the subnet would be lost from both NICs.
All NICs in the array must be of the same type (Ethernet, FDDI, ATM, etc.).
All NICs in the array must be the same speed and operational mode (full duplex, half duplex).
All NICs to be configured in the NetRAIN set must be idle (e.g., not currently configured).
You cannot run LAT on a NetRAIN virtual interface (nr) or any of the NICs that are configured within a NetRAIN set.

A network configuration with NetRAIN is shown in Figure 9-3.

click to expand
Figure 9-3: NetRAIN Interface Configuration

9.2.2.1 Delete the In-Use Interfaces (if any)

If one or more of the NICs are currently in use, delete them. The easiest method for doing so is to use the "sysman" command with the "interface" accelerator.

 # sysman interface

However, you can remove the interface using the manual approach as well.

Delete the running interface.

 # ifconfig delete tu0 tu0: delete inet address 192.168.0.69

 # ifconfig delete tu1 tu1: delete inet address 192.206.126.69

Decrease the NUM_NETCONFIG attribute by the number of interfaces that you are going to delete.
```
 # rcmgr get NUM_NETCONFIG 2 
```
```
 # rcmgr set NUM_NETCONFIG 0 
```

Find the NETDEV_# attributes for the interfaces you are removing.

 # grep -E "tu0|tu1" /etc/rc.config NETDEV_0="tu0" NETDEV_1="tu1"

Delete the NETDEV_# and IFCONFIG_# attributes.

 # for i in 'grep -E "tu0|tu1" /etc/rc.config' > do > DELNUM="${i##*_%%=*}" > print "deleting NETDEV_${DELNUM}" > rcmgr delete NETDEV_${DELNUM} > print "deleting IFCONFIG_${DELNUM}" > rcmgr delete IFCONFIG_${DELNUM} > done deleting NETDEV_0 deleting IFCONFIG_0 deleting NETDEV_1 deleting IFCONFIG_1

9.2.2.2 Configure the NetRAIN Interface

If you are configuring the NetRAIN interface on a NETDEV_# other than instance number zero, then increment NRDEV_# and NRCONFIG_# to be the same instance number as NETDEV_# and IFCONFIG_#. This only applies to the attribute itself and not the value. In other words, if the instance number was "1", then the attributes would be NETDEV_1, IFCONFIG_1, NRDEV_1, NRCONFIG_1. The NetRAIN interface would remain nr0.

Define the NetRAIN interface.
```
 # rcmgr set NRDEV_0 nr0 
```
Specify the array of adapters.

Make sure you do not use any space between "tu0", the "," and "tu1".
```
 # rcmgr set NRCONFIG_0 "tu0,tu1" 
```
Define or update the number of NetRAIN devices configured.

This number is analogous to NUM_NETCONFIG for NetRAIN interfaces.
```
 # rcmgr get NR_DEVICES <no output> 
```
```
 # rcmgr set NR_DEVICES 1 
```
Set the NETDEV_# to the NetRAIN interface.
```
 # rcmgr set NETDEV_0 nr0 
```

Set the IFCONFIG_# attribute.

 # rcmgr set IFCONFIG_0 "192.168.0.69 netmask 255.255.255.0 filter"

Increment NUM_NETCONFIG.

 # rcmgr get NUM_NETCONFIG 0

 # rcmgr set NUM_NETCONFIG 1

Reboot the system.
```
 # shutdown –sr now 
```

 # niffconfig -v Interface:   tu1,  description:  NetRAIN  internal, status:     UP, event: ALERT , state: GREEN          t1: 4, dt: 2, t2: 10, time to dead: 2, current_interval: 4, next time: 4 Interface:   nr0,  description:  NetRAIN  internal, status:     UP, event: ALERT , state: GREEN          t1: 4, dt: 2, t2: 10, time to dead: 2, current_interval: 4, next time: 2 Interface:   tu0,  description:  NetRAIN  internal, status:     UP, event: ALERT , state: GREEN          t1: 4, dt: 2, t2: 10, time to dead: 2, current_interval: 4, next time: 4

So which interface is active? Check the network counters using the netstat(1) command.

 # netstat -i | grep -E "tu0|tu1|nr0" nr0   1500   <Link>    2:10:64:30:16:12    1861    0   1734      0      0 nr0   1500   DLI       none                1861    0   1734      0      0 nr0   1500   192.168   sheridan            1861    0   1734      0      0 tu0   1500   <Link>    0:10:64:30:12:31     254    0    317      0      0 tu0   1500   DLI       none                 254    0    317      0      0 tu1   1500   <Link>    2:10:64:30:16:12    1607    0   1417      0      0 tu1   1500   DLI       none                1607    0   1417      0      0

Generate some traffic. You can just ping(8) the interface.

 # ping -f -c 100000 -I nr0 sheridan PING sheridan (192.168.0.69): 56 data bytes . ----sheridan PING Statistics---- 100000 packets transmitted, 100000 packets received, 0% packet loss round-trip (ms) min/avg/max = 0/0/3 ms

Check the network counters again.

 # netstat -i | grep -E "^(tu0|tu1|nr0|Name)" Name   Mtu   Network      Address               Ipkts    Ierrs    Opkts     Oerrs    Coll nr0    1500  <Link>       2:10:64:30:16:12      2014         0    2016          0       0 nr0    1500  DLI          none                  2014         0    2016          0       0 nr0    1500  192.168      sheridan              2014         0    2016          0       0 tu0    1500  <Link>       0:10:64:30:12:31       257         0     320          0       0 tu0    1500  DLI          none                   257         0     320          0       0 tu1    1500  <Link>       2:10:64:30:16:12      1757         0    1696          0       0 tu1    1500  DLI          none                  1757         0    1696          0       0

The tu1 interface is active.

9.2.3 NetRAIN Events

NetRAIN events are defined in /usr/share/evm/templates/sys/hw.net.evt. Below are the registered NetRAIN events:

 # evmwatch -A -i -f "[name *.netrain]" -t "[@priority] @name" [0] sys.unix.hw.net.netrain.down [0] sys.unix.hw.net.netrain.up [0] sys.unix.hw.net.netrain.alert

For additional information regarding NetRAIN, see the Tru64 UNIX Network Administration Guide for version 5.0A and version 5.1, or the Tru64 UNIX Network Administration: Connections Guide for version 5.1A. Also check out the nr(7) and sys_attrs_netrain(5) reference pages.