Chapter 20: Network Administration Tasks

Network administration can be even more of a black art than traditional system administration. Most people tend to lump the two functions under one common heading, "system administration." For our purposes, we will consider them separately. This chapter addresses network administration issues as they pertain to the world of clusters.

Last time we checked, the network administration team just stampeded down the hall to track down a routing/NFS/NIS/hardware/BIND/some_other_arcane_network problem. Imagine the team's reaction when we introduce clustered machines into their already bubbling over cauldron of responsibilities! In the previous chapter on System Administration Tasks, we saw that some commands worked unchanged in a cluster and were common to all members, some worked with slight changes, and others were specific to cluster members. Here too some of the network-oriented services will be common, others will be member-specific, and a few will not be recommended.

20.1 Routing

An article appeared in SysAdmin magazine a few years ago in which the author was discussing routing and subnet masks. Just a few columns were enough to convince us that several misconceptions were being presented. Considering that the article was written by someone who thought he understood the concept of routing well enough to be published in a magazine, it's fair to say that routing is one of the least understood and most error prone aspects of network administration. The TruCluster Server Cluster Administration Guide suggests that the cluster not be configured as a router. This doesn't mean that it can't function as a router; they just don't recommend it. In a way, this is good news for you.

Go out and buy a piece of hardware/software to deal with routing. Why waste the beauty of the cluster on the narrowly focused job of forwarding IP packets from one network to another? Your cluster has much more important processing to do.

20.1.1 Cluster Alias Routing

During the cluster alias discussion in Chapter 16, we mentioned some details on the responsibility of dealing with routing incoming requests for access to the cluster alias. This activity involves routing within the cluster. Indeed the necessity of routing within the cluster is the reason that the engineers suggest that the cluster not be used as a full-fledged IP router. The cluster already has a routing responsibility involving the routing of cluster alias traffic to an appropriate target member in the cluster. Each member in the cluster usually runs the gated dynamic routing daemon and references a special version of the gated.conf file named /etc/gated.conf.membern where "n" is the member number. This file is created and maintained by aliasd and is not to be directly edited by anyone. There is altogether too great a chance that an attempt to alter the gated.conf (4) file for your own purposes will end up preventing the cluster alias subsystem from doing its job. The member-specific gated.conf.membern file will provide the gated daemon the information necessary to establish the default cluster alias route and also request that the gated daemon advertise this route using the Routing Information Protocol (RIP).

20.1.2 Routing Options

Up until Patch Kit 1(PK1) for V5.1A, the gated (8) dynamic routing daemon was the only dynamic routing daemon supported. Several changes were implemented in PK1. Neither the other popular routing daemon, routed (8), nor support for static routes reflected in the /etc/routes file were compatible with the cluster alias subsystem and aliasd (8). The improved aliasd, while still optimized for gated, will work with routed or gated (not both concurrently) running on the member. In fact, aliasd will even run with neither routed nor gated running on the member. This would require the configuration of static routing. Don't lose sight of the fact that the dynamic routing daemons (routed and gated) have a job to do. Strange as it may seem, their job is not to do routing. Their job is to populate the in-memory routing table with correct information about routes so that the act of routing can happen in an intelligent and efficient manner.

20.1.3 Routing Information Protocol and gated

In a cluster, gated has always been implemented using the Routing Information Protocol (RIP). The RIP allows the convenient periodic (every 30 seconds) publicizing of information currently in a member's routing table and the acceptance of routing table information from other dynamic routing daemons if appropriate. The goal of the routing daemon is to keep the routing table information that exists in all network routers and end nodes as accurate as possible. HP suggests in the Release Notes that "configuring a cluster to perform non-standard routing tasks should only be configured by an experienced network administrator." For that reason, we will focus the rest of this chapter on the standard cluster routing configuration using the gated dynamic routing daemon.

20.1.4 Configuration Files for gated

As mentioned earlier, each member's aliasd daemon (part of the cluster alias component discussed in Chapter 16) will create a member-specific gated.conf file (not a CDSL) named /etc/gated.conf.membern (with the "n" replaced by a member number). The following example shows the member-specific gated.conf file. (For more information on the syntax of this file, see the following reference pages: gated.conf (4), gated.control (4), gated.proto (4), gated_intro(4)).

 # cd /etc && file gated* gated.conf:      symbolic link to ../cluster/members/{memb}/etc/gated.conf gated.conf.default:      c program text gated.conf.member1:      c program text gated.conf.member2:      c program text 

 # cat /etc/gated.conf.member1 # This file created automatically by "aliasd", DO NOT EDIT! # traceoptions "/usr/tmp/gated.log" replace size 1000k files 2 general; # any dead interfaces will be entered here with a high (i.e. undesireable) # preference, so that gated will be compelled to put a better route # in the kernel forwarding table # interfaces {     interface all passive; }; # broadcast routes with RIP if at least one interface is healthy # # make a default route through the cluster interconnect look expensive # # If an interface has been marked as "dead" by NIFF then we # will also exclude RIP routes via the affected interface. # This closes a hole where gated doesn't qualify the interfaces # over which it hears broadcasts, and will possibly install a route # through a dead interface. # # Lastly, we place a catch-all "all nripin noripout" here so that # if an interface is brought on-line after aliasd has started, it # won't automatically get used by gated. # rip yes {     broadcast;     interface ics0 ripin ripout metricout 10;     interface tu0 ripin ripout metricout 10;     interface tu1 ripin ripout;     interface all noripin noripout; }; # one "host" or network number per alias # # aliasd will also reroute direct routes for interfaces # marked as "dead" by NIFF by inserting a new gateway # for that route here # 

Note, the next few lines describe the default cluster alias.

 static {     host 192.168.0.70 interface 127.0.0.1; }; # arrange to export alias host/network routes: # # o don't publish routes learned through RIP # o publish the "static" routes given above, except for #   orphaned networks that we're routing around # o publish direct routes for virtual subnets which have real #   interfaces attached to them # # for the cluster interconnect, arrange to broadcast a # high-cost default route over it # # this default may have been learned from RIP (proto rip) # or it may have been statically configured (proto kernel) # we give a slight advantage to a default route via RIP # 

Note, the next few lines describe the Cluster Interconnect (CI) virtual address.

 export proto rip interface 10.0.0.1 {      proto rip {         default;      };      proto kernel {        default metric 3;      }; }; 

Note, the next few lines describe the LAN interconnect.

 export proto rip interface 10.1.0.1 {     proto rip {         default;     };     proto kernel {         default metric 3;     }; }; export proto rip interface 192.168.0.68 {     proto rip restrict; }; 

Pretty ugly, huh? You are probably starting to see why HP recommends not setting up your system as an IP router unless you are very experienced with gated and network administration in general. (See the first line of the gated.conf.membern file: "# This file created automatically by" "aliasd" ", DO NOT EDIT!")

Still want to give it a whirl? Then follow these steps according to the TruCluster Server Cluster Administration Guide in the documentation set (Chapter 6): If gated is running, stop it with the following command:

 # /sbin/init.d/gateway stop 

1. Enter the following command:

    # cluamgr -r start,nogated 

2. Modify gated.conf(to the name that you are using for the configuration file).

   Use the version of /etc/gated.conf.membern that was created by the "cluamgr -r nogated,start" command as the basis for edits to a customized gated configuration file. You will need to correctly merge the cluster alias information from the /etc/gated.conf.membern file into your customized configuration file.

3. Start gated with the following command:

    # /sbin/init.d/gateway start 

In this special case, you will be providing a non-standard configuration file for gated to use (perhaps requesting Open Shortest Path First (OSPF) instead of RIP); therefore, you will need to enter the string "nogated" for CLUA_ROUTE_ARGS in /etc/rc.config.common in order to prevent aliasd from restarting gated using the gated.conf.membern file upon reboot.

Retrieve the CLUA_ROUTE_ARGS from the rc.config.common file. We do this to verify that there is not any previous attribute value (and to merge the contents if necessary).

 # rcmgr –c get CLUA_ROUTE_ARGS 

Add in the new value for the CLUA_ROUTE_ARGS environment variable.

 # rcmgr –c set CLUA_ROUTE_ARGS "nogated" 

Note

Do not place the CLUA_ROUTE_ARGS environment variable in a member-specific rc.config file.