|< Day Day Up >|
A GPFS cluster requires an underlying RSCT peer domain. In this section, we describe the commands we used to create an RSCT peer domain on our experimental setup.
There are currently two types of domains in RSCT:
Note that we do not cover every aspect of peer domains here. For a complete description of RSCT, refer to the RSCT documentation:
Chapter 2 of the Administration Guide contains information about creating and administering peer domains. Section 4.3 in the IBM Redbook A Practical Guide for Resource Monitoring and Control (RMC) , SG24-6615, contains an example of a peer domain setup.
Our experimental setup for GPFS, and the underlying RSCT peer domain, consists of two nodes connected by a 10/100 Mbit Ethernet and a Gigabit Ethernet. One node will be primarily for control messages, and the second node will be used for GPFS communication.
Table 6-1 summarizes the names and interfaces of our nodes.
Table 6-1. Network interfaces and corresponding names
6.2.1 RSCT requisites
In our testing, we used RSCT version 2.3.2-1. These filesets must be installed on the nodes prior to defining the peer domains:
6.2.2 Overview of peer domain operations
RSCT provides commands for adding, removing, starting, and stopping nodes inside a peer domain and commands to create, start, stop, and remove peer domains. Commands acting on nodes end in rpnode . Commands for domains end in rpdomain .
6.2.3 Creating a peer domain
In case the GPFS nodes are part of a CSM cluster, you must tell RSCT that the commands need to operate on a peer domain. To do this, issue the command shown in Example 6-1.
Example 6-1. Setting the RSCT management scope
lpar1:~ # export CT_MANAGEMENT_SCOPE=2
Prepare the nodes
Before being incorporated in a peer domain, the security environment has to be prepared for all the nodes. During this, the nodes will exchange public keys. This is done with the preprpnode command, as shown in Example 6-2.
As an argument to the preprpnode command, a list of all the nodes from which this node can accept RSCT messages from. In a GPFS cluster, this command needs to be run on all the participating nodes.
Example 6-2. preprpnode command
r01n33:~ # preprpnode -V r01n33 r01n34 Beginning to prepare nodes to add to the peer domain. Completed preparing nodes to add to the peer domain.
Create the peer domain
Once the nodes are prepared we can proceed to creating the domain. This is done with the mkrpdomain command. A peer domain is characterized by a name , the names of the constitutive nodes, and the UDP ports that Topology Services and Group Services will use. The command shown in Example 6-3 creates the peer domain named itso . We run this command once on one of the nodes.
Example 6-3. Create a peer domain
r01n33:~ # mkrpdomain -V itso r01n33 r01n34 Making the peer domain "itso". Completed making the peer domain "itso".
The command lsrpdomain shows the status of the peer domain we have just created. Example 6-4 shows as "Offline" because we have not yet started it.
Example 6-4. Check the status of the peer domain
r01n33:~ # lsrpdomain Name OpState RSCTActiveVersion MixedVersions TSPort GSPort itso Offline 126.96.36.199 No 12347 12348
6.2.4 Starting and stopping a peer domain
To bring the peer domain online, we use the startrpdomain command as shown in Example 6-5 on page 288. This command can be run on any node in the domain, but it must be issued on only one node.
Immediately after, we check the domain status with lsrpdomain . It can take a while, up to one minute, for the domain to come up. The status should change from "Pending online" to "Online".
Example 6-5. Start the peer domain and watch it come online
r01n33:~ # startrpdomain itso r01n33:~ # lsrpdomain Name OpState RSCTActiveVersion MixedVersions TSPort GSPort itso Pending online 188.8.131.52 No 12347 12348 After a while ... r01n33:~ # lsrpdomain Name OpState RSCTActiveVersion MixedVersions TSPort GSPort itso Online 184.108.40.206 No 12347 12348
Upon startup of the peer domain, the Topology Services and Group Services subsystems will be started on the nodes. To check the proper operation of these services, use the lssrc command as shown in Example 6-6.
The lssrc command is part of System Resource Controller (SRC), which has been ported from AIX to Linux. It is a way to control subsystems. Subsystems must register to SRC in order to be controlled.
Example 6-6. Use lssrc to query Topology/Group Services
r01n33:~ # lssrc -a Subsystem Group PID Status IBM.ConfigRM rsct_rm 781 active ctcas rsct 4376 active cthats cthats 10839 active cthags cthags 10843 active ctrmc rsct 10933 active IBM.ERRM rsct_rm 10947 active IBM.AuditRM rsct_rm 10968 active IBM.HostRM rsct_rm inoperative
The status of individual nodes can be displayed with the lsrpnode command, as shown in Example 6-7.
Example 6-7. lsrpnode to check each node
r01n33:~ # lsrpnode Name OpState RSCTVersion gr01n34 Online 220.127.116.11 gr01n33 Online 18.104.22.168
To stop a peer domain, the nodes can be stopped individually with the stoprpnode command as shown in Example 6-8. We use the lsrpnode to verify that the node was stopped correctly.
Example 6-8. Stopping a node in the peer domain
r01n33:~ # stoprpnode gr01n34 r01n33:~ # lsrpnode Name OpState RSCTVersion gr01n34 Offline 22.214.171.124 gr01n33 Online 126.96.36.199
We can use the stoprpdomain command to stop all nodes at once. This is shown in Example 6-9.
Example 6-9. Stopping the whole peer domain
r01n33:~ # lsrpnode Name OpState RSCTVersion gr01n34 Online 188.8.131.52 gr01n33 Online 184.108.40.206 r01n33:~ # stoprpdomain itso r01n33:~ # lsrpnode lsrpnode: There are no nodes in the peer domain or an online peer domain does not exist.
6.2.5 Removing a peer domain
To remove the peer domain, you must bring it online first and then use the rmrpdomain command; an example is shown in Example 6-10.
For some unknown reason, the peer domain may survive. To remove it permanently, use the -f (force option).
Example 6-10. Removing a peer domain
r01n33:~ # rmrpdomain itso rmrpdomain: There are no nodes in the peer domain or an online peer domain does not exist. r01n33:~ # rmrpdomain -f itso
|< Day Day Up >|