11.7 Removing a Member from the Cluster

To end the chapter, we will discuss removing a member from the cluster. In order to avoid any quorum problems, you should remove only one cluster member at a time and adjust the votes on the quorum disk as appropriate to maintain quorum for the cluster. To remove a member from a cluster, we use the clu_delete_member(8) command. Let's see what the clu_delete_member command actually does before we see an example of its execution.

11.7.1 The clu_delete_member Command

The clu_delete_member command does the following:

It mounts the soon-to-be deleted member's boot_partition and deletes all files in the file system. From this point on, this cluster member can no longer boot from this disk.

Warning

If the "clu_delete_member –f" command is used, it will delete a member even when that member's boot disk is inaccessible. This lets you delete a member whose boot disk has failed or is inaccessible for any reason.

If the member-to-be-deleted has votes, and if its boot disk is accessible, it adjusts the value of cluster_expected_votes throughout the cluster.

Warning

If the clu_delete_member command cannot mount the boot partition, it is unable to determine the number of votes held by the member-to-be-deleted. The value of cluster_expected_votes will therefore not be reset throughout the cluster. If you encounter this scenario, after the clu_delete_member command completes, use the clu_quorum(8) command to check the value of cluster_expected_votes.^[5] Most likely an adjustment of the value will be necessary.

It removes all member-specific directories and files from the cluster-common file systems (cluster_root, cluster_usr, and cluster_var) for the member-to-be-deleted.

IMPORTANT

As the System Administrator can create member-specific files in other directories, such files must be manually removed after the completion of clu_delete_member command.

It removes the soon-to-be deleted member's host name for its cluster interconnect interface from the /.rhosts and /etc/hosts.equiv files.
It writes a log file, clu_delete_member.log, in the/cluster/admin directory.

The clu_delete_member command can be run either interactively or with a simple command line argument. For example, to delete member3 from a cluster, use the following command:

 # /usr/sbin/clu_delete_member -m 3

11.7.2 Steps to Deleting a Cluster Member

Now that we know what the clu_delete_member command actually does, let's go over the actual steps needed to delete a cluster member.

First, we need to determine whether the member-to-be-deleted is a critical voting member of the cluster. This is an essential step since we do not want to impact the cluster's operation by losing quorum.

Before halting the member-to-be-deleted, we use the clu_quorum(8) command to determine whether it is safe to do so.^[6]
Shut down the cluster member-to-be-deleted.
Make sure that all remaining cluster members are up and operational. Use the "clu_get_info –full" command for verification.
Now comes the actual deletion of a cluster member. On another cluster member, run the clu_delete_member command to remove the halted member from the cluster. The clu_delete_member command can be run either interactively or with command line arguments
Prior to the deletion of the member, if the cluster had an even number of members and a configured quorum disk, then it is important to delete the quorum disk or change the quorum disk's votes from 1 to 0.

 # /usr/sbin/clu_quorum -d remove  Collecting quorum data for Member(s): 1 2  Quorum disk successfully removed.

 # /usr/sbin/clu_quorum –d adjust 0 Collecting quorum data for Member(s): 1 2 Quorum disk votes successfully adjusted.

If, on the other hand, deleting the member reduces the number of members to an even number, then either the quorum disk should be added and/or the quorum disk's votes should be changed from 0 to 1.

 # /usr/sbin/clu_quorum -d add dsk5 1  Collecting quorum data for Member(s): 1 2    Initializing cnx partition on quorum disk : dsk5h  Quorum disk votes successfully adjusted.

 # /usr/sbin/clu_quorum –d adjust 1 Collecting quorum data for Member(s): 1 2 Quorum disk votes successfully adjusted.

Warning

Never add, delete, or modify a quorum disk while either the clu_delete_member or the clu_add_member commands are in process.

11.7.3 Execution of the clu_delete_member Command

In our example, let's execute the clu_delete_member interactively. Like the other commands needed to create a cluster or to add a new cluster member, the clu_delete_member command reveals exactly what it will do and then asks the user if he wants to continue.

 # /usr/sbin/clu_delete_member This is the TruCluster Delete Member Program You will need the following information in order to delete a member from the cluster:     - Member ID (1-63) The program will prompt for this information, offering a default value when one is available. To accept the default value, press Return If you need help responding to a prompt, either type the word 'help' or type a question mark (?) at the prompt. The program does not begin to delete the member until you answer all the prompts, and confirm that the answers are correct. Deleting a member involves the following steps:     Removing the files from the member boot disk. (If accessible)     Removing member specific areas from the /, /usr, and /var file systems.     Removing the deleted members entries in shared configuration files. Do you want to continue deleting a member from this cluster? [yes]:yes

Next, the clu_delete_member command prompts the cluster member to be deleted. If this cluster member is not halted and ready to be deleted, do not proceed and please review section 11.7.2.

In our example, we are deleting cluster member3.

 A member ID is used to identify each member in a cluster. Each member must have a unique member ID, which is an integer in the range 1-63, inclusive. Enter a cluster member ID []:3 Checking cluster member ID: 3 You entered '3' as the member ID. Is this correct? [yes]:yes You entered the following information:     Member's ID: 3 If any of this information is incorrect, answer 'n' to the following prompt. You can then enter the correct information. Do you want to continue to delete a cluster member?:[no]yes

Finally, we are presented with messaging of what the clu_delete_member command is doing as it occurs.

 Deleting member disk boot partition files   Member disk boot partition files deleted Initial cluster deletion successful, member '3' can no longer join the cluster. Deletion continuing with cleanup cluster_expected_votes: reconfigured Removing deleted member entries from shared configuration files   Removing cluster interconnect interface 'tester-ics0' from /.rhosts ... Deleting Member Specific Directories   Deleting: /cluster/members/member3/   Deleting: /usr/cluster/members/member3/   Deleting: /var/cluster/members/member3/ clu_delete_member: The deletion of cluster member '3' completed successfully.

After the clu_delete_member command has completed, we must adjust the number of votes for the quorum disk. Since we have a cluster with an even number of members, we change the number of votes on the quorum disk from 0 to 1.

 #/usr/sbin/clu_quorum -d adjust 1 Collecting quorum data for Member(s): 1 2 CNX MGR: Adjust quorum disk votes operation completed with quorum. Quorum disk votes successfully adjusted.

That's really about it when it comes to deleting a member from a cluster, which, as promised, is a straightforward and simple process.

^[5]For more detailed information on quorum, please see Chapter 17 on the Connection Manager.

^[6]For more information on adjusting votes for quorum, please see Chapter 17 on Connection Manager.