10.1 Before the Cluster Installation

If you've already skipped ahead to this chapter without reading Chapters 3, 4, and 5, we recommend that you stop and carefully review these three chapters at a minimum before continuing. Generally speaking, planning is essential to success.

For the best understanding of what it takes to design and plan a cluster, we suggest that you review Chapters 2 through 9, in addition to using the Cluster Preparation Worksheet and Checklist from section 4.8, to help design and plan your cluster implementation. Planning without a solid foundation of knowledge and understanding usually produces wasted time and energy.

Before we actually start the installation of the TruCluster Server software, we should explore some things on the standalone Tru64 UNIX installation. As we have stated in earlier chapters, Tru64 UNIX is the core foundation for any TruCluster Server cluster.

10.1.1 A Quick View of the Tru64 UNIX File Systems

Let's take a quick look at a Tru64 UNIX system before it is clustered and examine what file systems are mounted using the mount(8) command:

 # mount root_domain#root on / type advfs (rw) /proc on /proc type procfs (rw) usr_domain#usr on /usr type advfs (rw) var_domain#var on /var type advfs (rw)

If we examine only the system level file systems, we see pretty much what is expected for a Tru64 UNIX system – root (/), /usr, /var, and /proc. In this case, all file systems, except /proc, are AdvFS file systems. Nothing unusual, right?

Now let's expand on what we have seen by looking at where the file system domains for these system level AdvFS file systems reside.

 # ls /etc/fdmns .advfslock_fdmns          .advfslock_var_domain      var_domain/ .advfslock_root_domain    root_domain/ .advfslock_usr_domain     usr_domain/

Again, this is what is expected, but let's use this as a baseline for comparison later -- after the creation of the cluster.

10.1.2 Disks for the Cluster

Before we start the creation of the cluster, let's briefly review what disks and partitions are required to build the cluster-common file systems, the member-specific file systems, and the quorum disk.^[1] We'll also discuss some specifics about partition sizes to support these file systems and disk(s).

The following details what disks are required to build a single-node cluster:

One or more disks on the shared bus to hold the clusterwide AdvFS file systems: cluster_root (/), cluster_usr (/usr), and cluster_var (/var).
One boot disk per cluster member on the shared bus.
Depending on the number of members in the cluster, one entire disk on the shared bus to act as the quorum disk.

This list does not include the original system disk for Tru64 UNIX.

What needs a disk and what needs a partition? During the initial creation of the cluster, it is expected that physical disk partitions will be needed for the following cluster-common file systems:

cluster_root
cluster_usr
cluster_var

While the member boot disk will also use partitions for the creation of its two file systems and cluster status partition, the clu_create(8) program will only ask for the entire disk and then use the "a", "b", and "h" partitions. As for the quorum disk, while an entire disk is asked for and used, only the "h" partition has a cnx label on it.

As an aside, if you want to use LSM to mirror file systems on the cluster, you can set up LSM on the original Tru64 UNIX system but hold off on the mirroring of any file systems until after the first member of the cluster is booted. During the initial creation of the cluster using the clu_create command, LSM volumes may not be used.

After you have created a single-member cluster, you can use the volmigrate(8)command to move the cluster_root domain to an LSM volume and then mirror that volume. Please be aware that using the volmigrate command to move cluster_root actually moves the data from the disk partition you specified in the initial cluster creation command, clu_create, to new target storage that you must specify when you run volmigrate. For more information on using LSM within a TruCluster Server configuration, please refer to Chapter 14.

Information from Table 10-1, taken along with information from Table 4-5, provides a basis for determining the disks and the disk partitions that should be used in our Tru64 UNIX/TruCluster Server Planning Worksheet and Checklist from section 4.8. We will use this completed Worksheet as the blueprint to build our cluster.

Table 10-1: Cluster Partition Sizes
File System (Type)	Partition	HP Recommended Minimum Partition Sizes	Comments from HP on Partition Sizes	TruCluster Server Handbook Recommendations on Partition Sizes
Cluster Root(/) AdvFS	a	200 MB	The minimum partition size requirement is the larger of 125 MB or (1.125 x currently_used root size).	512 MB
Cluster /usr (AdvFS)	g	1000 MB	The minimum partition size requirement is the larger of 675 MB or (1.125 x currently_used/usr size). The absolute minimum partition size is 675 MB.	4. GB
Cluster /var (AdvFS)	h	1000 MB	The minimum partition size requirement is the larger of 360 MB or (1.125 x currently_used /var size). The absolute minimum partition size is 360 MB.	4.0 GB
Member boot disk: root (AdvFS)	a	256 MB	The only required file systems on member boot disks are root, swap, and a 1 MB partition used for cluster status.	256 MB
Member boot disk: swap (swap)	b	Depends on system.	The remainder of the member boot disk after the "a" and "h" partitions are allocated. The minimum size requirement is 256 MB.	Depends on the system. See section 4.2.2 for the discussion on swap in a cluster.
Member boot disk: CNX cluster status	h	Exactly 1 MB	The "h" partition (fstype cnx) stores cluster state information. The installation procedure creates this partition on each member's book disk.	Exactly 1 MB
Quorum disk	h	Exactly 1 MB	Use a small disk as the quorum disk. As all cluster members must be able to retrieve the information from the quorum disk, no I/O barriers are placed on this disk. Therefore, no file systems should be on this disk.	Exactly 1 MB

The "TruCluster Server Handbook Recommendations on Partition Sizes" column, from Table 10-1, contains average sizes taken from clusters in production at several sites so this is a more realistic recommendation. The term "currently_used" refers to the currently used disk space in the partitions and not the current partition sizes for the Tru64 UNIX system disk.

When a member is initially created or added to a cluster, the default disk partition sizes are used for the partitions – not the existing disk partitions. While the sizes of the partitions for boot_partition and for the "cnx" partition are appropriate (see Table 10-1), we expect that the partition used for swap will most likely be larger than the default partition. How can we create and use a partition of the appropriate size for swap when we do not have control over what the clu_create program does to the disk partitions for the member disk? Basically, we work "around" the clu_create program.

According to the TruCluster Server Installation Manual, "For disks that contain file systems, the installation procedure calculates whether the sizes are acceptable. If the required partitions are usable, the installation procedure prompts whether you want to use the existing partition sizes." Using this information, we can write a disklabel with the appropriate partition size information and a fstype of "AdvFS" for partition "a", a fstype of "swap" for partition "b", and a fstype of "cnx" for partition "h" to the disk that will be used as the member's boot disk. When we use this disk as the member's boot disk, the clu_create command will re-use the existing partition sizes because it sees the partitions are in-use.

How can we partition the member's boot disk setting the appropriate partitions in-use? The following provides an example on how this can be done. Before going any further, we partition the disk that will be used for the first member's boot disk.

Write a default label to the disk.
```
 # disklabel - rw dsk1 
```
Edit and write out the disklabel of the disk providing the appropriate partition sizes.
```
 # disklabel -re dsk1 
```
Note, if you're GUI inclined, you can also use the diskconfig(8) command.
Change the fstype for the partitions that we are interested in from "unused" to the appropriate in-use type.
```
 # disklabel -sF dsk1a AdvFS # disklabel -sF dsk1b swap # disklabel -sF dsk1h cnx 
```
Note, as of this writing, the diskconfig command does not support setting a "cnx" fstype.

Since you will likely have more than a one-member cluster and the members' boot disks will probably be identical, you can quickly partition the additional member boot disks by saving the first member's disklabel to a file and then writing it to the other member boot disks. For example, we plan to use dsk2 as our second member's boot disk.

Save the disklabel.
```
 # disklabel -r dsk1 > /tmp/bd.lbl 
```
Write the disklabel.
```
 # disklabel -Rr -t advfs dsk2 /tmp/bd.lbl 
```

10.1.3 About the Quorum Disk

One of the Connection Manager's (CNX) responsibilities is to ensure data integrity within the cluster in the event of a communications failure. It does this through a voting mechanism in which processing and I/O are allowed to occur but only when there is a majority of votes present or a quorum. The quorum disk can add to that voting mechanism; therefore, it can potentially add to the high availability of a cluster.

Although the quorum disk is not very large, a minimum of 1 MB, we want to emphasize that the entire disk is used. It should also be noted that the quorum disk does not require a disklabel and that there will be no file systems occupying any of its partitions. The clu_create program will write a 1 MB label of fstype: "cnx" to the "h" partition of the quorum disk.

When should you use a quorum disk? If you have a two-node cluster, a quorum disk is highly recommended. If your cluster is larger than two nodes, then it depends on how many nodes are in your cluster, although you should not configure a quorum disk in a cluster with an odd number of members. For more information on the Connection Manager and quorum within a cluster, please see Chapter 17.

During the initial creation of the cluster using the clu_create(8) command, you will be asked to provide a quorum disk. If your cluster will contain an even number of members and you plan to use a quorum disk, you can certainly provide the quorum disk when prompted. If you plan to operate the cluster as a single-member cluster for a while to "get the feel of the cluster," then we recommend that you refrain from creating the quorum disk during the initial creation of the cluster. After the second member is added to the cluster, the quorum disk can easily be added using the following command:

clu_quorum -d add <disk> <# of votes>.

If you find that you do not require a quorum disk and yet you plan to increase the number of members in your cluster at a much later date, then it would make sense to set aside a small disk on the shared bus that could be used as quorum disk. This disk can even be configured as a zero-vote quorum disk. See section 17.8.3 for further information.

10.1.4 Using the Cluster Preparation Worksheet and Checklist

In section 4.8, we created a Tru64 UNIX/TruCluster Server Planning Worksheet and Checklist. Using this Worksheet, you should be able to plan the implementation of your cluster in a very straightforward manner. Please use this as a guide upon which to build.

^[1]This was also specified earlier in Table 4.4.