| In the past, an Extended Serviceguard cluster was known under the heading of a Campus Cluster . The name Campus derived from the relatively short distance offered by Fibre Channel/FDDI technologies at the time. The maximum distances for a Campus Cluster solution were on the order of 10Km. With recent advances in Fibre Channel technologies, i.e., DWDM, the distances we are now talking about are up to 100Km. Hence, the Campus nomenclature was dropped. If you were to look at Extended Serviceguard Cluster, you would be hard pressed to distinguish it from a standard Serviceguard cluster; it is a single cluster, there are no additional software components installed, there are no special configuration files used, and there are no specific templates or README files to study. So what makes an Extended Serviceguard Cluster different? The main difference is that with an Extended Serviceguard Cluster we are aiming to protect from the loss of an entire data center by utilizing at least two complete data center environments. The main reason for implementing such a solution is to guard against the loss of a single data center due to fire, flooding, a power outage , or even terrorism. The distances involved could be anything from a few hundred meters up to 100Km. Extended Serviceguard Clusters follow specific design and architectural rules to ensure that no single component is a Single Point Of Failure. This is a cornerstone of the entire philosophy. I have spent some time ramming this idea home throughout our discussions regarding High Availability. I've talked about everything from having redundancy in your hubs/switches for your heartbeat networks to ensuring that your physical links between sites have a redundant component that is routed via a physically separate infrastructure. All these design rules are incorporated into the design for Extended Serviceguard clusters. Here are the key design rules for an Extended Serviceguard Cluster: 28.1.1 At least two separate data centers The main reason for employing an Extended Serviceguard cluster is to guard against the failure of an entire data center. In this way, an Extended Serviceguard cluster will always be spread over at least two physically separate sites. Using multiple sites is a possibility, but that requires additional design criteria to be taken into account. 28.1.1.1 TWO DATA CENTERS DESIGN LIMITATIONS For a two-site solution, we need to understand the limitations regarding the number of servers on each site. This limitation is in direct response to the cluster requirement for establishing a cluster lock. Let's look at the design limitations: -
Two separate data centers are used. -
Equal numbers of nodes are at each site. -
This is for two-node or four-node clusters only, not for three-node clusters. -
Lock disk is only supported for clusters with up to four nodes. -
A dual cluster-lock disk is required, one in each data center and each on a separate bus. -
Physical (SAN and network) connections between sites must be redundant and routed via separate physical paths. Let's discuss the crux of these design limitations: cluster lock requirements . If we were to lose all connectivity between nodes in the cluster, the cluster will reform. The nodes that form the new cluster will achieve cluster quorum with the rest of the nodes instigating a Transfer of Control (TOC). In order to achieve cluster quorum, we must have 50 percent of the nodes available and we must have obtained the cluster lock. Let's look at an example where we have an asymmetrical cluster design: three nodes in one site and one node in another. If we were to completely lose the site with three nodes, the cluster could not reform, because we cannot achieve cluster quorum . In a similar vein, we can understand the necessity for having two cluster lock disks; if a single cluster lock disk was used, we could be in a position where we cannot reform the cluster because the nodes attempting to form the cluster have lost contact with the cluster lock disk. 28.1.1.2 THREE DATA CENTERS DESIGN LIMITATIONS With a three-site solution, we are providing cluster arbitration capability on the third site. This negates the need for cluster lock disks. On the third site, we can use either a Quorum Server or nodes that are known as arbitrator nodes. Whichever solution for supplying cluster lock capabilities is used, it is strongly advised that at least two nodes on the third site participate in this function to avoid an SPOF in our design. Arbitrator nodes are simply nodes installed with the Serviceguard software that are members of the cluster. They are not connected to the main data storage device and, hence, will never run any application packages. They are simply there to arbitrate which main data center will achieve cluster quorum and reform the new cluster after a failure. In reality, this means that with a catastrophic failure of a data center (or all networking links between sites), the remaining data center will be the only one left and hence the nodes therein are the only nodes able to remain in network contact with the arbitrator nodes. This simple idea means that the remaining data center nodes + the arbitrator nodes will reform the cluster on their own. Here are the design limitations for a three-site solution: -
Not all three data centers need to have the same number of nodes in each. -
When using arbitrator nodes, the major data centers should contain an equal number of nodes in order to facilitate cluster quorum . -
The distribution of nodes must not introduce an SPOF being a single site. -
If using arbitrator nodes, no data center can have  half of the nodes in the cluster because losing that data center will mean the cluster will not be able to reform, e.g., a four-node cluster will require one site to have two nodes. Should this site fail, the entire cluster will fail because we cannot achieve cluster quorum. -
As a result, three-node or > four-node clusters are allowed. -
If using a Quorum Server, we could have two major data centers with the same number of nodes in each because the Quorum Server is not part of the main cluster; it is only supplying cluster lock functionality. Only nodes in the major data centers are connected to storage devices. Cluster lock disks not required (and not supported in a three-site configuration). Network connectivity between the three sites must not introduce a Single Point Of Failure. Ensure that all routes between sites have built-in redundancy and a routed via separate physical paths. Ensure that there are multiple routes to each and every network. The size of the cluster is limited only by the number of nodes allowed in any other Serviceguard cluster; that's currently 16 nodes. 28.1.2 Data replication in an Extended Serviceguard cluster Once we understand the design limitations of the number of data centers we are going to use, we can then discuss the limitations in how we implement data replication between sites. In an Extended Serviceguard Cluster, this is achieved by host-based data mirroring. If using LVM, this may require buying and installing the MirrorDisk/UX product; this product is included with the purchase of the HP-UX 11i Enterprise Operating Environment. If using VxVM, you need to buy a license for the Advance VxVM functionality that includes data mirroring. If we are using advanced RAID arrays such as HP's XP disk arrays, the design rules for an Extended Serviceguard Cluster dictate that no array-based data replication must be used. Using array-based data replication requires us to move to at least a Metrocluster solution. The reason for this distinction is that we do not need to employ sophisticated, and in some cases very expensive, disk arrays in order to participate in an Extended Serviceguard Cluster; using a JBOD is an adequate solution. 28.1.3 Networking in an Extended Serviceguard cluster As with any Serviceguard cluster, we need to ensure that we do not introduce an SPOF with our networking design. At the heart of Serviceguard is the heartbeat between nodes in the cluster. Using redundant switches with nodes having multiple links to multiple switches are all good high availability guidelines. In an Extended Serviceguard, we need to ensure that losing an entire data center does not mean we lose all subnet functionality. When using a multiple-site solution we need to ensure that we have multiple routes to and from all sites. This takes great care and the implementation of sophisticated technologies such as the OSPF routing protocol, which is far more robust at discovering and recovering from failed routes between nodes. We also need to consider the extended distance involved in an Extended Serviceguard cluster. We probably need to employ optical-based link-level technologies such as 100Base-FX, Gigabit Ethernet, and FDDI that can be supported over DWDM links. (FDDI requires two DWDM converters for a single FDDI subnet. For full redundancy, you would need to employ 2x FDDI subnets, which will require 4x DWDM converters.) |
