5.1 Planning the installation

 < Day Day Up > 

5.1 Planning the installation

The goal of this section is to provide you with enough information to guide you though the requirements gathering phase that is associated with a typical CSM installation on an IBM BladeCenter Cluster 1350. During this phase you will learn how to build a solid foundation, that will greatly reduce the amount of time that it takes to install and configure hardware and software on your cluster. The checklists in the back of this book will serve as an aid for you, as you work through the requirements gathering phase. This section also defines the prerequisites for a typical CSM based installation in terms of knowledge and environment (that is, Hardware, Software, Network(s), and so on).

5.1.1 Before you begin

You should be familiar with the following:

  • Red Hat Linux 7.3

  • Linux clustering concepts

  • Network Fundamentals

  • xSeries Model 335, 345, 360, and IBM Cluster 1350 hardware, including Advanced System Management Processors


Verify that your applications are able to run on this kind of cluster. If you have any doubt, contact your architect or IBM directly.

5.1.2 Develop a network plan

Determine what network architecture will be used for the various cluster networks: Ethernet, Gigabit Ethernet, or Myrinet.

Make sure you have enough connections and cables for all of the nodes, and their associated interfaces. If you treat each of the items in the following list, like an independent and functional network, then you will build a better foundation for your cluster. Part of demystifying cluster technology is grasping the benefits that are gained by separating processes over each of these networks. Some of the these networks include (the following list is ordered by functional hierarchy):

  • KVM or KVM over IP (Allows for Consolidated Management of the Nodes).

  • Equinox (Serial Network)

  • RSA, or RSA-II, or ASMA (Management Card for the ASM Interconnect Bus)

  • C2T or C2 + RS485 (Referred to as "ASM Interconnect Bus")

  • Ethernet and / or Gigabit Ethernet (Cat5 or Fiber Networks)

  • Myrinet (HBA / Fibre Channel Network)

Define your IP address ranges, gateway, and domain name for each functional network as applicable. Assign the host names and the IP addresses to your nodes and record the mac addresses for each of the adapters. Also, define which interfaces will be used on which network(s).


All nodes must reside on the same subnet. It is also a good idea to place the cluster networks on a series of private / internal networks.

The Internet Assigned Numbers Authority (IANA) has reserved the following three blocks of the IP address space for private internets:

  • - (10/8 prefix)

  • - (172.16/12 prefix)

  • - (192.168/16 prefix)

5.1.3 Develop a hardware resources plan

Verify that your current hardware configuration will adequately support your software and future processing needs, a couple of important requirements should be verified are: the memory, storage, and processor speeds.

Also, consider questions such as:

  • What are the size requirements for your partitions?

  • How many partitions will you need?


    A general reference guide, to assist in answering the first two questions can be found on the following page:108

  • How much data will be generated by your applications?

  • How often? Where will you store your data?

  • How will you protect your backup / make available / protect your data?

5.1.4 Develop a plan to update your hardware

IBM is continuously improving their hardware, and adding new capabilities to their software as well, due to this fact IBM has developed a optimal configuration guide. IBM has called this type of guide a "Best Recipe". Make sure that you are using the latest "Best Recipe" available for your cluster components, the best recipe is a living document and changes as newer technologies are released. The best recipe offers an integrated solution, that IBM has tested and verified will allow for your Cluster to operate at the peak of its capabilities. Here are just a few of the things that are included in a typical best recipe:

  • xSeries BIOS/Diags and ASMP / ISMP firmware revision levels

  • Kernel build level

  • Remote Supervisor Adapter firmware revision levels (RSA)

  • Ethernet PXE BIOS revision levels (OB NIC)

  • C2T firmware revision levels

  • ServeRAID BIOS and firmware revision levels

  • Fibre Channel Host Adapters revision levels (HBA)

  • Management Modules (MM)

  • Ethernet Switch Modules (ESM)

For additional details on the latest hardware specifications updates for the IBM Cluster 1350, refer to the following IBM site:


The following link contains information that will help you to plan and install your cluster, the documents at this link, contain helpful graphics and links for the latest copy of the "Best Recipe", you can find this information under a link entitled "Cluster 1350 drivers":


5.1.5 Develop your security plan

Here is a listing of initial security issues that you should address:

  • You should define and document your root password along with the different accounts that will have access to the cluster.

  • Consider how you will manage your users. (Will you manually manage a small number of users or shared accounts, or will you use something like NIS to manage a large number of accounts?)

  • Decide whether you need a firewall to protect the cluster or isolate it from the rest of your network.

  • Determine how the access control lists (ACLs) should be configured on your switches and routers.

  • Determine which protocols to allow on your network, and what type of traffic that you should allow on your network.

  • Determine what types of out-of-band management that you will allow, in order to further automate the management of your cluster.

  • Make sure that your auxiliary systems and networks (that is, Backup Networks, Production Networks, and so on) do not allow uneccessary risks into your cluster or your environment.

  • Ensure that you have a clean, stable and possibly redundant power source for your cluster.

  • Ensure that steps have been taken to provide the appropriate level of physical security for your cluster.

  • Decide whether you will want to install an IDS solution on your Clusters.

    • Determine if Host Based (HIDS) Sensors should be installed on the Nodes and Network Sensors (NIDS) should be placed on the Clusters' networks.

5.1.6 Installation media

For both the Linux and CSM installation you should check that you have all the necessary material to perform the installation. It is important to gather the requirements prior to the installation of CSM, this is due to the scripts that automatically change your environment during the CSM installation. If you are are skillful enough to master this portion of the process, then the rest of the process will become much easier and prevent you from wasting many hours manually configuring your system for a single rpm. Some of the items that you will want to locate are listed here:

  • CD-ROMs:

    • Linux CD-ROMs (specific to your Distribution and Version)

      • In our test lab we are currently using RedHat 7.3.

    • CSM installation CD-ROMs

    • and Possibly the GPFS CD-ROMs

  • Drivers:

    • External Network Interface Drivers

    • Equinox Drivers

    • Myrinet Drivers

  • RPMs:

    • AutoUpdate RPM

    • atftp Server RPM

    • fping RPM

    • IBMJava2 RPM

    • UUCP RPM

    • Equinox Configuration and Diagnostic Utility

    • Kernel Source RPMs

    • Updated Libraries should also be kept on hand for the newer Kernel

    • and any other required packages for your configuration

Define how you want to install Linux and the rest of the supporting software. You can do this through the network with an NFS server or an FTP server, or from the CD-ROMs. You can also define a KickStart file. You also have to plan what packages you want to install by default along with your version of Linux. Remember to professionally balance security, organizational need(s), and functionality while selecting which default packages to install, then document, patch and test them after installation.

5.1.7 Documenting the cluster configuration

The key to a successful installation of any cluster is in accurately documenting, planning, and tracking the deployment of the cluster.

Use the planning worksheets

The various worksheets that may be used when installing the cluster are a powerful tool. You should familiarize yourself with the information needed and fill out the Node attributes worksheets provided in the CSM Setup HOWTO documentation or in Appendix D, "Planning worksheets" on page 295. These worksheets may be duplicated as necessary to record your configuration. In addition to their value during cluster configuration, the diagrams and worksheets are especially useful when making modifications or troubleshooting the cluster.

Make a cluster diagram

We recommend that you draw a cluster diagram (or use a graphic arts program to visually represent your cluster) that shows the configuration that represents all of the components of your cluster. It is a good idea to develop multiple levels of diagrams to properly show, a high-level overview (gear placement), mid-level design (show functional configuration and interconnectivity), close-ups of front and back (including buttons, sockets, and wiring diagrams). All of these documents will save you hours of time during future troubleshooting exercises. These documents will also be a valuable resource for training, and turnover.

Cluster configuration example

Here we provide an example of a cluster diagram and completed planning worksheets. This is a scaled down representation of the actual cluster that is being used for this redbook project. As this book progresses, we will attempt to portray, the cluster in more detail and accuracy. The cluster (as shown below) is comprised of:

  • One xSeries Model 345 server, acting as the management, installation, and user node

  • One xSeries Model 345 server, acting as a storage server

  • Four xSeries Model 335 servers for compute nodes

They are interconnected via an Ethernet 100/10 Mb switch for the cluster and the management VLAN and via an Equinox ESP-16 Terminal Server to provide serial connectivity to the compute and storage nodes. The management node has a second Ethernet card for the public VLAN.

We have Remote Supervisor Adapter cards in the first compute node (node1) and in the storage node (storage001). The RSA card in node1 provides power control for all compute nodes.

We also include a Myrinet high-speed network (not shown) to improve message passing and storage performance. Installation of the Myrinet adapter is discussed in 7.3.6, "Myrinet adapter installation" on page 193.

Figure 5-1 shows the cluster configuration diagram used in our lab.

click to expand
Figure 5-1: Lab cluster configuration

Figure 5-2 on page 106 shows some of the completed planning worksheets we used during the installation in our lab. They are:

  • TCP/IP attributes worksheets

    Contains TCP/IP network information, such as domain name, default gateway IP address, nameservers, compute nodes host names, IP addresses, and MAC addresses, as well as the cluster, management, and public VLANS.

  • Node attributes worksheets

    Contains Linux and CSM installation information, such as compute nodes host names, hardware control point for each node, power method, console server host name, number, console method and port number, hardware type, and installation method.

click to expand
Figure 5-2: CSM planning worksheets

 < Day Day Up > 

Linux Clustering with CSM and GPFS
Linux Clustering With Csm and Gpfs
ISBN: 073849870X
EAN: 2147483647
Year: 2003
Pages: 123
Authors: IBM Redbooks

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net