7.4 GPFS installation

---

7.4 GPFS installation

Up to this point, we have prepare the environment and covered the basis to proceed with the GPFS installation. This section describes the steps to install the GPFS software using the rpm command. For detailed information, refer to the IBM General Parallel File System for Linux: Concepts, Planning, and Installation Guide, GA22-7844.

The GPFS installation procedures are the following and will be described in the next sections:

1. Install all the source files.

2. Build the GPFS portability layer.

7.4.1 Installing the source files

Prior to installing GPFS, the following packages for both the SRC and RSCT components need to be installed and up and running:

• src-*.i386.rpm

• rsct.core.utils-*.i386.rpm

• rsct.core-*.i386.rpm

• rsct.basic-*.i386.rpm

However, if CSM has already been installed and configured on the nodes GPFS is about to be installed on, which would be our recommendation, most of the packages listed above have already been installed during the CSM installation. The only remaining package that needs to be installed on all nodes is rsct.basic-*.i386.rpm. If CSM has not been installed on the nodes, all SRC and RSCT packages must be included in the installation steps described below.

Keep in mind that GPFS and CSM should use the same level of SRC and RSCT code and CSM management server node should not be part of GPFS cluster nodes, because it adds other resource classes to SRC and RSCT subsystems. At the time of writing this book, the current levels of SRC and RSCT are SRC Version 1.2 and RSCT Version 2.3.

GPFS consists of four source files, which are:

• gpfs.base-*.i386.rpm

• gpfs.msg.en_US-*.i386.rpm

• gpfs.gpl-*.i386.rpm (only for the node that builds the GPFS portability layer)

• gpfs.docs-*.i386.rpm (optional, but recommended)

Because all nodes in our cluster have CSM installed, there is no need to include all SRC and RSCT packages. The following are the four packages that need to be installed:

1. rsct.basic-*.i386.rpm

2. gpfs.base-*.i386.rpm

3. gpfs.msg.en_US-*.i386.rpm

4. gpfs.docs-*.i386.rpm

Note that the gpfs.gpl-*.i386.rpm package will only be installed on the management node. After that, the GPFS portability layer will be built and distributed to all the other nodes.

The following are the installation steps:

1. Copy the rsct.basic-*.i386.rpm, gpfs.base-*.i386.rpm, gpfs.msg.en_US-*.i386.rpm, and gpfs.docs-*.i386.rpm files under the /csminstall/Linux/RedHat/7.3/i386/updates directory:

    # cp rsct.basic-*.i386.rpm gpfs*.rpm          \      /csminstall/Linux/RedHat/7.3/i386/updates 

2. Move to the /csminstall/Linux/RedHat/7.3/i386/updates directory:

    # cd /csminstall/Linux/RedHat/7.3/i386/updates 

3. Run the smsupdatenode command to install the files to all nodes. For more information on smsupdatenode command, see Chapter 6, "Cluster management with CSM" on page 149:

    # smsupdatenode -ai rsct*.rpm gpfs*.rpm 

7.4.2 Building the GPFS open source portability layer

You have to build the GPFS open source portability layer manually in one node (in our case, the management node), then copy them through all nodes. Make sure that you have a prepared kernel source files. See 7.3.3, "Prepare kernel source file for GPFS and Myrinet adapter" on page 190 on how to prepare your kernel source.

Below are the steps to build GPFS open source portability layer. Also, check the /usr/lpp/mmfs/src/README file for more up to date information on building the GPFS Open Source portability layer:

1. Install gpfs.gpl-*.i386.rpm in the management node:

    # rpm -ivh gpfs.gpl-1.3.0-0.noarch.rpm 

2. Modify and export the SHARKCLONEROOT variable to the /usr/lpp/mmfs/src value:

    # export SHARKCLONEROOT=/usr/lpp/mmfs/src 

3. Move to the /usr/lpp/mmfs/src/config directory:

    # cd /usr/lpp/mmfs/src/config 

4. Make a copy of the site.mcr.proto file, renaming it to site.mcr:

    # cp site.mcr.proto site.mcr 

5. Edit the /usr/lpp/mmfs/src/config/site.mcr file. There are some sections that need to be checked, as shown in Example 7-18.

   Example 7-18: Values changed in the /usr/lpp/mmfs/src/config/site.mcr file

    [root@masternode config]# more site.mcr /* Copyright IBM 1995 */ /* $Id: site.mcr.proto,v 1.371.4.3 2002/10/04 18:23:52 gjertsen Exp $ */ ........ /* #define GPFS_ARCH_POWER */ #define GPFS_ARCH_I386 ........ /* Linux distribution (select/uncomment only one) */ LINUX_DISTRIBUTION = REDHAT_LINUX ........ /* Linux kernel versions supported for each architecture */ #ifdef GPFS_ARCH_I386 #define LINUX_KERNEL_VERSION 2041800 ........ [root@masternode config]# 

   In addition, uncomment any desired Linux kernel patches listed under the LINUX_PATCH_DEFINES section. The kernel will also need to be rebuilt and installed with the appropriate patches, which can be obtained at the following site:

   http://www.ibm.com/developerworks/oss/linux/patches/

   For detailed information on each patch, check the GPFS Frequently Asked Questions at:

   http://www.ibm.com/servers/eserver/clusters/software/gpfs.html

   Note 

   If you are building a portability layer for the SuSe distribution of Linux, you must also copy the autoconf.h and version.h header files into the Linux source tree:

    cp /boot/vmlinuz.autoconf.h \         /lib/modules/'uname -r'/build/include/linux/autoconf.h cp /boot/vmlinuz.version.h \         /lib/modules/'uname -r'/build/include/linux/version.h 

6. Create your customized portability layer under the /usr/lpp/mmfs/src directory:

    # make World # make InstallImages 

   Make sure that you type World with a capitalized W instead of world.

   After you have issued the make InstallImages command, the mmfslinux, lxtrace, tracedev, and dumpconv files will be installed in /usr/lpp/mmfs/bin.

7. Copy the above binaries to the /cfmroot/usr/lpp/mmfs/bin directory and distribute them to all nodes using the cfmupdatenode command:

    # mkdir -p /cfmroot/usr/lpp/mmfs/bin # cd /usr/lpp/mmfs/bin # cp mmfslinux lxtrace tracedev dumpconv /cfmroot/usr/lpp/mmfs/bin # cfmupdatenode -a 

---