9.5 Automating Domino startup and shutdown

< Day Day Up >

The Domino server, either running under Linux natively in an LPAR or running as a guest operating system under VM, presents an unusual problem. It needs to be shut down properly and completely before the Linux operating system is shut down. If Linux is shut down without Domino closing all databases, corruption will likely occur in one or more databases.

The problem is a bit more complex when Domino is running in a Linux virtual machine. In this case, Domino is two levels down from the base operating system, z/VM.

Ideally, when the VM system is IPLed, we would like to have both the Linux guest operating systems and the Domino servers started. Additionally, when it is necessary to shut down the VM system for maintenance or other reasons, we need to insure that the Domino server is safely shut down before shutting down the Linux guests and VM operating system. Abnormal shutdown of the Domino server is a prime cause of database corruption.

To solve these problems, automating the startup and shutdown of the Linux guests and the Domino servers is made possible by using a mixture of VM and Linux command functions.

9.5.1 Automated startup

Starting and stopping a Domino server when Linux is IPLed and shut down is accomplished by executing a script which resides in the /etc/init.d directory. This script is enabled from the chkconfig command.

If Domino servers are present, multiple copies of the script can be enabled and executed to safely manage the server. The steps for setting up the server for automatic start and stop will work regardless of whether the Domino server is being run under Linux in an LPAR or as a VM guest.

Setup required at the Linux level

The script shown in Example 9-13 on page 236 will start the Domino server when Linux is logged on, and shut down the Domino server when a Linux shutdown has been requested. On shutdown, it will wait until the server or servers have shut down before allowing Linux to shutdown.

Example 9-13: Sample automated startup/shutdown script

 #!/bin/sh # # /etc/init.d/domino # ### BEGIN INIT INFO # Provides: domino # Required-Start: $network $remote_fs # Required-Stop: $network # Default-Start: 3 5 # Default-Stop: 0 1 2 6 # Description: Domino server ### END INIT INFO . /etc/rc.status # set the domino path and user name under which domino will run DOMINO_BIN=/opt/lotus/bin/server test -x $DOMINO_BIN || exit 5 DOMINO_USR=domservc DOMINO_TESTS=/opt/lotus/notes/latest/zlinux/server DOMINO_TESTC=/opt/lotus/notes/latest/zlinux/jvm/bin/exe/java rc_reset case "$1" in     start)         if checkproc $DOMINO_TESTS; then                 echo -n "Domino server is already running."                 rc_status -v                 exit         fi         if checkproc $DOMINO_TESTC; then                 echo -n "Domino Server Controller is already running."                 rc_status -v                 exit         fi         if test -e /$DOMINO_USR/notesdata/.jsc_lock;then                 rm /$DOMINO_USR/notesdata/.jsc_lock                 echo -n "Controller was not shutdown properly. .jcs_lock file removed. "         fi    echo 131072 > /proc/sys/fs/file-max    echo 15 > /proc/sys/net/ipv4/tcp_fin_timeout    echo 16384 > /proc/sys/net/ipv4/tcp_max_syn_backlog    echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse    echo "1024    65535" > /proc/sys/net/ipv4/ip_local_port_range         MAX_OPEN_FILES=`ulimit -n`         if [ $MAX_OPEN_FILES -lt 20000 ]; then             echo "setting maximum open files to 20000"             ulimit -n 20000         fi    echo -n "Starting Domino server"    echo "$DOMINO_BIN -jc -c &" | su - $DOMINO_USR    rc_status -v    echo    ;;     stop)    echo -n "Shutting down Domino server"    echo "Y" > /tmp/domino_$$    su - -c "$DOMINO_BIN -jc -q" $DOMINO_USR < /tmp/domino_$$    rm /tmp/domino_$$    rc_status -v    echo    ;;     restart)         $0 stop         $0 start         rc_status         ;;     status)         echo -n "Checking for Domino server: "         checkproc $DOMINO_TESTS         rc_status -v     echo        ;;     *)    echo "Usage: $0 {start|stop|restart|status}"    exit 1    ;; esac rc_exit

For the sake of clarity, we refer to this as the domservb script for the remainder of the section (you can use any name you wish for the script). Our personal preference was to use the actual name of the server (that is, domservb), as this allows for multiple copies of the script to be used to start multiple DPARs.

For the script to work, the .profile for the server must contain path statements for the location of the notes.ini file and the server:

 export DOMINO_LINUX_SET_PARMS=1 export PATH=/usr/local/bin:/usr/bin:/bin:/opt/lotus/bin:/domservb/notesdata

The following variables need to be modified in the script to allow it to work properly:

 DOMINO_BIN=/opt/lotus/bin/server DOMINO_USR=domservb

This script was developed and tested as part of the redbook project. It can be freely modified to suit your needs. While it worked in our environment, it may need modification to work correctly in a different environment. (No warranty is expressed or implied.) Check the release notes for your version of Domino for any information on starting or stopping Domino that might be provided.

Important:

The automated startup script (domservb) requires root authority to run.

To use this script, create a file called domservb. Type in the script and save it into the /etc/init.d directory. It must be placed into this directory in order to work.

Important:

When setting up the script, you should be working in the /etc/init.d directory. When issuing the domservb command from the /etc/init.d directory, you should use the ./domservb form of the command. To issue this command from any other directory when logged on as root or superuser, you must have /etc/init.d in the path.

Use the chmod command to correctly set the execute (x) permission bits on the file.

 linuxc:/etc/init.d # ls -al dom* -rw-r--r--    1 root     root         1588 Aug 26 17:47 domservb linuxc:/etc/init.d # chmod 755 domservb linuxc:/etc/init.d # ls -al dom* -rwxr-xr-x    1 root     root         1588 Aug 26 17:47 domservb

Use the chkconfig command to enable the script:

 linuxc:/etc/init.d # chkconfig domservb on

With this command entered, the domservb script will start the Domino server and the Java-based remote console controller on Linux startup, and shut down the server and controller on Linux shutdown.

Use the chkconfig command to disable the script:

    linuxc:/etc/init.d # chkconfig domservb off

You can issue the chkconfig command without the on or off parameter to check the status of the script:

    chkconfig domservb

The script accepts a number of commands which can be used to control the server:

stop	Stops the server and the remote java-based remote console controller
start	Starts the server and the java-based remote console controller
restart	Stops an restarts the server and the java-based remote console controller
status	Returns the status of the server

To check whether the Domino script works properly, issue the following command with the server down:

    domservb start

With the server start verified through the Domino remote console, test server shutdown:

    domservb stop

Status of a server can also be checked:

    domservb status

Possible responses are:

    Checking for Domino server:                               running    Checking for Domino server:                               unused

These show whether the server is running or unused (down).

How to run multiple DPARs using the script

Multiple servers can be stopped and started automatically simply by renaming the scripts to the server's name, modifying the DOMINO_USR= variable in the script to the server's name, placing the scripts in the /etc/init.d directory, and enabling the scripts with chkconfig.

9.5.2 Automated startup on IPL of the VM operating system

When the z/VM operating system is IPLed, a virtual service machine is automatically logged under the user ID AUTOLOG1; this is the default for z/VM. The PROFILE EXEC contains CP XAUTOLOG statements which automatically log on virtual service machines and virtual machines running guest operating systems like Linux.

When the guest virtual machines are logged on, the guest's PROFILE EXEC is executed if IPL CMS PARM AUTOCR has been specified in the VM directory entry for the user. The PROFILE EXEC contains commands which can be conditionally executed to set up the environment and IPL the guest OS.

Upon IPLing Linux, various start/shutdown scripts can be configured to run. We can add a script to Linux to start up a Domino server or multiple servers; see 9.5.1, "Automated startup" on page 235. Using a combination of XAUTOLOG, PROFILE EXEC and a Domino startup script in Linux, we can automate the startup of Domino.

If AUTOLOG1 is not used to logon the Linux guests, a REXX EXEC can be run from the VM OPERATOR user ID, which is typically the ID that is running on the VM console. This REXX EXEC would contain the CP XAUTOLOG statements for the Linux guests.

Setup required at the VM level

Add a line to the PROFILE EXEC on the AUTOLOG1 191 minidisk for each of the LINUX guests to be automatically started:

 CP XAUTOLOG LINUXA CP XAUTOLOG LINUXB CP XAUTOLOG LINUXC

This would cause LINUXA, LINUX B, and LINUXC to be autologged at IPL time.

The VM directory entries for each of the Linux guests running Domino servers must have the following statement:

 IPL CMS PARM AUTOCR

Each of the Linux guests running Domino servers must have a PROFILE EXEC that determines whether the guest virtual machine has been started in disconnected state, and if so, IPL the Linux OS. Additional setup tasks can be done as shown in Example 9-14:

Example 9-14: Sample profile exec

 /* Profile Exec    'ACCESS 19F Z'                       /* Access local tools disk.  */    'CP SET PF11 RETRIEVE FORWARD'       /* Retrieve first command     */    'CP SET PF12 RETRIEVE'               /* Retrieve last command */    'CP SET PF23 RETRIEVE FORWARD'       /* Retrieve first command  */    'CP SET PF24 RETRIEVE'               /* Retrieve last command */ /* If the ID has been autologged in disconnected state, start linux /* /* Else exit to CMS. if word(diagrc(24,-1),2)=2 then       /* Is the user disconnected */ 'exec swapgen 301 200000 (DIAG'        /* Setup VDisks for swapping */ 'exec swapgen 302 200000 (DIAG'  'exec swapgen 303 200000 (DIAG'  'exec swapgen 304 200000 (DIAG'  'exec swapgen 305 200000 (DIAG'  'exec swapgen 306 200000 (DIAG'  'exec swapgen 307 200000 (DIAG'  'exec swapgen 308 200000 (DIAG'  'exec swapgen 309 200000 (DIAG'  'exec swapgen 310 200000 (DIAG'  'cp set run on'  'CP IPL 200 clear'                    /* IPL Linux                  */  /* Exit

This PROFILE EXEC, which we used in our redbook project, will determine if the guest has been started in disconnected state (autologged). If so, it will use the SWAPGEN exec to prepare the virtual disks for use as Linux swap, and then IPL Linux (IPL 200). Upon IPL of the Linux OS, the startup script for the Domino server is called and the Domino server is started automatically as part of Linux startup.

Automating Server shutdown

The same domservb script that starts the server also stops the server when the Linux shutdown command is entered. The script issues a quit to the Domino server and controller, then waits until the Domino server shutdown is complete before returning control to the Linux shutdown process. No interaction is needed other than to issue the shutdown command to Linux. If multiple DPARs are running within the Linux image, and if they have been started using the automation script, they too will be shutdown before Linux terminates.

9.5.3 Automated shutdown when running Linux under VM

Linux guests can be shut down from VM using a REXX EXEC started at the VM operator console. The REXX EXEC will use two CP commands to accomplish the passing of the shutdown command to the LINUX guests.

CP SEND	This CP command sends console input through VM to a guest virtual machine. This guest (Linux, in our case) acts on the command as though it had been issued by a logged on user. To use CP send, a user must previously set the CP SET SECUSER command. This command requires a class C user privilege.
CP SET SECUSER	This command is used to set the issuing ID as a secondary user for a specified target virtual machine (in our case, a Linux guest). Until it is reset, the issuing ID will also receive the console responses from the target virtual machines console. This command requires the issuing ID be privilege class C.

The Linux shutdown command must be submitted by someone with root authority. Under normal circumstances, commands sent to the Linux guests using the CP SEND command will require the use of a password. There are two methods of dealing with this requirement:

Append a root password to the CP SEND command sent from class C operator ID.
Open a root console login on the Linux guest.

The first option is undesirable, since it exposes the critical root password to any user with access to the STPLINUX exec. Although the exec could be crafted in such a way as to require that the operator supply a password when the STPLINUX exec is run, the better approach is to create a shell script to log on root when Linux is initialized.

To enable root as login on Linux initial load:

Create the /root/bin/consolescript file:
```
 #!/bin/sh exec /bin/login -f root 
```
Set the execute permissions for the script:
```
 chmod 0700 /root/bin/consolescript 
```
Make a copy of /etc/inittab:
```
 cp -p /etc/inittab inittab.bkup 
```

Edit /etc/inittab..

Replace the following line:

 1:1235:respawn:/sbin/mingetty console

With this line:

 1:1235:respawn:/sbin/agetty -L -n -l /root/bin/consolescript 9600 console dumb

Set up the STPLINUX EXEC on an ID with sufficient privilege to issue CP SEND and CP SET SECUSER. The OPERATOR ID is recommended as the proper place for this exec since it will be used to shut down the VM system.

The STPLINUX exec should be incorporated into the execs that are used to shut down service virtual machines prior to VM shutdown. It is further recommended that it be one of the first execs executed in order to give the Linux guest sufficient time to shut down properly.

 /* stplnx - stops linux -                    */  trace all  'Pipe cms id () | spec w1 | var user'    /*Set user to current user */  say ''  say ' Shutting down the Linux systems '  say ''  /* list Linux system(s) in variable linuxid                         */  /* If multiple systems list them all separated by comma             */  /* List user ID of the system issuing this exec in variable user     */  user = maint  linuxid = linuxc linuxb  'pipe',     ' literal 'linuxid,     '| split',     '| stem linuxid.' do i=1 to linuxid.0  /* Set secuser, issue command, and reset secuser                    */    'SET SECUSER 'linuxid.i user  address command 'CP SEND 'linuxid.i 'shutdown -t -h 1 now'    'SLEEP 10 SEC'  'SET SECUSER 'linuxid.i' RESET'  end  Exit

When the STPLINUX exec is executed, it will issue the shutdown command to Linux. Linux will then shut down the Domino servers that are running in the guest. If there are multiple Linux guests to shut down, enter the user ID of each Linux guest in the variable linuxid, separated by a comma.

9.5.4 OS Maintenance with CheckOS

CheckOS is a script used to verify that the operating system contains the appropriate patch level in order to run Domino 6. The script is installed during Domino installation and resides in the Domino binaries directory (/opt/lotus/bin/checkos).

This script will run when you install Linux. You will see a couple of lines, including a link to the latest patches, and a line of information while the tool gathers the data. Next, the script checks and reports which OS you are running on the system, followed by the machine type and filesets required for the Domino 6 server to run properly. If there are any files missing, they will be reported in the section "The following OS patches are required:" You need to install the missing patches before continuing.

Note

At the time of writing, changes to CheckOS were still being considered. Check the Release Notes with your Domino version for the latest information on its capabilities.

< Day Day Up >