21.7 Essential Services Monitor Daemon (esmd)

21.7 Essential Services Monitor Daemon (esmd)^[1]

Sometimes a cluster will stop behaving properly because one of the critical daemons has died, for one reason or another. This should be a rare situation, but if it happens, it's up to the system administrator to identify the problem, figure out what went wrong, and correct it. To eliminate some of this problem, there is a new feature implemented in what is called the Essential Services Monitor daemon (esmd). This daemon watches certain essential daemons/services (thus its name) and will automatically restart them if they terminate. But "who watches the watchers?," you ask. The init process starts esmd and will restart it automatically if it terminates. So what are these "essential services" anyway? At least for now, it's the Event Manager daemon and the CAA daemon (if in a cluster). If you have an "essential service," you should implement it as a CAA resource. For more on CAA resources, please refer to Chapter 23 and Chapter 24.

Since all state changes and restart failures are logged through syslogd (to the daemon.log) or to the console (when syslogd is not running), you can see what the esmd is doing. If esmd is unable to restart a daemon, it will send a high priority message via syslogd. If this happens, the system administrator should investigate the problem, fix it, and manually restart the daemon since esmd will not try to restart a daemon once it fails to do so. After a failed attempt is corrected by the system administrator and the daemon is running again, esmd will again monitor that daemon and attempt to restart it if it terminates. You can verify that esmd has resumed monitoring the daemon by checking the syslogd log.

For more information about esmd, see the esmd (8) reference page.

^[1]This functionality is available in V5.1A Patch Kit 2 and later.