54 Ensuring That System cron Jobs Are Run | Wicked Cool Shell Scripts

#54 Ensuring That System cron Jobs Are Run

Until recently, Unix systems were all designed and developed to run as servers, up 24 hours a day, 7 days a week, forever. You can see that implicit expectation in the design of the cron facility: There's no point in scheduling jobs for 2:17am every Thursday if the system is shut down at 6pm for the night.

Yet many modern Unix and Linux users do shut down their systems at the end of the day and start it back up the following morning. It's quite alien to Mac OS X users, for example, to leave their systems running overnight, let alone over a weekend or holiday period.

This isn't a big deal with user crontab entries, because those that don't run due to actual shutdown schedules can be tweaked to ensure that they do eventually get invoked consistently. The problem arises when the daily, weekly, and monthly system cron jobs that are part of the underlying system are not run at the predefined times.

This script enables the administrator to invoke the daily, weekly, or monthly jobs directly from the command line, as needed.

The Code

 #!/bin/sh # docron - Runs the daily, weekly, and monthly #          system cron jobs on a system that's likely #          to be shut down during the usual time of day when #          the system cron jobs would occur. rootcron="/etc/crontab" if [ $# -ne 1 ] ; then   echo "Usage:  #!/bin/sh # docron - Runs the daily, weekly, and monthly # system cron jobs on a system that's likely # to be shut down during the usual time of day when # the system cron jobs would occur. rootcron="/etc/crontab" if [ $# -ne 1 ] ; then echo "Usage: $0 [dailyweeklymonthly]" >&2 exit 1 fi if [ "$(id -u)" -ne 0 ] ; then # or you can use $(whoami) != "root" here echo "$0: Command must be run as 'root'" >&2 exit 1 fi job="$(awk "NR > 6 && /$1/ { for (i=7;i<=NF;i++) print \$i }" $rootcron)" if [ -z $job ] ; then echo "$0: Error: no $1 job found in $rootcron" >&2 exit 1 fi SHELL=/bin/sh # to be consistent with cron's default eval $job 
 [dailyweeklymonthly]" >&2   exit 1 fi if [ "$(id -u)" -ne 0 ] ; then    # or you can use $(whoami) != "root" here   echo "  #!/bin/sh # docron - Runs the daily, weekly, and monthly # system cron jobs on a system that's likely # to be shut down during the usual time of day when # the system cron jobs would occur. rootcron="/etc/crontab" if [ $# -ne 1 ] ; then echo "Usage: $0 [dailyweeklymonthly]" >&2 exit 1 fi if [ "$(id -u)" -ne 0 ] ; then # or you can use $(whoami) != "root" here echo "$0: Command must be run as 'root'" >&2 exit 1 fi job="$(awk "NR > 6 && /$1/ { for (i=7;i<=NF;i++) print \$i }" $rootcron)" if [ -z $job ] ; then echo "$0: Error: no $1 job found in $rootcron" >&2 exit 1 fi SHELL=/bin/sh # to be consistent with cron's default eval $job 
 : Command must be run as 'root'" >&2   exit 1 fi job="$(awk "NR > 6 && // { for (i=7;i<=NF;i++) print $i }" $rootcron)" if [ -z $job ] ; then   echo "  #!/bin/sh # docron - Runs the daily, weekly, and monthly # system cron jobs on a system that's likely # to be shut down during the usual time of day when # the system cron jobs would occur. rootcron="/etc/crontab" if [ $# -ne 1 ] ; then echo "Usage: $0 [dailyweeklymonthly]" >&2 exit 1 fi if [ "$(id -u)" -ne 0 ] ; then # or you can use $(whoami) != "root" here echo "$0: Command must be run as 'root'" >&2 exit 1 fi job="$(awk "NR > 6 && /$1/ { for (i=7;i<=NF;i++) print \$i }" $rootcron)" if [ -z $job ] ; then echo "$0: Error: no $1 job found in $rootcron" >&2 exit 1 fi SHELL=/bin/sh # to be consistent with cron's default eval $job 
 : Error: no  job found in $rootcron" >&2   exit 1 fi SHELL=/bin/sh          # to be consistent with cron's default eval $job

How It Works

Located in either /etc/daily , /etc/weekly , and /etc/monthly or /etc/cron.daily , /etc/ cron.weekly , and /etc/cron.monthly , these cron jobs are set up completely differently from user crontab files: Each is a directory that contains a set of scripts, one per job, that are run by the crontab facility, as specified in the /etc/crontab file. To make this even more confusing, the format of the /etc/crontab file is different too, because it adds an additional field that indicates what effective user ID should run the job.

To start, then, the /etc/crontab file specifies the hour of the day (in the second column of the output that follows ) at which to run the daily, weekly, and monthly jobs:

  $ egrep '(dailyweeklymonthly)' /etc/crontab  # Run daily/weekly/monthly jobs. 15      3       *       *       *      root    periodic daily 30      4       *       *       6      root    periodic weekly 30      5       1       *       *      root    periodic monthly

What happens to the daily, weekly, and monthly jobs, though, if this system isn't running at 3:15am every night, at 4:30am on Saturday morning, and at 5:30am on the first of each month?

Rather than trying to force cron to run the cron jobs, this script locates the jobs and runs them directly with eval . The only difference between invoking the jobs from this script and invoking them as part of a cron job is that when jobs are run from cron , their output stream is automatically turned into an email message, whereas with this script the output stream is displayed on the screen.

Running the Script

This script must be run as root and has one parameter: either daily , weekly , or monthly , to indicate which group of system cron jobs you want to run. To run as root , sudo is recommended.

The Results

This script has essentially no output and displays no results unless an error is encountered either within the script or within one of the jobs spawned by the cron scripts.

Hacking the Script

A subtle problem here is that some jobs shouldn't be run more than once a week or once a month, so there should be some sort of check in place to ensure that that doesn't happen. Furthermore, sometimes the recurring system jobs might well run from cron , so we can't make a blanket assumption that if docron hasn't run, the jobs haven't run.

One solution would be to create three empty timestamp files, one each for daily, weekly, and monthly jobs, and then to add new entries to the /etc/daily , /etc/weekly , and /etc/monthly directories that update the last-modified date of each timestamp file with touch . This would solve half the problem: docron could then check to see the last time the recurring cron job was run and quit if an insufficient amount of time had passed.

What this solution doesn't avoid is the situation in which, six weeks after the monthly cron job last ran, the admin runs docron to invoke the monthly jobs. Then four days later someone forgets to shut off their computer and the monthly cron job is invoked. How can that job know that it's not necessary to run the monthly jobs after all?

Two scripts can be added to the appropriate directory. One script must run first from run-script or periodic (the standard ways to invoke cron jobs) and can then turn off the executable bit on all other scripts in the directory except its partner script, which turns the execute bit back on after run-script or periodic has scanned and ascertained that there's nothing to do: None of the files in the directory appear to be executable, and therefore cron doesn't run them. This is not a great solution, however, because there's no guarantee of script evaluation order, and if we can't guarantee the order in which the new scripts will be run, the entire solution fails.

There might not be a complete solution to this dilemma, actually. Or it might involve writing a wrapper for run-script or periodic that would know how to manage timestamps to ensure that jobs weren't executed too frequently.