Running Regularly Scheduled Commands

Running Regularly Scheduled Commands

Regularly scheduled commands are an important part of system administration, and virtually all Unix systems provide a tool called cron that enables this. A typical " cron job" would be a script that rebuilds the locate database every Sunday at 3:15 a.m. (See also the sidebar "Run a Command Once in the Future.")

Starting in Mac OS X 10.4, Apple has introduced an additional tool, called launchd , that provides many of the capabilities of cron and adds other desirable features as well. This chapter will cover both the cron and launchd systems, which allow you to specify a schedule for running any command or script (often called jobs ). The most common schedules involve running a job at the same time every day, every week, or every month, but both cron and launchd allow you to schedule jobs down to the minute.

About the periodic Command

The periodic command is designed to run all the executable files in a single directory one after the other, in alphabetical order, and is common on Unix systems (such as Darwin) that inherit from FreeBSD. By default the periodic command uses three directories in which scheduled system-administration scripts are kept: /etc/periodic/daily/ , /etc/periodic/weekly/ , and /etc/periodic/monthly/ . For example, the script that updates the database used by the locate command is /etc/periodic/weekly/500.weekly . That same script also runs the makewhatis command that updates the file /usr/share/man/whatis used by man -k (aka the apropos command). You may also specify an arbitrary directory for periodic to look in for scripts to run.

The basic idea is that you have a set of scripts that should all be run at more or less the same time, so you schedule a run of the periodic command for a certain time and tell it which directory to use. Then you can add and remove scripts from the target directory without having to worry about messing with the scheduling function.

In Mac OS X before 10.4, the periodic command was run, um, periodically, via cron .

In Mac OS X 10.4, it is run periodically by launchd . See man periodic for more about this command, but note that the man page (as of Mac OS X 10.4.2) still refers to cron .

The periodic command is designed specifically to be run from a crontab file (although in Mac OS X 10.4 it is run via launchd ). The argument to the periodic command is the name of a directory inside the /etc/periodic/ directory. For example, the argument daily means that the periodic command will look inside /etc/periodic/daily/ and execute every executable file in that directory in alphabetical order.

The periodic command has a number of configuration options, which are set in the file /etc/defaults/periodic.conf . If you need to override any of the settings, you should do so by editing one of the other files used by periodic (see man periodic ). The default settings in /etc/defaults/periodic.conf are set to cause output from the daily scripts to go in the log file /var/log/daily.out , the weekly scripts in /var/log/weekly.out , and the monthly scripts in /var/log/monthly.out .

Scheduling jobs using cron

Let's focus on cron before moving on to launchd . cron is a special kind of program that in Unix we call a daemon . The cron daemon executes programs according to a schedule. Daemons are (often small) programs that are launchedsometimes we say "spawned"to run in the background, waiting to handle various system tasks as they arise. Most server software, such as Web servers, runs as daemons, as do many pieces of the operating system that are constantly running, waiting to do some particular kind of work. In Darwin/Mac OS X, daemons are managed by a master daemon called launchd , which we'll cover later in this chapter.

Every minute, the cron daemon checks the directory /var/cron/tabs for files whose names match user namesfor example, rootand reads any new or changed files into memory. The cron daemon also checks the system-administration file /etc/crontab . These files, called crontabs (for chronological tables ), are in a special format (described below) that tells cron what commands to execute and when. Commands in a crontab file are executed as the user who owns the crontab file (except for the special /etc/crontab file; see "About /etc/crontab ," below).

When a command from a crontab file is executed, any output from the command is e-mailed to the user owning the crontab file.

Users' crontab files are created and edited with the crontab command, which uses the vi editor (or the editor specified in your VISUAL environment variable) and performs a syntax check on the file before installing it. (This is similar to the working of the visudo command, covered earlier in this chapter.) The system-administration crontab file, /etc/crontab , must be edited directly using an editor such as vi . Unfortunately, the crontab command cannot be used to perform a syntax check on /etc/crontab .

You can view the man page for the crontab command with man crontab , and the man page for what goes inside crontab files with man 5 crontab . (To see why you need to add the 5 , review the man command in Chapter 3, "Getting Help and Using the Unix Manual.")

Make certain you are familiar with the text editor you are going to use ( vi by default, or whatever is in your VISUAL environment variable) before editing your crontab file.

Run a Command Once in the Future

Mac OS X comes with a standard Unix command called at that is used to run a command once at some future time. The at command depends on another command, atrun , which is disabled by default in Mac OS X, although it may be enabled (see man at ).

To run a command every day using cron:

1.	crontab -e The -e option says, "I want to edit my crontab file."
2.	Add the crontab command line. Let's say you want a summary of your disk-space usage to be added to a file every day at 10:23 p.m. The basic command you want to run is /usr/bin/du -skc /Users/vanilla >> /Users/vanilla/disk_use.log You should always use full paths for commands in crontab files because cron runs them with a very limited environment (you can find the path to a command with which command ). You add a series of five items to the crontab file, specifying when you want the command to run, and then the command line goes after the fifth item. It must be one long line. Figure 11.25 is an annotated example of a crontab entry and shows the meaning of each of the time fields and the entry to run the above command every day at 10:23 p.m. Figure 11.25. Example of a crontab file with an entry to run a command every day at 10:23 p.m. Since it will run every day, you put asterisks in the Day of month, Month, and Day of week fields.
3.	Save the file and quit the editor. If you are using vi , you press and then type :wq .

Tip

• When redirecting the output of a command line in your crontab, as in the example above, you can't have multiple commands on the command line. So if you want to run a cron job that consists of a series of commands, then you create a script and run it from your crontab.

  For example, you might have a script like this:

   #!/bin/sh /bin/date /usr/bin/du -skc /Users/vanilla 

  
  

  You could then run the script with cron and redirect the output of the script (which includes the date) into a log file. Figure 11.26 shows a series of examples of crontab enTRies. man 5 crontab gives the full, detailed specification for what is possible in a crontab file. In particular, notice that you can specify ranges of times or dates, so you can run a command every other month, every third hour , or whenever you want. Also, note that lines beginning with # are comments, which can be very helpful, so add some!

Figure 11.26. Examples of several crontab entries.

 # This is a comment. I love comments. # Set the SHELL to use for running cron commands, overrides whatever # the system default is. SHELL=/bin/sh # Mail any output to 'puffball', regardless of whose crontab this is. MAILTO=puffball # Use a day name for the day of week 5 4 * * sun echo "run at 5 minutes after 4 every Sunday" # run five minutes after midnight, every day. # Also redirects stderr to stdout so errors get emailed. 5 0 * * *   $HOME/bin/daily.job >> $HOME/tmp/out 2>&1 # run at 2:15pm on the first of every month 15 14 1 * * $HOME/bin/monthly # run at 10 pm on weekdays, annoy Joe # You can use % to include a <return> in an argument. 0 22 * * 1-5echo "Joe,%%Where are your kids?%"  mail -s "It's 10pm" joe # You can use ranges, e.g. 1-5 means 1,2,3,4,5 # Run at 9:00am every Monday, Wednesday and Friday 0 9 * * 1,3,5 mail -s "bring me flowers" [e-mail] # You can use step values with ranges by putting a / after the range. #  0-10/3 means 0,3,6,9 (the 10 never gets used) and */2 means "every two" 15 12 * */2 3 echo "run at 12:15 every second month but only on Wednesdays" 17 8-20/2 * * * echo "run at 17 minutes after every 2nd hour from 8am to 7pm everyday" 

Controlling who uses cron

You don't have to allow every user on the system to use cron . If you have a large number of users, there might be some you want to bar from using cron , or you might want to restrict the use of cron to a few users. This is easy to do.

If the file /var/cron/allow exists, then only users who are listed in that file may use cron .

If the file /var/cron/deny exists, then any user listed in that file may not use cron .

To restrict cron use to a list of specific users:

1.	Edit the file /var/cron/allow . This must be done by root, so if you are using the vi editor, the command would be sudo vi /var/cron/allow Enter your password if prompted for it. If the file already exists, make a copy of it first, in case you mess it up and need to go back to the original version.
2.	Add the user names of those who are allowed to use cron . These should appear one per linefor example, vanilla puffball
3.	Save the file and quit the editor.

To create a list of users who are not allowed to use cron:

1.	Edit the file /var/cron/deny . This must be done by root, so if you are using the vi editor, the command would be sudo vi /var/cron/deny Enter your password if prompted for it. If the file already exists, make a copy of it first, in case you mess it up and need to go back to the original version.
2.	Add user names to the file. Enter one user name per linefor example, jackie wesley
3.	Save the file and quit the editor. Any user listed in the file is unable to use cron .

About /etc/crontab

There is one special crontab file, /etc/crontab , already on your computer for system-administration purposes. Note that starting in Mac OS X 10.4, this file has no entries because the jobs that were handled here in earlier versions of Mac OS X are now handled by launchd . Because it is quite a bit easier to create a new cron job than a new launchd job, you might want to use /etc/crontab for system-administration tasks instead of the more complex launchd (see the next section, "Scheduling jobs using launchd ").

Because the /etc/crontab file is run by root, it uses a format a little different than the crontab files for users. The command entries in /etc/crontab use an extra field (the sixth field) that specifies which user the command will run as. This allows root to schedule commands to be run by less privileged users. So instead of the actual command line being everything after the first five fields, the command line to be run from an /etc/crontab enTRy is everything after the first six fields. (The command line is still the last thing in each line.)

Figure 11.27 is an example of an /etc/crontab file that runs a command to show the disk usage of all users. The command is set to run once a week, on Monday at 10:15 a.m. (but it won't run if your computer isn't on!).

Figure 11.27. Sample of an /etc/crontab file.

 # /etc/crontab SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/var/log MAILTO=[e-mail] # #minute hourday  month    wday who  command # #*/5   *    *    *   * root/usr/libexec/atrun # # Run daily/weekly/monthly jobs. 1510   *    *    1   root du -skch /Users/* 

Note the MAILTO setting that will cause the output of any cron jobs in the file to go to [e-mail], and how the job is set to be run as the root user.

Scheduling jobs using launchd

Remember the locate command mentioned in Chapter 4, "Useful Unix Utilities"? We told you that locate uses a database that is rebuilt every week. But how? By a process that is managed by the launchd daemon. See the sidebar "About launchd ."

The plist ( Property List ) format is widely used in Mac OS X/Darwin. You configure new scheduled jobs (also called agents ) for launchd by creating an XML file in the plist format using options specified in the launchd.plist man page (see man launchd.plist ), and then passing the file to launchd using a utility called launchctl . Also see man plist and the sidebar "More About the Property List Format."

The launchd framework allows for many configuration options, such as limiting the amount of CPU time or memory used by jobs. The launchctl command controls other aspects of launchd configuration, such as loading and unloading launchd jobs, logging, and monitoring launchd resource usage. (See man launchctl for more.)

The proper location of the XML files you create for launchd agents depends on whether they are per-user or systemwide agents and whether they are installed by regular users or by system administrators. Table 11.4 shows the standard directories used by launchd . The idea is that some launchd files may be installed or changed only by an administrator, while regular users can install and change their own launchd files, much as they can their own crontab entries.

Table 11.4. Directories Used by launchd

D IRECTORY	P URPOSE
~/Library/LaunchAgents	Per-user agents provided by the user
/Library/LaunchAgents	Per-user agents provided by the administrator
/Library/LaunchDaemons	Systemwide daemons provided by the administrator
/System/Library/LaunchAgents	Per-user agents installed with the operating system ( essentially by Apple)
/System/Library/LaunchDaemons	Systemwide agents and daemons installed with the operating system

More About the Property List Format

The plist , or Property List, format is widely used in Mac OS X/Darwin for system-configuration files and by virtually all applications to store user preferences. Have a look in ~/Library/Preferences and you will see many .plist files. The OmniGraffle diagramming application even uses the plist format for its documents (www.omnigroup.com/applications/omnigraffle).

In addition to the man page for the plist format (see man plist ), you should also check out the GUI tool for creating and editing plist files at /Developer/Applications/Property List Editor (which is only present if the Developer Tools are installed).

The plist format is an XML ( eXtensible Markup Language ) format. For more on XML, see W3C's "Extensible Markup Language (XML)" page (www.w3.org/XML) and XML for the World Wide Web: Visual QuickStart Guide , by Elizabeth Castro (Peachpit Press; www.peachpit.com).

The following task describes creating a scheduled job for launchd that is installed and runs from your user account. The XML file will be in your home directory, and the process that runs will be owned by your user account. Let's say you want to run a script called diskusage that is installed in the bin directory of your home directory, so it looks like this:

/Users/vanilla/bin/diskusage

You can of course name the file whatever you like. We recommend that the name end in .plist because the file is in the plist format. See Table 11.4 for other places to put launchd configuration files for different purposes.

You can use a text editor such as vi to create the file, or you can use the GUI application Property List Editor (located in /Developer/Applications/ ).

About launchd

Introduced in Mac OS X 10.4, launchd provides an extensive framework for managing both daemons and non-daemon jobs that launchd calls agents . That is, in launchd documentation, you will see references to agents, which means commands that are managed by launchd but do not run as daemons.

In Mac OS X/Darwin, launchd is used to launch several processes during system startup (see "The Boot Sequence," later in this chapter). It also provides a capability similar to cron you can use launchd to set up scheduled, repeated execution of commands and scripts, although not with quite the same level of fine control that cron provides when the jobs execute.

As seen in a typical property list in Figure 11.28 , the ProgramArguments key is for an array of one or more <string> elements. The first one is the actual command or script that will be executed. Any additional <string> elements are passed as arguments to the command. However, unlike what you do with cron jobs, you are not supplying a command lineyou cannot use shell operators like * , > , >> , and . You are simply supplying an executable and an (optional) series of arguments.

Figure 11.28. Example of an XML file for a launchd agent that runs at 3:45 a.m. every Monday. See man launchd.plist for the complete specification of what may go in this type of file.

 <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict>   <key>Label</key>   <string>diskusage</string>   <key>ProgramArguments</key>        <array>             <string>/Users/vanilla/bin/diskusage</string>        </array>   <key>ServiceDescription</key>   <string>Shows a summary of disk usage in my home directory.</string>   <key>StartCalendarInterval</key>   <dict>             <key>Minute</key>  <integer>45</integer>             <key>Hour</key>    <integer>3</integer>             <key>Weekday</key> <integer>1</integer>        </dict> </dict> </plist> 

Example of a Systemwide launchd Agent

One example of a systemwide launchd job (or agent) is com.apple.periodic-daily . This job simply executes /usr/bin/periodic with an argument of daily , and runs at 3:15 a.m. every day ( assuming the computer is on).

The launchd configuration file for this agent is /System/Library/LaunchDaemons/com.apple.periodic-daily.plist .

See "The Boot Sequence," later in this chapter.

To run a command every day using launchd:

1.	Create the XML file ~/Library/LaunchAgents/ filename For example (refer to Figure 11.28): ~/Library/LaunchAgents/diskuse.plist
2.	If you want to use shell operators, put those in the script that is being executed. For example, Figure 11.28 refers to a (made-up) script, /Users/vanilla/bin/diskusage , which might look like this: #!/bin/sh du -sk /Users/vanilla/* >> /tmp/vanilla.diskusage When launchd runs that script, the * and >> will work as expected because the script is a Bourne shell script (as indicated by the first line). See Chapter 9, "Creating and Using Scripts," for more on Bourne shell scripts.
3.	Once you have created the XML file, load it into launchd using launchctl load ~/Library/LaunchAgents/ filename For example: launchctl load ~/Library/ LaunchAgents/diskuse.plist This tells launchd to read the file and attempt to load it. You'll get an error message if the file doesn't conform to the specification in man launchd.plist or if the command you use in the ProgramArguments doesn't exist. If you have indicated a StartInterval and/or a StartCalendarInterval , then you are donethe agent will run at the times specified (assuming the computer is turned on).

Tips

• Read the man page for launchd.plist . There are quite a number of options available for configuring launchd agents. For example, WatchPaths can cause an agent to run whenever a listed path is modified, and EnvironmentVariables will set environment variables before running a job. See Table 11.5 for more examples.

• You can use the plutil command to check a plist file to see if it is a well- formed XML plist file (this won't catch misspelled keys, though):

  plutil file

• Use the StartInterval key to specify an interval in seconds. For example, to run the agent every 5 minutes:

   <key>StartInterval</key> <integer>300</integer> 

  
  

• You may load an entire directory of launchd configuration files by passing a directory name to launchctl load instead of a filename:

  launchctl load directory

• You can run an agent immediately at any time with

  launchctl start Label

  for example,

  launchctl start diskusage

  The Label is taken from the Label enTRy in the XML file. In Figure 11.28 the Label is diskusage .

• The folks at codepoetry have created a shareware GUI tool, Launchd Editor, that simplifies most of the work in creating new launchd agents and daemons (www.codepoetry.net/products/launchdeditor).

Table 11.5. launchd Configuration Options

Only the Label and ProgramArguments keys are required; all others are optional. See man launchd.plist for the complete list.
K EY	D ESCRIPTION AND E XAMPLE OF U SE
Label	Required. A unique name for the job. <key>Label</key> <string> uniquelabel </string>
Disabled	Disables the agent if true . See man launchctl and the entry for the -w option to the load and unload commands. Default is false . <key>Disabled</key> <true/>
UserName	If the agent is run by root, this sets the user to run the job as <key>UserName</key> <string> username </string>
GroupName	If the agent is run by root, this sets the group to run the job as <key>GroupName</key> <string> groupname </string>
Program	If present, this specifies the program (command, script, and so on) to be executed. If this key is not present, then the first item in the ProgramArguments array is used instead. (See next entry.) <key>Program</key> <string>program</string>
ProgramArguments	Required. An array of strings that are passed as arguments to the program. If the Program key is not present, then the first item is the program to run. <key>ProgramArguments</key> <array> <string> argument1 </string> <string> argument2 </string> </array>
WorkingDirectory	Causes launchd to do a cd into a particular directory just before running the job. <key>WorkingDirectory</key> <string> fullpath </string>
EnvironmentVariables	Environment variables to be set just before running the job. The format is a "dictionary of strings," which is the same format as the top-level structure of a launchd plist file itself: <key>EnvironmentVariables</key> <dict> <key> VARNAME </key> <string> value </string> <key> VARNAME2 </key> <string> value </string> </dict>
Umask	Specifies an argument to pass to the umask command just before running the job. (See "Default Permissions for File Creation," in Chapter 8.) The value should be a valid umask such as 077 . <key>Umask</key> <integer> value </integer>
WatchPaths	The job will be started if any of the paths are modified. <key>WatchPaths</key> <array> <string> /path1/to/watch </string> <string> /path2/to/watch </string> </array>
QueueDirectories	Same as WatchPaths except the job will be started only if the modified path is a directory and is not empty, so basically only if something is added to or removed from one of the listed directories. <key>QueueDirectories</key> <array> <string> /path1/to/watch </string> <string> /path2/to/watch </string> </array>
StartInterval	Specifies an interval in seconds to run the job, so if you want to run the job every 10 seconds, set it to 10, every 5 minutes, to 300, and so forth. <key>StartInterval</key> <integer> interval </integer>
StartCalendarInterval	A dictionary of integersyou may have one key each for Minute , Hour , Day , Weekday , and Month . You cannot specify multiple values, so (unlike with cron ) you cannot run a job on MondayFriday. If a key is missing, then the value for that key is every one , so you could run a job every day and then have the script figure out which day it is and decide whether or not to do anything. For the Day value use day of month, and watch out for 2931 ; not every month has a 31 st ! For the Weekday value, both and 7 are Sunday ( 1 is Monday). <key>StartCalendarInterval</key> <dict> <key>Minute</key> <integer> value 0 59 </integer> <key>Hour</key> <integer> value 0 23 </integer> <key>Day</key> <integer> value 1 31 </integer> <key>Weekday</key> <integer> value 0 7 </integer> <key>Month</key> <integer> value 1 12 </integer> </dict>
StandardOutPath	Path where all stdout from the job goes. Similar to using > to redirect stdout . (See "About Standard Input and Output," in Chapter 2.) <key>StandardOutPath</key> <string> /path/somewhere </string>
StandardErrorPath	Similar to StandardOutPath but redirects stderr for the job. <key>StandardErrorPath</key> <string> /path/somewhere </string>
HardResourceLimits	Allows you to impose resource limits on the job. See man launchd.plist for a list of available settings.

To unload a launchd agent:

• launchctl unload path
  
  The path is a path to a launchd configuration file or a directory containing one or more such files.
  
  

Tip

• If you add the -w option to the unload subcommand, a Disabled key set to true will be added to the configuration file(s) and the file(s) written back to disk. The -w option must come after the unload subcommand:

  launchctl unload -w path

To list jobs loaded into launchd:

• sudo launchctl list
  
  This displays a list of the labels of the jobs you have loaded into launchd .
  
  If you do not run the command line as rootthat is, if you omit the sudo commandthen only your personal launchd agents, the ones you have loaded without using sudo , will be listed. Figure 11.29 shows typical output, although yours may differ depending on which agents you have loaded.
  
  

  Figure 11.29. Example of typical output from launchctl list showing loaded launchd agents. Your output may differ.

   localhost:~ vanilla$  sudo launchctl list  Password: com.apple.KernelEventAgent com.apple.mDNSResponder com.apple.nibindd com.apple.periodic-daily com.apple.periodic-monthly com.apple.periodic-weekly com.apple.portmap com.apple.syslogd com.vix.cron org.postfix.master org.xinetd.xinetd com.openssh.sshd localhost:~ vanilla$