Section 13.3. Simple Things Done Often

13.3. Simple Things Done Often

Here are some automation examples that are simple things we do a lot. Windows system administrators take heedthese examples are fairly Unix/Linux-centric, but the general principles apply to all operating systems.

13.3.1. Command Shortcuts

Most command-line systems have some kind of alias facility. This enables you to create new commands out of old ones. The syntax is different for every kind of command line. Unix has many different shell (command-line) languages, the most popular being bash and csh. They are different in many ways, but what you'll notice here (mostly) is that bash requires an equals sign. I'll give examples for both shells.

The bash examples will work for any shell modeled after the original Bourne Shell by Steve Bourne (/bin/sh), such as the Korn Shell (/bin/ksh), and the Z Shell (/bin/zsh). Likewise, the csh examples will work for any shell with csh roots, including the Tenex C shell (/bin/tcsh).

13.3.1.1. Getting to the right directory

For example, I often need to change directory (cd) to a specific directory that has a very long path. This is a good example of where an alias is useful.

Bash:

     alias book='cd ~tal/projects/books/time/chapters'

csh:

     alias book 'cd ~tal/projects/books/time/chapters'

Now I can type book whenever I want to be in the right directory for working on my current book. If I start working on a new book, I update the alias. (I've been typing "book" for the last six or so years!)

This not only saves typing, it records the location so that you don't have to memorize it. One less thing that you have to remember is always a good idea.

To make an alias permanent, you have to add the above line to your .profile, .bashrc (bash), or .cshrc file (csh). These files are only read at login, so either log out and log back in, or source the files to read them in again:

Bash:

     . ~/.profile

csh:

     source ~/.cshrc

(Note: the bash command to source a file is the period, or dot.)

An alias can contain multiple commands. Separate them with semicolons. Here's an example where we need to change to a particular directory and set an environment variable based on whether we're using the A system or the B system:

Bash:

     alias inva='cd ~tal/projects/inventory/groupa ; export INVSTYLE=A'     alias invb='cd ~tal/projects/inventory/groupb ; export INVSTYLE=B'

csh:

     alias inva 'cd ~tal/projects/inventory/groupa ; setenv INVSTYLE A'     alias invb 'cd ~tal/projects/inventory/groupb ; setenv INVSTYLE B'

Instead of using a semicolon, use && to indicate "Do this next command only if the first one succeeded." This can be useful to protect against running a command while in the wrong directory. For example, you want to go to a particular directory and write a timestamp to a logfile. However, if the cd fails (the server is unavailable), you don't want to accidentally create a logfile in your current directory.

Bash:

     alias rank='cd /home/rank/data && date >>.log'

csh:

     alias rank 'cd /home/rank/data && date >>.log'

Don't try to turn one OS into another. Aliases are great, but don't overdo it. I've often seen people developing dozens of aliases so that they can type DOS commands in Unix. I think this is a bad idea. You're never going to learn Unix that way, and the next time you are on someone else's machine and don't have access to those aliases, you'll be stuck.

13.3.2. Hostname Shortcuts

If there are particular hostnames you type over and over, you can save some time by creating aliases. For example, if you are often dealing with a machine called ramanujan.company.com, you can create an alias (a DNS CNAME record) called ram.company.com. That's a little less typing.

The problem with this technique is that it can become a maintenance nightmare. If people start to depend on both names, you're stuck maintaining both names. So how can you create an alias that only you know about that won't bother other people?

Typically, if there is a machine I access a lot, I'm accessing it almost exclusively via Secure SHell (SSH). SSH is a secure (encrypted) replacement for telnet and rsh. You can also use it to copy files (scp, a replacement for rcp), and many programs, such as rsync, use SSH. Unix SSH (OpenSSH and its brothers) lets you set up host aliases for all users on a Unix machine or aliases that are private for you.

To affect only your SSH sessions, add aliases to the ~/.ssh/config file. To affect all users of the system, add aliases to either /etc/ssh_config or /etc/ssh/ssh_config, depending on how your system was configured. In this example, I create an alias, es, so that I don't have to type www.everythingsysadmin.com all the time:

     Host es         HostName www.everythingsysadmin.com

Not only can I use ssh es where I used to type ssh www.everythingsysadmin.com, but the alias works for all SSH-related commands: scp, sftp, rsync, and so on. In fact, scripts and programs that I can't change will automatically pick up these settings. Some examples:

     $ ssh es     $ scp file.txt es:/tmp/     $ rsync ex:/home/project/alpha ~/project/alpha

I need to use ssh es so often that I actually created a shell alias to reduce my typing further:

Bash:

     alias es='ssh es'

csh:

     alias es 'ssh es'

The result is that I can now type es on the command line to log into the machine, or I can use es to refer to the machine when using scp or rsync. Same two letters either way. Cool, huh?

It is tempting to create two-letter aliases for every server in the world. However, you will soon find yourself spending more time remembering your coding system than using it. Personally, I limit myself to a few common machines that I access via SSH.

The ssh_config(5) manpage lists many other configuration options. For example, there is one machine that I occasionally access that requires a very specific combination of options on the command line. (It's a home-grown version of the SSH server that not only doesn't implement all the features but gets confused if you try to negotiate anything it doesn't understand.) The command I have to type to get it just right is:

     $ ssh -x -o RSAAuthentication=yes -o PasswordAuthentication=yes -o     ChallengeResponseAuthentication=no -1 peter.example.net

I could have set up a shell alias, but instead I can modify the SSH configuration, and all systems that use SSH will do the right thing. If a script that I can't modify uses SSH to reach that machine, these settings will still be used.

The lines in my ~/.ssh/config file look like this:

     Host peter.example.net         ForwardX11 no         RSAAuthentication yes         PasswordAuthentication yes         ChallengeResponseAuthentication no         Compression no         Protocol 1

SSH clients for Windows tend to have a GUI that will let you save profile settings to be used for a particular host or hosts.

The more you learn about SSH, the more you can do with it. There are many good books and online tutorials on the finer points of SSH, such as SSH, The Secure Shell: The Definitive Guide (O'Reilly). If there is one thing every system administrator should, but may not, know about SSH, it is how to set up public/private keys to securely eliminate the need to type passwords when SSHing from one specific machine to another.

13.3.3. A Makefile for Every Host

This section applies to Unix/Linux systems. Windows folks might want to skip it.

Unix/Linux systems often maintain critical information in plain text files that are edited by hand. Sometimes, after editing a file, you have to run a command to inform the system that the information has changed.

SSH to the Right Server in a Web Farm Every Time

Suppose you have three servers: server1.example.com, server2.example.com, and server3.example.com. You have many web sites divided among them, and remembering which site is on which server is getting to be a drag. Is www.everythingsysadmin.com on server 1 or 3? You think it's on 3, but someone may have moved it to 2 when you ran low on disk space. Why try to remember at all? No need to set up a configuration file, just SSH to the web site's hostname! For example, type ssh www.everythingsysadmin.com and soon you'll find yourself on the right machine. OK, that's pretty obvious, but you'd be surprised how often people forget that it works!

For example, after editing /etc/aliases (part of sendmail, Postfix, and various mail-transport-agent packages), you must run the newaliases command. That's pretty easy to remember, right?

After editing Postfix's transports file, should you run the newtransports command? No, that would be too obvious. You must run postmap transports. And there is the m4 command to run after editing .m4 files, and so on and so on.

Who has time to remember which command is used after which file is edited? Details like that are what computers are for.

make to the rescue! You might think of make as a programming toolthe program you run when compiling software. In reality, it lets you set up any kind of relationship involving the need to run a command to update one file if another changes.

make is one of the most powerful system administration tools ever invented. I hear programmers find it useful, too!

make has more features than Liz Taylor has had husbands, so I'll give a short introduction. (If you read the first two chapters of most books on make, you'll know 99 percent of what you need to for most system administration tasks and 10 times more than what your coworkers know.)

13.3.4. A Brief Introduction to make

make reads a configuration file aptly named Makefile. In this file, you will find recipes. They instruct make how to do its work.

Each recipe looks like this:

     whole: partA partB partC         command that creates whole

The recipe begins with the file that is going to be created, then a colon, and then it lists the files needed to build the main file. In this example, the recipe is about whole and establishes a relationship between it and partA, partB, and partC. If partA, partB, or partC is ever updated, then we need to (re)run the command that generates whole.

A real-world example helps:

     aliases.db: aliases         newaliases         @echo Done updating aliases

This code means that if aliases is changed, regenerate aliases.db using the command newaliases. Then the recipe outputs "Done updating aliases" to announce its success.

Notice that the second and third lines of the recipe are indented. They must be indented with a tab, not multiple spaces. Why? My theory is that the original creator of make wanted to punish me every time I use cut-and-paste on a system that turns tabs into spaces. However, I don't take it personally.

The update doesn't happen automatically. You have to run make to make it happen:

     Server1# make aliases.db     newaliases     Done updating aliases     Server1#

That's it! make read its configuration file, figured out that aliases was newer than aliases.db by checking the timestamp of the files, and determined that running newaliases would bring aliases.db up-to-date. If we run it again:

     Server1# make aliases.db     Server1#

There's no output. Why? Because now the timestamps on the files indicate that there is no work to be done: aliases.db is newer than aliases. make is lazy and will calculate the minimum amount of work required to do what you ask. It makes these decisions based on the timestamps of the files.

Here's another Makefile code sample:

     file1.output: file1.input         command1 <file.input >file.output     file2.output: file2.input         command2 file2.input >$@

In the first example, the command to be run uses stdin and stdout (file redirection using < and >) to read file.input and write file.output. The second example is similar, but the command takes the input filename on the command line and redirects the output to...what? Oh, $@ means "The file that this recipe is creating," or, in this case, file2.output. Why isn't it something simple like $me or $this? Who knows! You don't have to use $@, it just makes you look smarter than your coworkers.

make with no command-line parameters runs the first recipe in Makefile. It is traditional to name the first recipe all and have it run all the recipes you would expect as the default. This way, running make makes all the important recipes. It might not be literally all the recipes, but it is all the recipes you want to make by default. It might look like this:

     all: aliases.db access.db

make with no options then makes sure that aliases.db and access.db are up-to-date. Since there is no command as part of all, no file called all will be created. Thus, make always thinks that all is out-of-date ("Doesn't exist" equals "Is out of date"). You'll soon see why that is important.

Remember that make is lazy. If access.db is out-of-date but the other file isn't, it just runs the commands to bring access.db up-to-date. In fact, if bringing access.db up-to-date required something else, and that required something else, and so on, make would very intelligently do just the minimum work required.

In addition to all, I usually include a couple of other useful commands:

     reload:             postfix reload     stop:             postfix stop     start:             postfix start

Think about what that means. If I run make reload, make is going to notice that there is no file called reload, so it will run postfix reload thinking that the command will create a file called reload. Ah ha! I fooled them, didn't I? The command I listed tells postfix to reload its configuration. That command doesn't create a file called reload at all! Therefore, the next time I run make reload, make will run the command again. In other words, if you want something to always happen, make sure the recipe simply doesn't create the file that make is expecting to create.

With the above code in my Makefile, I can reload, stop, and start postfix by typing make reload, make stop, or make start, respectively. If there are other things that should be stopped (for example, an IMAP server, web-based email client, and so on), I can include commands to do those things in the recipes. I don't have to remember all the commands.

This is a good time for me to point out a little lie that I told earlier. I said that each recipe begins with the file that is going to be created, followed by a colon, and then it lists the files that make up the main file. make doesn't know whether those files really make up the file to be created. There's no way it could tell. Those items listed after the colon are really just dependencies that must be up-to-date.

Here's a simple Makefile from a system that runs Postfix and includes recipes for rebuilding the index for aliases and access. You'll notice that at the top are some constants (NEWALISES, PDIR, and so on) that are used throughout the file. Also, a backward slash (\) at the end of the line is used to continue long lines:

     NEWALISES=/usr/sbin/newaliases     PDIR=/etc/postfix     POSTMAP=/usr/local/postfix/sbin/postmap     # The "commands"     all: $(PDIR)/aliases.pag $(PDIR)/aliases.dir \             $(PDIR)/access.dir $(PDIR)/access.pag reload     reload:             postfix reload     stop:             postfix stop     start:             postfix start     #     # When aliases changes, generate the .pag and .dir files     #     $(PDIR)/aliases.pag $(PDIR)/aliases.dir: $(PDIR)/aliases             $(NEWALIASES)     #     # When access changes, generate the .pag and .dir files     #     $(PDIR)/access.dir $(PDIR)/access.pag: $(PDIR)/access             $(POSTMAP) $(PDIR)/access

Now I can edit either aliases or access and type make. I don't have to remember that the commands to update the indexes are extremely different. And I don't have to remember to tell postfix to reload its configuration each time because the all recipe includes that. The reload at the end of all will trigger that recipe every time.

make can also be useful for keeping files on various servers up-to-date. For example, let's suppose the aliases file in our example needs to be the same on both of our email servers. We decide that we'll update the file on this server, and push it to server2. That recipe might look like this:

     push.aliases.done: $(PDIR)/aliases         scp $(PDIR)/aliases server2:$(PDIR)/aliases         touch $@

We push the file to server2 using scp, then touch a file called push.aliases.done. Since this file is created after the successful copy of the file, we can build recipes so that the push is only done if it's absolutely needed. We can also force the file to be recopied by simply deleting the push.aliases.done file and typing make. traditionally, there is a recipe called clean that deletes all the *.done files and other machine-generated files.

There is nothing special about files that end with .done. It is simply customary to name-flag or timestamp files with .done at the end.

Here's a complete example. There are two files that need indexing if they change: aliases and access. If either of them has been reindexed, postfix is told to reload. They also are both pushed to server2 if they change. Finally, the command cd /etc && make is sent to server2 if and only if one or more of the files has been pushed to it.

By carefully constructing the recipes with proper dependencies, and touching *.done files where required, make will do the absolute minimal amount of work to bring the system up-to-date:

     #     # Makefile for server1     #     NEWALISES=/usr/sbin/newaliases     PDIR=/etc/postfix     POSTMAP=/usr/local/postfix/sbin/postmap     #     # High-level "commands"     #     all: aliases.done access.done reload_if_needed.done push     push: push.done     reload:         postfix reload     stop:         postfix stop     start:         postfix start     reload_if_needed.done: aliases.done access.done         postfix reload         touch reload_if_needed.done     clean:         rm -f \             $(PDIR)/aliases.pag $(PDIR)/aliases.dir \             $(PDIR)/access.dir $(PDIR)/access.pag \             push.aliases.done push.access.done reload_if_needed.done     #     # Recipes for particular files that need indexing/regeneration     #     # When aliases changes, generate the .pag and .dir files     aliases.done: $(PDIR)/aliases.pag $(PDIR)/aliases.dir     $(PDIR)/aliases.pag $(PDIR)/aliases.dir: $(PDIR)/aliases         $(NEWALIASES)     # When access changes, generate the .pag and .dir files     access.done: $(PDIR)/access.dir $(PDIR)/access.pag     $(PDIR)/access.dir $(PDIR)/access.pag: $(PDIR)/access             $(POSTMAP) $(PDIR)/access     #     # pushes     #     push.done: push.aliases.done push.access.done         ssh server2 "cd /etc && make"         touch $@     push.aliases.done: aliases.done         scp $(PDIR)/aliases server2:$(PDIR)/aliases         touch $@     push.access.done: access.done         scp $(PDIR)/access server2:$(PDIR)/access         touch $@

This Makefile is a good starting point for you to use on your systems. It is rather sophisticated because we do things to make sure Postfix isn't reloaded unless absolutely necessary.

With a Makefile like this, you no longer have to remember a multitude of commands and which ones should be used for which updated files. You never have to worry about forgetting to type a command. Many complicated procedures are reduced to:

1. Edit the appropriate file.

2. Type make.

make can be the ultimate tool for bringing together many smaller automated processes. Once, I had to merge the processes and procedures for three large networks into one. Each network had a different way of managing its aliases, hosts, and other administrative files. As I learned the procedures for each network, I constructed a Makefile specific to that network's master server. The high-level recipe names were the same in all three networks, but the commands they ran to accomplish the work were specific to each network.

The strategy was to create a new master server that would eventually replace all the legacy servers. Initially, the new master's Makefile simply initiated a make on the three legacy masters via rsh (this was long before ssh). I then migrated recipes to the new master one at a time. For example, first I decided that the new master would be the single source for the aliases file. I merged the aliases files of the three legacy networks and put the result on the new master. Once it was tested there, I added recipes on the new master to push that merged file to the legacy masters as if it were their own. I continued this process for each file or database.

Since each change was small and specific, I could test it incrementally. After literally hundreds of small changes, all the servers were "singing from the same songbook." At that point, it was easy to eliminate the legacy masters and let the new master be the authoritative master for all clients.

Any file that is automatically pushed to other servers should always have a comment at the top of the file warning other system administrators where the file came from and where to edit it.

Here's the warning I use:

     # THIS FILE IS MAINTAINED ON: server1.example.com     # Edit it with this command: xed file.txt     # If you edit this file on any other machine,     # it will be overwritten.  BE CAREFUL!

Since the previous note mentioned xed, I should explain what it is. There are many programs called xed, but this one can be found on http://www.nightcoder.com/code/xed. This program calls whatever editor you usually use ($EDITOR can be set to vi, pico, emacs, and so on) after locking the file. It is a must for any site that has multiple system administrators working on the same machine. If you are using RCS to track changes to a file, it does all the "check in" and "check out" work for you. This gives you infinite undo and a logfile of who changed what. If you find that a system has been acting funny for the last month, just check the log to see who changed the file a month ago and, well, be nicewe all make mistakes.