87 Avoiding Disaster with a Remote Archive

#87 Avoiding Disaster with a Remote Archive

Whether or not you have a good backup strategy, with tape rotation and so forth, it's still a nice insurance policy to identify a half- dozen critical files and have them sent to a separate off-site archive system. Even if it's just that one key file that contains customer addresses, invoices, or even email from your sweetheart, having an occasional off-site archive can save your life when you least expect it.

This sounds more complex than it really is, because as you'll see in this script, the archive is just a file emailed to a remote mailbox and could even be pointed to a Yahoo! or Hotmail mailbox. The list of files is kept in a separate data file, with shell wildcards allowed therein. Filenames can contain spaces too, something that rather complicates the script, as you'll see.

The Code

 #!/bin/sh # remotebackup - Takes a list of files and directories, #    builds a single archive, compressed, then emails it off to a #    remote archive site for safekeeping. It's intended to be run #    every night for critical user files, but not intended to #    replace a more rigorous backup scheme. You should strongly #    consider using unpacker, Script #88, on the remote end too. uuencode="/usr/bin/uuencode" outfile="/tmp/rb.$$.tgz" outfname="backup.$(date +%y%m%d).tgz" infile="/tmp/rb.$$.in" trap "/bin/rm -f $outfile $infile" 0 if [ $# -ne 2 -a $# -ne 3 ] ; then   echo "Usage: $(basename 
 #!/bin/sh # remotebackup - Takes a list of files and directories, # builds a single archive, compressed, then emails it off to a # remote archive site for safekeeping. It's intended to be run # every night for critical user files, but not intended to # replace a more rigorous backup scheme. You should strongly # consider using unpacker, Script #88, on the remote end too. uuencode="/usr/bin/uuencode" outfile="/tmp/rb.$$.tgz" outfname="backup.$(date +%y%m%d).tgz" infile="/tmp/rb.$$.in" trap "/bin/rm -f $outfile $infile" 0 if [ $# -ne 2 -a $# -ne 3 ] ; then echo "Usage: $(basename $0) backup-file-list remoteaddr {targetdir}" >&2 exit 1 fi if [ ! -s "$1" ] ; then echo "Error: backup list $1 is empty or missing" >&2 exit 1 fi # Scan entries and build fixed infile list. This expands wildcards # and escapes spaces in filenames with a backslash, producing a # change: "this file" becomes this\ file so quotes are not needed. while read entry; do echo "$entry"  sed -e 's/ /\\ /g' >> $infile done < "$1" # The actual work of building the archive, encoding it, and sending it tar czf - $(cat $infile)  \ $uuencode $outfname  \ mail -s "${3:-Backup archive for $(date)}" "$2" echo "Done. $(basename $0) backed up the following files:" sed 's/^/ /' $infile echo -n "and mailed them to $2 " if [ ! -z "$3" ] ; then echo "with requested target directory $3" else echo "" fi exit 0 
 ) backup-file-list remoteaddr {targetdir}" >&2   exit 1 fi if [ ! -s "" ] ; then   echo "Error: backup list  is empty or missing" >&2   exit 1 fi # Scan entries and build fixed infile list. This expands wildcards # and escapes spaces in filenames with a backslash, producing a # change: "this file" becomes this\ file so quotes are not needed. while read entry; do   echo "$entry"  sed -e 's/ /\ /g' >> $infile done < "" # The actual work of building the archive, encoding it, and sending it tar czf - $(cat $infile)  \   $uuencode $outfname  \   mail -s "${3:-Backup archive for $(date)}" "" echo "Done. $(basename 
 #!/bin/sh # remotebackup - Takes a list of files and directories, # builds a single archive, compressed, then emails it off to a # remote archive site for safekeeping. It's intended to be run # every night for critical user files, but not intended to # replace a more rigorous backup scheme. You should strongly # consider using unpacker, Script #88, on the remote end too. uuencode="/usr/bin/uuencode" outfile="/tmp/rb.$$.tgz" outfname="backup.$(date +%y%m%d).tgz" infile="/tmp/rb.$$.in" trap "/bin/rm -f $outfile $infile" 0 if [ $# -ne 2 -a $# -ne 3 ] ; then echo "Usage: $(basename $0) backup-file-list remoteaddr {targetdir}" >&2 exit 1 fi if [ ! -s "$1" ] ; then echo "Error: backup list $1 is empty or missing" >&2 exit 1 fi # Scan entries and build fixed infile list. This expands wildcards # and escapes spaces in filenames with a backslash, producing a # change: "this file" becomes this\ file so quotes are not needed. while read entry; do echo "$entry"  sed -e 's/ /\\ /g' >> $infile done < "$1" # The actual work of building the archive, encoding it, and sending it tar czf - $(cat $infile)  \ $uuencode $outfname  \ mail -s "${3:-Backup archive for $(date)}" "$2" echo "Done. $(basename $0) backed up the following files:" sed 's/^/ /' $infile echo -n "and mailed them to $2 " if [ ! -z "$3" ] ; then echo "with requested target directory $3" else echo "" fi exit 0 
 ) backed up the following files:" sed 's/^/   /' $infile echo -n "and mailed them to  " if [ ! -z "" ] ; then   echo "with requested target directory " else   echo "" fi exit 0 

How It Works

After the basic validity checks, the script processes the file containing the list of critical files, which is supplied as the first command argument, to ensure that spaces embedded in its filenames will work in the while loop (remember, by default spaces delimit arguments, so without some additional help, the shell will think that "test file" is two arguments, not one). It does this by prefacing every space with a backslash. Then it builds the archive with the primitive but useful tar command, which lacks the ability to read standard input for its file list and thus must be fed the filenames via a cat invocation.

 tar czf - $(cat $infile) 

The tar invocation automatically compresses the archive, and uuencode is then utilized to ensure that the resultant archive data file can be successfully emailed without corruption. The end result is that the remote address receives an email message with the uuencoded tar archive as an attachment. This should be a straightforward script.

Note  

The uuencode program wraps up binary data so that it can safely travel through the email system without being corrupted. See man uuencode for more information.

Running the Script

This script expects two arguments: the name of a file that contains a list of files to archive and back up, and the destination email address for the compressed, uuencoded archive file. The file list can be as simple as

 $  cat filelist  *.sh *.html 

The Results

 $  remotebackup filelist [e-mail]  Done. remotebackup backed up the following files:    *.sh    *.html and mailed them to [e-mail] 

A more sophisticated use of this script lets us tie it in to the system mirroring tool presented as Script #88, Mirroring a Website , with the third argument specifying a target unpacking directory:

 $  cd /web  $  remotebackup backuplist [e-mail] mirror  Done. remotebackup backed up the following files:    ourecopass and mailed them to [e-mail] with requested target directory mirror 

Hacking the Script

First off, if you have a modern version of tar , you might find that it has the ability to read a list of files from stdin , in which case this script can be shortened even further by updating how the file list is given to tar (for example, GNU's tar has a - T flag to have the file list read from standard input).

The file archive can then be unpacked (as explored in Script #88, Mirroring a Website ) or simply saved, with a mailbox trimmer script run weekly to ensure that the mailbox doesn't get too big. Here's a sample trimmer script:

 #!/bin/sh # trimmailbox - A simple script to ensure that only the four most recent #    messages remain in the user's mailbox. Works with Berkeley Mail #    (aka Mailx or mail): will need modifications for other mailers!! keep=4  # by default, let's just keep around the four most recent messages totalmsgs="$(echo 'x'  mail  sed -n '2p'  awk '{print }')" if [ $totalmsgs -lt $keep ] ; then   exit 0          # nothing to do fi topmsg="$(( $totalmsgs - $keep ))" mail > /dev/null << EOF d1-$topmsg q EOF exit 0 

This succinct script deletes all messages in the mailbox other than the $keep most recent ones. Obviously, if you're using something like Hotmail or Yahoo! Mail for your archive storage spot, this script won't work and you'll have to log in occasionally to trim things.