82 Synchronizing to a Remote Directory via FTP

#82 Synchronizing to a Remote Directory via FTP

This is the partner to Script #81, ftypsyncup , and it proves to be quite a bit simpler. It utilizes the ftp mget command to automatically retrieve the contents of all files in the remote directory, copying them one by one to the local system.

The Code

 #!/bin/sh # ftpsyncdown - Given a source directory on a remote FTP server, #   downloads all the files therein into the current directory. tempfile="/tmp/ftpsyncdown.$$" trap "/bin/rm -f $tempfile" 0 1 15      # zap tempfile on exit if [ $# -eq 0 ] ; then   echo "Usage: 
 #!/bin/sh # ftpsyncdown - Given a source directory on a remote FTP server, # downloads all the files therein into the current directory. tempfile="/tmp/ftpsyncdown.$$" trap "/bin/rm -f $tempfile" 0 1 15 # zap tempfile on exit if [ $# -eq 0 ] ; then echo "Usage: $0 user@host { remotedir }" >&2 exit 1 fi user ="$(echo $1  cut -d@ -f1)" server="$(echo $1  cut -d@ -f2)" echo " open $server" > $tempfile echo "user $user" >> $tempfile if [ $# -gt 1 ] ; then echo "cd $2" >> $tempfile fi cat << EOF >> $tempfile prompt mget * quit EOF echo "Synchronizing: Downloading files" if ! ftp -n < $tempfile ; then echo "Done. All files on $server downloaded to $(pwd)" fi exit 0 
 user@host { remotedir }" >&2   exit 1 fi user="$(echo   cut -d@ -f1)" server="$(echo   cut -d@ -f2)" echo "open $server" > $tempfile echo "user $user" >> $tempfile if [ $# -gt 1 ] ; then   echo "cd " >> $tempfile fi cat << EOF >> $tempfile prompt mget * quit EOF echo "Synchronizing: Downloading files" if ! ftp -n < $tempfile ; then   echo "Done. All files on $server downloaded to $(pwd)" fi exit 0 

How It Works

This script works almost identically to Script #81, Synchronizing Directories with FTP , and you'll find the helpful "How It Works" description there will also apply directly to this script. Also, as with Script #81, if you have a version of ftp that doesn't properly return a nonzero failure code to the shell when a transfer fails, simply remove the conditional block completely, leaving only

 ftp -n < $tempfile 

Running the Script

This script is invoked with the account name and server name of the remote system and an optional remote directory name that's the target from which to copy files. The current working directory on the local system receives whatever is copied .

The Results

Copying the contents of the remote archive directory to a new server is a breeze :

 $  ftpsyncdown [e-mail] archive  Synchronizing: Downloading files Password: Interactive mode off. Done. All files on intuitive.com downloaded to /home/joe/archive 

Hacking the Script

Like its partner script, ftpsyncup, ftpsyncdown doesn't deal with transferring directories in a graceful manner. It will stumble and output an error message for each subdirectory encountered in the remote directory.

Solving this problem is tricky because it's difficult to ascertain the directory and file structure on the remote ftp server. One possible solution would be to have the script execute a dir command on the remote directory, step through the output results to ascertain which of the remote matches is a file and which is a subdirectory, download all the files to the current local directory, make any necessary subdirectories within the local directory, and then, one by one, step into each new local subdirectory and reinvoke ftpsyncdown .

As with the suggested solution to a similar directory problem in Script #81, if you have ncftp you'll find that it has built-in support for recursive get commands. Rewriting this script to utilize that ncftp capability makes a lot more sense than continuing to struggle with the more primitive ftp command.

For a brief note on when to rewrite shell scripts in a "real" programming language, see the "Hacking the Script" section in Script #81.