Section 3.6. I Keep Having to Start an FTP Download from Scratch | Linux Annoyances for Geeks: Getting the Most Flexible System in the World Just the Way You Want It

3.6. I Keep Having to Start an FTP Download from Scratch

In many cases, Internet connections are less than reliable. The latest DVD for your favorite Linux distribution or a DVD-length video of your friends' wedding in Las Vegas requires several gigabytes of data. Even with broadband connections, such downloads can take many hours. If you're downloading over a wireless network associated with the 802.11b/g protocols, you may experience interruptions from ordinary devices such as microwave ovens and some cordless telephones, because they operate on the same range of frequencies.

Cordless telephones may be the most common problem for wireless networks. To avoid interference with either my 802.11a (5.8Ghz) or my 802.11b/g (2.4Ghz) networks, I have a 900Mhz cordless phone.

Even if you're using a wired Internet connection, there are a number of ways your download may get interrupted. If there's a thunderstorm anywhere between you and your download server, if there's electromagnetic interference near a cable, or if someone flips a switch in the wrong place, you may experience interruptions during your download.

There are many good applications that can help you restart connections after a download. I explore only two options in this annoyance: lftp from the command line and gFTP from the GUI.

Whichever you choose, there are other things you can do. If you have doubts about the integrity of a download, many services provide a checksum file you can use to verify it. If you administer an FTP server from which users download CD-length files, you can provide these same checksums for your users.

3.6.1. lftp

One reason I focus on lftp is that it is the FTP client of choice, at least as listed in an older version of the Red Hat Certified Engineer (RHCE) Exam Prep guide.

For more information on RHCE certification, see http://www.rhce.com.

The lftp client is a flexible file-transfer program that supports downloads from more than just FTP services. In fact, it can handle downloads from regular and secure web services (HTTP, HTTPS), Secure FTP services (SFTP, FTPS), and more. In this annoyance, I show you how you can access, start, and resume a download using this client. For more information on lftp, refer to its home page (it was developed in Russia) at http://lftp.yar.ru/.

To access a server such as kernel.org, you can run the following command:

 lftp mirrors.kernel.org

Alternatively, if you've already started the lftp client and see the associated lftp :~> prompt, you can connect to the desired FTP server with the following command:

 open mirrors.kernel.org

If you're familiar with FTP and bash shell commands, lftp should be easy. For example, you can check your current directory with pwd, list files and directories with ls, navigate to different directories with cd, download the file of your choice with get, and so on.

The lftp service also supports a command history, as well as tabbed command completion. For example, if you want to see what's in the fedora/ subdirectory, you can navigate with a double tap on your Tab key, as far as you need in the directory tree. In that way, you can easily find the appropriate subdirectory with the DVD ISO file that you need with the following command:

 lftp mirrors.kernel.org:/> cd fedora/core/4/i386/iso/

Then you can review available downloads and use standard FTP commands to get the ISO file you need. For example, if you want to download the Fedora Core 4 DVD, you can run the following command:

 get FC4-i386-DVD.iso

Now, if you were interrupted while doing this download and a partial file was left in your local directory, you can resume the interrupted download by adding the -c switch (otherwise, the download starts from scratch):

 get -c FC4-i386-DVD.iso

If you're downloading more than one file, you can mget those files; for example, if you really want to download all of the CD-based ISO files, you can run the following command:

 mget -c FC4-i386-disc?.iso

If you're not sure what's already available on your system, you don't have to exit lftp. With the ! prefix, you can use the shell commands of your choice; for example, the following command at the lftp prompt lists files in the current local directory:

!ls

3.6.2. gFTP

While gFTP is the GNOME FTP client, it also works well on the KDE desktop environment. As a GUI client, it is fairly easy to use, so it's suitable for your regular users. If you want to provide instructions, you might start with the following. It's best if you modify these instructions to use downloading examples that are of practical use to people in your organization. In addition, the location of gFTP in your GNOME and KDE desktop menus may vary. This case describes how regular users can download aircraft-related documents from the NASA FTP server.

To download one or more documents on airplanes from the NASA FTP server, use the following instructions:

Open gFTP. In GNOME, select Applications Internet gFTP.
Internet gFTP.
Data Exchange submenu.
If your connection is successful, you'll see a list of files and directories in the righthand listbox.
Navigate to the pub/ directory and then to the aircraft/ subdirectory. You can do so with appropriate double-clicks, or by entering /pub/aircraft in the righthand text box.
Highlight the documents you need. To start the transfer, you can click and drag these documents, click the left-facing arrow, or click Transfers Retrieve Files.
When you're finished, exit from gFTP. Choose FTP Quit, or press Ctrl-Q.

For more information on gFTP, review its home page at http://gftp.seul.org/.

3.6.3. Checksums

Those who provide large downloads would do well to provide checksums. They can help users make sure that the download will work as intended. Perhaps the most popular option for checksums is MD5, the message-digest algorithm designed by Ronald Rivest of MIT and documented in RFC 1321 (http://www.ietf.org/rfc/rfc1321.txt).

Checksums are commonly available for ISO file downloads associated with Linux distribution CDs and DVDs. For example, if you download the Fedora Core 4 ISO for a DVD, you should find a SHA1SUM text file in the same download directory.

When your download is complete, run the sha1sum command on the downloaded ISO file. For the Fedora Core 4 DVD ISO file, the specific command is:

 sha1sum FC4-i386-DVD.iso

The sha1sum command calculates a 128-bit checksum based on the noted file. The output from the command should match the checksum provided by the supplierin this case, in the SHA1SUM text file. As an example, the following is an excerpt from this file in the Fedora Core 4 ISO download directory:

 2f151a7329846da685c2a72fcb40eba3e8a355a0  FC4-i386-DVD.iso 3fb2924c8fb8098dbc8260f69824e9c437d28c68  FC4-i386-disc1.iso 31fdc2d7a1f1709aa02c9ea5854015645bd69504  FC4-i386-disc2.iso 032455cdf457179916be3a739ca16add75b768b7  FC4-i386-disc3.iso f560f26a32820143e8286afb188f7c36d905a735  FC4-i386-disc4.iso

If the checksums do not match, there may have been a problem with the download. The CD or DVD that you create from that ISO will almost certainly be faulty. In that case, it's best to try the download again.

If you're responsible for an FTP server from which users download CD-length files, you may want to provide checksums. It may be a SHA1SUM or MD5SUM checksum. All you need to do is run the md5sum or sha1sum commands on the files in question and add the checksums to a text file similar to the SHA1SUM file shown above.