Mail Clients

 < Day Day Up > 

Browsing the Web: lynx

lynx is a command-line web client. Surprising as it might seem, many people prefer browsing the World Wide Web in a text-only application. There are, of course, many pages that simply can't be browsed without a graphics-capable application, but those pages are written by people who aren't concerned with making their information as widely available as possible and don't seem to be of interest to people who prefer to browse in text only.


There's a reasonable chance that lynx isn't installed on your system. It's currently not distributed by Apple. It's a favorite, however, among command-line aficionados, so you might find it already installed at work or school. We're going to use it for a few things later in the chapter, so if you don't have it, you'll be installing it in Chapter 14, "Command-Line Software Installation and Troubleshooting." The install we do for lynx will be of the easiest possible command-line install style, so you might even want to flip forward a few pages and do the install now.

The basic syntax of lynx is lynx <URL>. This gives you a textual representation of the page and a few lines of prompting information as to what you can do from there. For example, looking at, lynx produces the following output:

 [ryoohki:~] sage% lynx #                                                                           Apple    #home index    Apple The Apple Store iPod+iTunes .Mac QuickTime Apple Support Mac OS X    Hot News Switch Hardware Software Made4Mac Education Pro business    Developer Where to Buy    Our hearts reach out to those hurt by the Indian Ocean tsunamis.    Help survivors and their families by making monetary donations to these    organizations:      * American Red Cross International Response Fund      * AmeriCares South Asia Earthquake Relief Fund      * CARE USA Asia Quake Disaster      * Direct Relief International International Assistance Fund      * M?decins Sans Fronti?res International Tsunami Emergency Appeal      * Oxfam Asian Earthquake & Tsunami Fund      * Sarvodaya Relief Fund for Tsunami Tragedy      * Save the Children Asia Earthquake/Tsunami Relief Fund      * SEVA South Asia Emergency Fund      * UNICEF South Asia Tsunami Relief Efforts      * World Food Programme Tsunami Disaster Appeal    United States Agency for International Development Donate to the    International Response Fund Support South Asia Tsunami Relief Efforts    Information resource for the humanitarian relief community      * Important Safety Recall -- Rechargeable Battery for 15-inch        PowerBook G4      * Expanded iBook Logic Board Repair Extension Program (12/17/2004)    _________________________ Search    Site Map | Search Tips    Visit the Apple Store online or at retail locations.    1-800-MY-APPLE    Find Job Opportunities at Apple.    Visit other Apple sites around the world:    [Choose...____]    Contact Us | Terms of Use | Privacy Policy    Copyright ? 2005 Apple Computer, Inc. All rights reserved.    Powered by MacOSXServer (NORMAL LINK)   Use right-arrow or <return> to activate.   Arrow keys: Up and Down to move.  Right to follow a link; Left to go back.  H)elp O)ptions P)rint G)o M)ain screen Q)uit /=search [delete]=history list 

Not bad, it's useable, and it sure loads faster than all those fancy graphics if you've got a slow connection!

If you want to move down the page, you can press the spacebar. Use the up- and down-arrow keys to move from link to link. Use the right-arrow key or press the Return key to select a link. The right-arrow and left-arrow keys take you, somewhat predictably, to the target of the currently selected link or back to the previous page. As you might have noticed, some of the comments we have made here appear at the bottom of the screen output. As you use lynx, it provides helpful hints on what you might want to do. These appear near the bottom of the screen and contain helpful information, such as how to enter text in a text entry field or move to the next page by using the spacebar.


I mentioned that lynx provides helpful hints on what to do because a significant amount of human-computer-interface research indicates that, to many people, omnipresent onscreen help is almost invisible. Keep your eye on the onscreen hints even if it's not the best user interface design, it is there to help, and it's a lot faster than searching the help pages.

Because you don't have a mouse and cursor with which to navigate pages being displayed in lynx, a number of keyboard commands are available to perform various actions. Table 13.1 shows common one-key commands within lynx.

Table 13.1. Common One-Key Commands Within the lynx Interactive Web Browser




Move up the page.


Move down the page.


Move up the page.

<space bar>

Move down the page.

<right arrow>, <return>

Go to selected link.

<left arrow>

Go back.

<up arrow>

Select previous link, downloadable element, or form field.

<down arrow>

Select next link, downloadable element, or form field.


Download the target of the currently selected link or downloadable element.


Go to the lynx help pages. These pages are implemented as HTML pages, so you can go forward and back in them with the forward and back arrows.


Go to the lynx Options page. Here you can set an assortment of internal parameters such as where your lynx bookmarks are stored.


Print the current page.


Go to a new URL.


Go back to the Main page, by which lynx means the page that you first started on.


Quit the program.


Search in the page.


Show the history for the current browser window.

A veritable plethora of additional one-key options are explained in the lynx help, under the Key-stroke Commands heading.

The lynx browser also sports a wide range of command-line options that enable or modify advanced behaviors. These include items such as sending the data to STDOUT and collecting a list of the URLs contained in the document.

Finally, it should be mentioned that lynx, like much Unix software, works great as a command-line building-block utility. Ever wanted to process the contents of a web page, perhaps to do something such as collect all the links from someone's page of interesting links, without having to dig through the source by hand? Using the -dump option causes lynx to send the target document of the URL to STDOUT, followed by a list of the URLs in the document. For example, if you wanted to collect a list of URLs to the files available on (specifically, the stuff in the downloads subdirectory), you could use lynx like this:

 [ryoohki:~/downloads] sage% lynx -dump                               Index of /downloads      * [1]Parent Directory      * [2]26FIG112.gif      * [3]26FIG133.gif      * [4]CGvirusscan.tgz      * [5]Python-2.2.tgz      * [6]addme.c      * [7]appendix.pdf      * [8]arpwatch.tar.Z      * [9]clref.pdf      * [10]cupsomatic      * [11]file      * [12]gdbm-1.8.0.tar.gz      * [13]ispell-3.1.20.tar.gz      * [14]      * [15]jpegsrc.v6b.tar.gz      * [16]libpng-1.0.10.tar.gz      * [17]logsentry-1.1.1.tar      * [18]lynx.1      * [19]lynx.10_1.gz      * [20]lynx.cfg      * [21]lynx.gz      * [22]majora.tar.gz      * [23]netpbm-9.12.tgz      * [24]nmap-2.54BETA25.tar.gz      * [25]page1440.pdf      * [26]parallel      * [27]pine4.43.tar.Z      * [28]portsentry-1.0.tar.gz      * [29]portsentry-2.0b1.tar.gz      * [30]postpipe      * [31]serial      * [32]serial-darwin      * [33]spell-1.0.tar.gz      * [34]termcap-1.3.tar.gz      * [35]unleashed.jpg     Apache/1.3.33 Server at Port 80 References    1.    2.    3.    4.    5.    6.    7.    8.    9.   10.   11.   12.   13.   14.   15.   16.   17.   18.   19.   20.   21.   22.   23.   24.   25.   26.   27.   28.   29.   30.   31.   32.   33.   34.   35. 

If you wanted to parse just the URLs out of this output, you could simply run lynx and pipe the output though grep looking for URL patterns. Something like lynx -dump | grep "http:" will produce the following output:

[ryoohki:~ downloads] sage% lynx -dump | grep "http:" 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35.

The -dump option turns out to be useful for doing things that don't relate to processing the URLs as well, such as downloading files from FTP or HTTPD servers. You see examples of this use of lynx during the software installs in Chapter 14. Table 13.2 shows the lynx syntax and most interesting options.

Table 13.2. The Command Documentation Table for lynx


Textual web browser

lynx [options] [<file>]


You can find out which options are available by running lynx -help. Here is the listing of command-line options for the current version of lynx:


Receive options and arguments from STDIN.


Accept cookies without prompting if Set-Cookie handling is on (off).


Authentication information for protected documents.


Prepend a request URL comment and BASE tag to text/html for-source dumps.


Use the bookmark page as the start file (off).


<NUMBER> of documents cached in memory.


Specify a lynx.cfg file other than the default.


Log keystroke commands to the given file.


Read keystroke commands from the given file.


Set the <N>-second connection timeout (18000).


Specify a file to use to read cookies.


Specify a file to use to store cookies.


Toggle handling of Set-Cookie headers (on).


Toggle forced core dumps on fatal errors (off).


With -traversal, output each page to a file. With -dump, format output as with -traversal, but to STDOUT.


Inhibit wrapping of text in <pre> when -dumping and -crawling, mark wrapped lines in interactive session (off).


Dump the first file to STDOUT and exit.


Toggle transmission of From headers (on).


User data for get forms, read from STDIN, terminated by '---' on a line.


Print this usage message.


Set home page separate from start page.


Toggles inclusion of links for all images (off).


Set the default index file to <URL>.


Disable URLs that point to remote hosts (off).


Include MIME headers and force source dump.


Disable bold video attribute.


Disable directory browsing.


Turn off color support.


Disable transmission of referer headers for file URLs (on).


Disable the link list feature in dumps (off).


Don't follow Location: redirection (off).


Disable transmission of referer headers (off).


Disable reverse video attribute.


Disable the miscellaneous information messages (off).


Disable underline video attribute.


Force numbering of links as well as form input fields (off).


Force numbering of links (off).


Authentication information for protected proxy server.


Toggle handling of single-choice SELECT options via pop-up windows or as lists of radio buttons (off).


User data for post forms, read from STDIN, terminated by '---' on a line.


Toggle pseudo-ALTs for inlines with no ALT string (on).


Restrict access to URLs in the starting realm (off).


Flush the cache on a proxy server (only the first document affected) (off).


Dump the source of the first file to STDOUT and exit.


Traverse all HTTP links derived from start file.


Set alternative Lynx User-Agent header.


Toggle [LINK], [IMAGE], and [INLINE] comments with filenames of these images (on).


Screen width for formatting of dumps (default is 80).


Omit backspaces in output if -dumping or -crawling (like man does) (off).

Accessing FTP Servers: ftp

ftp is the command name for the program that implements the FTP protocol (creative, no?). On the Macintosh, the Anarchie and Fetch programs have historically been the FTP clients of preference, and both of these provide features sadly lacking in the default command-line ftp interface. The command-line interface, however, is again a quick and convenient way to get or put a file or three without needing to launch a graphical client. It also tends to be better for diagnosis purposes when an FTP transfer fails or when a file can't be found. All the messages from the server can be seen immediately and are directly in response to the commands you issue, so if something's wrong, it's much clearer at what point it went that way.

To connect to a remote site using ftp, simply issue the command as ftp <ftp site>. Presuming that all goes well, this command connects you to the remote site and requests your user ID and password. If you're trying to connect to a public site, the default guest user ID is anonymous. After that, the site asks you for a password, which if you're connecting as an anonymous user, should be given as your email address. Responding properly to both these queries (anonymous and your email address or your correct user ID and password) takes you to an internal prompt in the ftp program from which you can traverse the site's directories and upload or download files.

Following is a sample of what you might see after connecting to a site that doesn't really want you there. This sort of information is largely hidden in the graphical FTP clients, frequently leaving you clicking Retry indefinitely; in reality, the site is trying to give you some helpful information.

 [ryoohki:~/downloads] sage% ftp Connected to 220 FTP server (Version wu-2.6.1(1) Tue Nov 6 12:29:49 EST 2001) ready. Name ( anonymous 331 Guest login ok, send your complete e-mail address as password. Password: 530-Sorry, the limit of 30 users logged in has been exceeded (30). 530-We've had to cut back to avoid swamping our outside link. 530- 530-Please try again later. 530- 530-To report problems, please contact 530 Login incorrect. 

And this is an example of what you might see if you have connected properly:

 [ryoohki:~ downloads] sage% ftp Connected to 220 FTP server (Version wu-2.6.1(1) Tue Nov 6 12:29:49 EST 2001) ready. Name ( anonymous 331 Guest login ok, send your complete e-mail address as password. Password: 230-Hello [unknown] 230- 230-This is the anonymous FTP archive of the Computer and Information 230-Science Department and The Ohio State University. 230- 230-You are user 3 out of 30 users currently allowed in. 230- 230-This FTP server is running on a Sun Enterprise 2, with approximately 230-10GB of disk space.  The directory space was recently reorganized 230-and cleaned. 230- 230-Mirrors of other sites are in /mirror 230-Everything else is in /pub 230- 230-Please report any problems to 230- 230 Guest login ok, access restrictions apply. Remote system type is UNIX. Using binary mode to transfer files. ftp> 


If you attempt to access these servers and get different results, don't be alarmed. These are simply examples of various results that can occur, and your results might be different depending on when you connect, where you're connecting from, and whether the site has changed its configuration since these dialogs were captured.

From this ftp> prompt, you can issue commands, such as help, rhelp, get, put, cd, ls, pwd, and potentially others, depending on the server configuration. The output from the help command gives you a list of commands available to you in your client (don't worry if this looks like a long list; we cover the good ones in the text that follows), and the output of rhelp tells you about commands on the server:

 ftp> help Commands may be abbreviated.  Commands are: !               features        mls             prompt          site $               fget            mlsd            proxy           size account         form            mlst            put             sndbuf append          ftp             mode            pwd             status ascii           gate            modtime         quit            struct bell            get             more            quote           sunique binary          glob            mput            rate            system bye             hash            mreget          rcvbuf          tenex case            help            msend           recv            throttle cd              idle            newer           reget           trace cdup            image           nlist           remopts         type chmod           lcd             nmap            rename          umask close           less            ntrans          reset           unset cr              lpage           open            restart         usage debug           lpwd            page            rhelp           user delete          ls              passive         rmdir           verbose dir             macdef          pdir            rstatus         xferbuf disconnect      mdelete         pls             runique         ? edit            mdir            pmlsd           send epsv4           mget            preserve        sendport exit            mkdir           progress        set ftp> rhelp 214-The following commands are recognized (* =>'s unimplemented).    USER    PORT    STOR    MSAM*   RNTO    NLST    MKD     CDUP    PASS    PASV    APPE    MRSQ*   ABOR    SITE    XMKD    XCUP    ACCT*   TYPE    MLFL*   MRCP*   DELE    SYST    RMD     STOU    SMNT*   STRU    MAIL*   ALLO    CWD     STAT    XRMD    SIZE    REIN*   MODE    MSND*   REST    XCWD    HELP    PWD     MDTM    QUIT    RETR    MSOM*   RNFR    LIST    NOOP    XPWD 214 Direct comments to 

Usually, the commands you'll be most interested in are the ones for moving around the filesystem, and for retrieving and sending files. The commands you're most likely to use frequently are the cd and lcd commands (which are analogous to the command-line cd command for the remote and local directories, respectively) and the get and put commands (which retrieve files from the server and send files to it).

Additionally, you can ask for help on specific commands one of the more interesting ones to ask about in the listing shown is the site command:

 ftp> rhelp site 214-The following SITE commands are recognized (* =>'s unimplemented).    UMASK           GROUP           INDEX           GROUPS    IDLE            GPASS           EXEC            CHECKMETHOD    CHMOD           NEWER           ALIAS           CHECKSUM    HELP            MINFO           CDPATH 214 Direct comments to 

The site command implements FTP-site-specific command options. You would need to contact the administrator to find out exactly what the command options are and which ones you are allowed to use.

Files that you get from the FTP server are placed into the same directory from which you issued the ftp command unless you specify otherwise by giving a download path along with the get command at the prompt.

An advantage of the ftp client included with Mac OS X is that you can eliminate some of the uses of cd and ls to navigate to where you want to be in the ftp TRee simply by issuing the ftp command with a complete URL. For example, to get the gdb-6.3.tar.gz file at, you could navigate the FTP site to get to the file or you could issue this ftp command:


Table 13.3 shows the syntax and most interesting options for ftp.

Table 13.3. The ftp Command Syntax and Useful Options


File transfer program.

ftp [-46AadefginpRtvV] [-N <netrc>] [-o <outfile>] [-P <port>] [-r <seconds>]

[-T <dir>,<max>[,<inc>][[<user>@]<host> [<port>]]] [<host>:<path>[/]]

[file:///<file>] [ftp://[<user>[:<pass>]@]<host>[:<port>]/<path>[/]]

[http://[<user>[:<pass>]@]<host>[:<port>]/<path>] [...]

ftp -u <url> <file> [...]

The remote host with which ftp is to communicate can be specified on the command line. Done this way, ftp immediately tries to establish a connection with the remote host. Otherwise, ftp enters its command interpreter mode, awaits commands from the user, and displays the prompt ftp>.


Forces active mode ftp. By default, ftp TRies to use passive mode ftp and falls back to active mode if passive mode is not supported by the server.


Causes ftp to bypass normal login procedure and use an anonymous login instead.


Forces a cache reload for transfers that go through the FTP or HTTP proxies.


Disables filename globbing (that is to say, it doesn't allow wildcard filename expansions).


Turns off interactive mode when transferring multiple files.


Does not attempt auto-login on initial connection. If auto-login is not disabled, ftp checks for a .neTRc file in the user's directory for an entry describing an account on the remote machine. If no entry is available, ftp prompts for the login name on the remote machine (defaults to the login name on the local machine), and if necessary, prompts for a password.


Enables passive mode operation for use behind connection filtering firewalls. This option has been deprecated because ftp now tries to use passive mode by default, falling back to active mode if the server does not support passive connections.


Enables verbose and progress. Default if output is to a terminal (and for progress, if ftp is in the foreground). Shows all responses from the remote server as well as transfer statistics.


Disables verbose and progress, overriding the default of enabled when output is to a terminal.

-o <output>

When auto-fetching files, saves the contents in output. If output is not - or doesn't start with |, only the first file specified is retrieved into output; all other files are retrieved into the basename of their remote name.

-P <port>

Sets the port number to <port>.

-r <seconds>

Retries the connection attempt if it failed, pausing for <seconds> seconds. Please be kind to the FTP servers and don't set this to a value smaller than 20 seconds or so larger would be better.

-T <direction>,<maximum> [,<increment>]

Sets the maximum transfer rate for <direction> to <maximum> bytes/second, and if specified, the <increment> to <increment> bytes/second.

-u <url> <file>

Uploads files on the command line to <url> where <url> is one of the ftp URL types as supported by auto-fetch (with an optional target filename for one-file uploads), and <file> is one or more local files to be uploaded.

When ftp is in its command interpreter mode awaiting instructions from the user, there are many commands that the user might issue. Some of them include


Sets the file transfer type to network ASCII. Although this is supposed to be the default, it is not uncommon for an FTP server to indicate that binary is its default.


Sets the file transfer type to support binary image transfer.


Same as binary.


Terminates the ftp session and exits ftp. An end of file also terminates the session and exits.

cd <remote_directory>

Changes the current working directory on the remote host to <remote_directory>.


Changes the current working directory on the remote host to the parent directory (same as cd ../)

lcd <directory>

Changes the working directory on the local machine. If no directory is specified, the user's home directory is used.


Terminates the ftp session with the remote host and returns to the command interpreter.

dir [<remote-directory> [<local_file>]]

Prints a listing of the directory on the remote machine. Most Unix systems produce an ls -l output. If <remote_directory> is not specified, the current directory is assumed. If <local_file> is not specified or is -, the output is sent to the terminal.

open <hostname> [<port>]

Attempts to establish an ftp connection on <hostname> at <port>, if <port> is specified.


Toggles filename expansion for mdelete, mget, and mput. If globbing is turned off, filename arguments are taken literally and not expanded.

delete <remote_file>

Deletes the specified <remote_file> on the remote machine.

mdelete <remote_files>

Deletes the specified <remote_files> on the remote machine.

get <remote_file> [<local-file>]

Downloads <remote_file> from the remote machine to the local machine. If <local_file> is not specified, the file is also saved on the local machine with the name <remote_file>.

mget <remote_files>

Downloads the specified <remote_files>.

put <local_file> [<remote_file>]

Uploads the specified <local_file> to the remote host. If <remote_file> is not specified, the file is saved on the remote host with the name <local_file>.

mput <local_files>

Uploads the specified <local_files>.

help [<command>]

Displays a message describing <command>. If <command> is not specified, a listing of known commands is displayed.

ls [<remote_directory> [<local_file>]]

Prints a list of the files in a directory on the remote machine. If <remote_directory> is not specified, the current working directory is assumed.If <local_file> is not specified or is -, the output is printed to a terminal. Note that if nothing is listed, the directory might have only directories in it. Try ls -l or dir for a complete listing.

mkdir <directory>

Makes the specified <directory> on the remote machine.

rmdir <directory>

Removes the specified <directory> from the remote machine.

passive [auto]

Toggles passive mode if no argument is given. If auto is given, acts as if FTPMODE is set to auto. If passive mode is turned on (default), the ftp client sends a PASV command for data connections rather than a PORT command. PASV command requests that the remote server open a port for the data connection and return the address of that port. The remote server listens on that port, and the client then sends data to it. With the PORT command, the client listens on a port and sends that address to the remote host, who connects back to it. Passive mode is useful when FTPing through a firewall. Not all ftp servers are required to support passive mode.


Displays a status bar indicating the progress of each transfer as it occurs. Seeing that something is actually happening can be a real comfort when doing large transfers over slow lines.


Prints the current working directory on the remote host.

rate <direction> [<maximum>

Throttles the maximum transfer rate to <maximum> bytes/second.


If <maximum> is 0, disables the throttle. Not yet implemented for ascii mode.

<direction> may be any one of these: get (incoming transfers), put (outgoing transfers), or all (both).

<maximum> can be modified on the fly by <increment> bytes (default: 1024) each time a given signal is received: SIGUSR1 (increments <maximum> by <increment> bytes, SIGUSR2 (decrements <maximum> by <increment> bytes. The result must be a positive number).

If <maximum> is not supplied, displays current throttle rates.

Terminals in Terminals: telnet, rlogin, ssh

Because one of the primary methods for interacting with a Unix machine that you're sitting in front of is via a textual terminal, it should come as no surprise that a number of network tools are available to allow you to access remote machines through that same interface. The three primary examples of these are the telnet, rlogin, and ssh/slogin (secure shell) clients. Each of these provides a connection to a remote machine that is analogous to the one that provides to your local machine you get access to a command prompt and can run software on the remote machine just like software in on the local machine.

The telnet Program

telnet is a venerable connection program that speaks a language compatible with the over-the-wire communication protocol used by many Internet services. The protocol is a fundamental building block of much of the Internet and has been used to provide everything from web services to file transfer services to terminal services. It is, unfortunately, as trivial as it is ubiquitous and provides almost no built-in security. Because of this, terminal services implemented directly in the protocol are inherently insecure, and the telnet client and server fall into this category.

The syntax of the telnet command is telnet <host> [port number].

If you're communicating with a system that's either not connected to the Internet or run by a particularly non-security-conscious system administrator, you might actually be able to use it as a terminal application. In that case, if you issue the telnet command, you might see something like the following:

 ryoohki:~ sage$ telnet Trying Connected to ( Escape character is '^]'. Red Hat Linux release 7.0 (Guinness) Kernel 2.4.2 on a 2-processor i686 login: adam Password: Last login: Thu Apr 19 19:36:23 on vc/1 You have mail. Terminal: vt100. Printer set to newsioux krpan adam % 

At that point, you're at a shell prompt on the remote machine and can interact with it just as you interact with your local machine via its shell prompt in the terminal.


Don't expect to actually be able to telnet to to test this. We don't know any system administrators who leave telnet available on their machine, and we had to enable it specifically for the example.

If everyone you know is concerned about security and has their telnet daemons disabled, there are still a number of interesting uses for the telnet client. Because many servers for other Internet applications speak the same protocol, you can use the telnet protocol to talk to them as well. It might not seem like a useful idea to be able to talk to a web server with a terminal program that doesn't understand anything about the HTTP language and can't display the data properly, but it turns out to have a number of interesting applications.

For example, your web browser tells you that a server isn't responding can you tell whether it's the web server software that's not responding or the machine that hosts it that's not responding? telnet to the HTTP port (port 80) on the server, and see what the response is. If the web server software and machine are both okay, your session should look something like this:

 ryoohki:~ sage$ telnet 80 Trying Connected to Escape character is '^]'. 

If the machine is okay, but the web server software isn't speaking, the session might instead look more like this:

 ryoohki:~ sage$ telnet 80 Trying telnet: connect to address Connection refused telnet: Unable to connect to remote host 

If the machine is completely absent from the network, such as catbert in the following example, the response gets only to the TRying line and hangs there, well, trying I pressed Ctrl-C in the example to convince it to give up.

 ryoohki:~ sage$ telnet 80 Trying ^C 

Finally, if there really isn't a machine by that name at all, you'll see

 ryoohki:~ sage$ telnet 80 No address associated with nodename 


Please don't use the telnet program as a terminal program unless you are connecting to a machine that has no connection to the Internet. The program transfers all data in plain text, and anyone with physical access to any of the communication hardware involved in the connection (such as the phone lines, ethernet wiring, or the air [if you're using AirPort]) can read everything you type, including user IDs and passwords out of the data stream. There are better alternatives that we'll cover shortly.

The rlogin Program

Whereas the telnet communication package was conceived with hardly any concern for security, the rlogin communications package was developed under the seemingly quaint notion that certain connections could be trusted, based only on their self-proclaimed credentials. Passing its data using the same unprotected protocol as telnet, rlogin is supposed to give the administrator some confidence in the identity of a connecting visitor by virtue of the fact that the connection came from a trusted port. Using it is similar to telnet, except that it doesn't accept an optional connection port and it automatically fills in your user ID on the remote system based on your local system user ID. The syntax is simply rlogin <remotehost>.


A long, long time ago, in a decade almost two removed, the Internet and the perceptions regarding users who could connect to it were very different. Along with the lack of all over the place and the lack of spam in your email, Unix was an expensive commercial operating system. Machines that ran it were expensive, and the people who ran them, even people on different ends of the earth who had never met each other, thought of each other as fellow members of a professional fraternity. Professional courtesies were extended, and if you were a system administrator, a concern about another's security was the same as a concern about your own security.

Because of this expectation that any person running a Unix machine was another security-conscious professional, early security measures were based on utilizing this trust as a form of security credential. Security-conscious professionals were as worried about allowing security risks on others' machines as incurring security risks on their own. A security-conscious professional would never let a "bad" user use his system. The only people who could connect using the rlogin client were users on some Unix machine somewhere. Taken together, any user with valid credentials on a Unix machine, verified by their being allowed to run the rlogin program, must, by association, be a trustable user.

Taken in the context of today's rampant attacks against system security, this might sound like a naively bad security method, but until the advent of "personal Unixes" such as Linux, it worked surprisingly well. Given that much of today's data is passed around with equally insecure connections, without even a trust-based attempt to verify the authenticity of the content, we probably shouldn't poke too much fun at the naiveté of the early communication packages.

As you are coming into the world of having your own personal Unix machine connected to the Internet, we encourage you to adopt some of the historic notions regarding administrator responsibility and fraternity, but not their naive notions regarding security.

As with the telnet program, if you're connecting to machines that aren't connected to the Internet, the rlogin client is just as good as any. If you're connecting to machines that are connected to the Internet, please don't use the rlogin program, even if the remote machine makes it available. Doing so only risks your accounts and data on both local and remote machines, and the security of both machines as well.

The Secure Shell Software Suite: slogin/ssh, scp, sftp and Others

The Secure Shell collection of programs provides strongly encrypted communications between your machine and a remote server. The implementation that Apple has chosen to provide is based on the OpenSSH ( distribution of the protocols. The protocol requires both client software, which we cover in this chapter, and server software, which is covered in Chapter 21, "Accessing and Controlling Tiger Remotely." Here, we assume that you already have a server to talk to and detail the use of the client software on the Unix side of your Mac OS X machine to talk to your remote server.

slogin The starting point for use of the Secure Shell client is the slogin (also available under the name ssh) program. This program replaces the functionality of the telnet and rlogin programs and provides some additional capabilities as well. Unlike telnet and rlogin, slogin passes all information between the machines as encrypted data, using a public-key encryption method.


Public-key encryption is a clever method of encrypting data. Basically, in public-key encryption schemes, every person interested in exchanging encrypted information creates two keys. One of the created keys is the person's private key and the other is the person's public key. These keys are mathematically related, but one cannot be derived from the other. The cleverness resides in the mathematical relationship between the keys.

When you encrypt data, you encrypt it using two keys: your private key and the public key of the message's intended recipient. The keys are related in such a fashion that data encrypted with your private key, and another's public key, can be decrypted only with a combination of your public key and the other person's private key. This encryption method is used in both systems such as the PGP (Pretty Good Privacy) email encryption software and in data transmission software such as ssh. In email encryption, you use your private key and the email recipient's public key to encrypt mail destined for them, and the recipient uses your public key and their private key to encrypt mail destined for you. In the encryption of data transmission in software such as ssh, the system again uses your private and public keys and a pair of private and public keys belonging to the remote system to which you are connecting.

The basic use of slogin is much like that for rlogin simply issue the command slogin <machinename>, where <machinename> is the name or IP address of the remote machine to which you want to connect. If the remote machine is running a Secure Shell server and it is configured to allow you to connect, the server responds by asking for your password. If you respond correctly, you are left at a shell prompt on the remote machine and can type into it and execute commands, just as though you were in a window typing to your local machine.


The command ssh is equivalent to the command slogin. We use slogin in our examples and discussion here to make it clear where we're talking about slogin the program, and use SSH as the acronym for the Secure Shell package, but you can substitute the command ssh wherever you see slogin used here.

A successful slogin attempt might look something like this:

 brezup:ray testing $ slogin's password: Last login: Tue May 13 2003 01:16:06 -0500 from dhcp065-024-074- You have new mail. ...Remote login... Rosalyn ray 1 > 

Again, at this point we're at a shell prompt on the remote machine

If you don't want to log in to the remote machine as the same user ID as you are on the current machine, you can specify a user ID using -l <username> after the hostname. Alternatively, you can use <username>@<hostname> to specify the user and host. If I wanted to log in to rosalyn as user testing (regardless of what user I am on my local machine), I could use this syntax:

 brezup:ray testing $ slogin's password: Last login: Tue Jun 24 2003 15:30:04 -0500 You have new mail. ...Remote login... Rosalyn testing 1 > 

Some system administrators choose not to allow remote logins through simple password authentication. Passwords are generally too short to be difficult for a computer to guess by simple brute-force methods. Instead, the Secure Shell suite allows the use of arbitrarily long, multiword passphrases. An slogin connection requiring this type of login looks like this:

 brezup:ray testing $ slogin -l joray Enter passphrase for key '/Users/ray/.ssh/id_dsa': Last login: Tue Aug 06 2003 14:39:47 -0500 from cvl232015.columb You have new mail. ...Remote login... Rosalyn joray 1 > 

If the remote machine is running this more restrictive security (and we recommend that you do so if you choose to enable remote connections to your machine when we get to Chapter 26, "Creating a Mail Server"), you will be asked, not for your password, but for your passphrase if you have created one. The connection will be refused if you have not created a passphrase.

Creating a passphrase involves a bit of work on your part. This is because if you really want security, you can't allow the encrypted keys that identify you to be seen on the network. Therefore, after the key has been created, you need to transfer it to the remote machine via some old-fashioned, physical method, such as writing it to a floppy disk and taking that disk directly to the remote machine.


If you're in charge of setting up both machines, you could leave password access under Secure Shell on long enough for you to copy the keys back and forth on the encrypted channel, and then turn off password access to tighten security.

Creating a passphrase for yourself involves the following: On your Mac OS X machine, generate a key pair by running

 ssh-keygen -t <type> 

The -t option specifies the key type to be generated. This can be rsa for RSA or dsa for DSA (isn't case sensitivity fun?) RSA is used in SSH1 servers, whereas either RSA or DSA can be used with SSH2 servers. RSA (which stands for Rivest-Shamir-Adelman, its developers) is the most commonly used public key algorithm. DSA, Digital Signature Algorithm, is a signature-only algorithm, based on the Diffie-Hellman discrete logarithm problem.

When you run ssh-keygen, you are asked for a passphrase to protect the private key. It is recommended that the passphrase be at least 11 characters long and include as many character types as possible: uppercase letters, lowercase letters, numbers, and special characters. Spaces may be included as part of the passphrase.

Here is a sample run:

 brezup:miwa miwa $ ssh-keygen -t dsa Generating public/private dsa key pair. Enter file in which to save the key (/Users/miwa/.ssh/id_dsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /Users/miwa/.ssh/id_dsa. Your public key has been saved in /Users/miwa/.ssh/ The key fingerprint is: 7d:25:3e:87:3b:25:24:cf:5a:05:0e:1d:19:ad:67:10 miwa@brezup 

As ssh-keygen tells us, user miwa does indeed have the promised keys, as shown in the following output. The private key was saved as id_dsa, and the public key was saved as; both are stored in the directory ~/.ssh/.

 brezup:miwa miwa $ ls -al ~/.ssh total 16 drwx------   4 miwa  miwa  136 17 Aug 22:09 . drwxr-xr-x  12 miwa  miwa  408 17 Aug 22:08 .. -rw-------   1 miwa  miwa  744 17 Aug 22:09 id_dsa -rw-r--r--   1 miwa  miwa  601 17 Aug 22:09 

Next, we need to transfer the file to the remote host. Because you might be generating different keys for different hosts, it's most convenient if you rename the file first this also helps prevent you from overwriting it the next time you create a key or overwriting the key on the remote host when you transfer it. You might also want to consider using the -f option to specify a different filename when you generate your public key. However, we wanted to show you what to expect by default. Because it's your public key, it doesn't matter whether the world can see it you can copy it to your remote host via FTP, move it there with a floppy, or paste it across a logged-in terminal session.

On the remote host, the public key you just created needs to be added to the file authorized_keys (~/.ssh/authorized_keys) in the .ssh directory in your home directory (~/.ssh/). If the file does not exist, it must be created. If you copied the key over in a file, you can do this by simply using the cat command:

 cat <mynewkeyfile> >> ~/.ssh/authorized_keys 

When adding the new key to the file, make sure that the key is added as a single long line of data. If your key arrived in one long line of data in a file, the cat command shown will work fine. Otherwise, if you're pasting the key in via the terminal or aren't sure it's in a single long line in the file, it's best to check ~/.ssh/authorized_keys to make sure that it arrived correctly.


Many terminals will be friendly and line-wrap the key, if you try to paste it through a logged-in terminal window. If your passphrase refuses to work, make sure that there are no extra blank lines in your ~/.ssh/authorized_keys file, that the key is on a line by itself (rather than attached to the backside of another key), and that the key hasn't accidentally accumulated any line breaks.

Having done all this, if you now try to slogin to the remote host where you just added your key (and assuming that the remote host is running sshd2!), you should be greeted with a login process asking for your passphrase rather than your password. Enter the passphrase exactly as you did to create the keys, and you will enjoy a data connection that is almost impossible to decrypt, and an access code (your passphrase) that is much more secure than a simple password.


There are a number of variations on the movement of the public key and its installation on the remote host. These revolve around the version of server running on the remote machine. Considerably more detail and examples of these options are given in Chapter 21, where we cover getting these outside machines to talk to your Mac OS X box.

The slogin program also provides a neat method for protecting data transmissions other than terminals. This is implemented as an encrypted tunnel between the two machines connected by the slogin terminal connection. Essentially, slogin can be instructed to watch for connections that come to your local machine, package the data from these connections, encrypt it, ship it off to the other end of the tunnel, and unpackage it again. You then use your ftp, or any other network connection program, to connect to your local machine (not the remote machine!), and slogin tunnels that connection to the remote machine and makes the connection at the other end. Because your user ID and password for the FTP server are carried over the encrypted tunnel, they're never in clear text on the network, and your login information and any data you transmit are protected.

To demonstrate this, the following slogin connection sets up a tunnel from the local machine to a remote machine named waashu, over which ftp connections can be carried.

brezup:root testing # slogin testing -L21:waashu:21 The authenticity of host ' (' can't be established. DSA key fingerprint is 3d:1d:6b:78:c9:7e:63:b9:8b:6d:13:5f:e5:3b:f1:20. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added ',' (DSA) to the list of known hosts.'s password: Last login: Tue Jul 15 2003 15:37:15 You have new mail. /usr/local/testing WAASHU testing 1 >

In this case, we've never connected waashu before, so slogin asks whether we really believe that we're making a connection to the right host and that it's really giving us valid credentials. This is the one point in all our communications where an imposter in the middle of the communication could easily insert false information and fool us into transmitting our information insecurely. Again, this leaves the terminal connected to the remote machine and sitting at a shell prompt on the remote machine. The-L21:waashu:21 part of the command sets up the tunneling magic. It tells slogin to start listening on port 21 (which is the port that the FTP server would usually listen to), capture anything it sees, package it securely, and transmit it to waashu, where it is to be unpackaged and sent to waashu's port 21 (thereby connecting to waashu's FTP server).


Note that only the root user can map to ports numbered lower than 1024. For this reason, the slogin forwarding as shown here isn't quite what you want to do for day-to-day use. It's the easiest for basic illustration, though a more practical example comes a little later.

When slogin is connected like this, it is connecting port 21 the normal ftp port on our machine (localhost) to port 21 on the remote host we're logged in to. Open another terminal window. The second terminal window is used to invoke ftp to connect over the tunnel (by connecting to our local machine, usually available as localhost and always available as like so:

 brezup:miwa miwa $ ftp localhost Connected to 220 FTP server ready. Name (localhost:miwa): testing 331 Password required for testing. Password: 230 User testing logged in. Remote system type is UNIX. Using binary mode to transfer files. ftp> passive Passive mode on. ftp> cd osx-misc 250 CWD command successful. ftp> binary 200 Type set to I. ftp> put developer-1.tiff local: developer-1.tiff remote: developer-1.tiff 227 Entering Passive Mode (140,254,12,239,60,59) 150 Opening BINARY mode data connection for 'developer-1.tiff'. 226 Transfer complete. 1255376 bytes sent in 16.2 seconds (77490 bytes/s) ftp> quit 221 Goodbye. 

To check whether it arrived okay, we go to the waashu terminal:

 WAASHU osx-misc 203 > ls -l dev*tiff -rw-r--r--    1 testing    user    1255376 Apr 21 20:35 developer-1.tiff 

Note that when we ftp to localhost, ftp reports that we're connected to localhost, but waashu responds. The tunnel is working as expected.

As noted earlier, use of port 21 is restricted to the root user, but for your first introduction, it made sense to direct the ftp port to the ftp port. There is nothing that limits the forwarding to connecting identically numbered ports, though, and ftp can also connect to ports other than the usual port 21. For use on a day-to-day basis, a normal user can replace the -L21:<machinename>:21 section of the command with-L2000:<machinename>:21. The ftp command then is extended by adding the port number for the local connection as ftp localhost 2000. This probably sounds more complicated than it really is. It really doesn't look much different than just directing the ftp port to the ftp port. In one window, run this:

 brezup:miwa Documents $ slogin -l testing -L2000:waashu:21 

And in another, run the ftp command as like so:

 brezup:miwa Documents $ ftp localhost 2000 

This works identically to having root route the tunnel as shown in the first example.

If your machine doesn't know the target by a short name (such as <waashu>), you need to use an IP address or fully qualified hostname for the -L<sourceport>:<target hostname>:<targetport> part of the command as well as the base slogin itself.

Another option, if all you want to do is forward a port without receiving a shell prompt on the remote host, is using the -N option to slogin. This doesn't cause it to return to the command line but is useful in stored terminal scripts if you're not interested in leaving a prompt open and unused (which is usually a good idea for security purposes).

Table 13.4 shows the syntax and additional options for the operation of slogin.

Table 13.4. The Syntax and Some Interesting Options for ssh and slogin


Secure shell remote login client.



ssh [-l <login_name>] <hostname> | <user>@<hostname> [<command>]

ssh [-1246AaCfgkNnqsTtVvXxY] [-b <bind_address>] [-c <cipher_spec>] [-e <escap_char>] [-F <configfile>] [-i <identity_file>] [-L <port>:<host>:<hostport>] [-l <login_name>] [-m <mac_spec>] [-o <option>] [-p <port>]:<hostport>] [-R <port>:<host>] [[<user>@]<hostname>] [<command>]]


Disables forwarding of the authentication agent connection.


Enables forwarding of the authentication agent connection. This can also be specified on a per-host basis in a configuration file.


Requests ssh to go to background just before command execution. Implies -n. The recommended way to start X11 programs at a remote site is ssh -f <host> xterm.


Verbose mode. Causes debugging messages to be printed.


Disables X11 forwarding.


Enables X11 forwarding. This can also be specified on a per-host basis in a configuration file.


Requests compression of all data.


Does not execute a remote command. Useful for just forwarding ports. SSH2 only.


Uses a nonprivileged port for outgoing connections. Useful if your firewall does not permit connections from privileged ports. Turns off RhostsAuthentication and RhostsRSAAuthentication.


Forces SSH1 protocol only.


Forces SSH2 protocol only.

-e ch|^ch|none

Sets escape character for sessions with a pty (default: ~). The escape character is recognized only at the beginning of a line. Followed by a . closes the connection; followed by ^Z suspends the connection; followed by itself sends the escape character once. Setting it to none disables any escapes and makes the session fully transparent. You might want to set this to something other than the default if you find that you're using mail and the ~ command in mail keeps being absorbed by the slogin client.

-i <identity_file>

Specifies the file from which the identity (private key) for RSA authentication is read. Default is $HOME/.ssh/identity.

-l <login_name>

Specifies the user to log in as on the remote machine. This may also be specified on a per-host basis in a configuration file.

-o <option>

Can be used for giving options in the format used in the configuration file. Useful for specifying options that have no separate command-line flag. Option has the same format as a line in the configuration file.

-p <port>

Specifies the port to connect to on the remote host. This can be specified on a per-host basis in the configuration file.

-F <configfile>

Specifies an alternative per-user configuration file. If a configuration file is given on the command line, the systemwide configuration file (/etc/ssh_config) is ignored. Default per-user configuration file is $HOME/.ssh/config.

-L <port>:<host>: <hostport>

Specifies that the given port on the client (local) host is to be forwarded to the given host and port on the remote side.

-R <port>:<host>:<hostport>

Specifies that the given port on the remote (server) host is to be forwarded to the given host and port on the local side.

scp, sftp, and Others In addition to the slogin program, the Secure Shell suite of programs provides additional data encryption and protection functions to the user. There are components (scp) that function analogously to the cp command that you learned about in Chapter 10, "Common Unix Shell Commands: File, Directory and Disk Operations," and to the ftp command that you learned about earlier in this chapter (sftp).

The scp command can copy a file either from or to a Secure Shell remote host. The syntax, like cp, is scp <from> <to>. Either <from> or <to> can be specified as a remote machine and file, in the syntax of [<username>@]<remotemachine>:<pathtofile>. For example, the following command copies ~ray/public_html/my_bookmarks.html from the machine soyokaze (soyokaze is a host alias to on this machine) to a file by the same name in the local folder ~/Documents/:

 brezup:ray testing $ scp ray@soyokaze:public_html/my_bookmarks.html ~/Documents/'s password: my_bookmarks.html                            100%  271KB  45.4KB/s   00:05 

Likewise, the following copies the file myfile from the current directory to the directory /tmp on the machine known as soyokaze (again, you will need a long name here if your local machine doesn't know the target machine by a short alias) and names it yourfile on the remote machine soyokaze, again logging in using the user ID ray:

 brezup:ray testing $ scp ./myfile's password: myfile                                    37%  208KB  20.6KB/s   00:17 ETA 

Note that scp doesn't make complaints about the host key the second time because it has already accepted and stored it.

Table 13.5 shows the syntax and interesting options for scp.

Table 13.5. The Syntax and Interesting Options for scp


Secure remote copy.

scp [-1246BCpqrv] [-c <cipher>] [-F <ssh_config>] [-i <identity_file>] [-l <limit>] [- o <ssh_option>] [-P <port>] [-S <program>] [[<user>@]<host1>:]<file1> [...] [[<user>@]<host2>:]<file2>


Preserves modification times, access times, and modes from the original file.


Recursively copies entire directories.


Enables compression. Passes the flag to ssh to enable compression.

-F <ssh_config>

Specifies an alternative per-user configuration file for ssh. Option is directly passed to ssh.

-P <port>

Specifies the port to connect to on the remote host.

-i <identity_file>

Specifies the file from which the identity (private key) for RSA authentication is read.

-o <ssh_option>

Passes specified options to ssh in the format used in ssh_config.

The sftp command can also be used to securely transfer files. It was not available in the original Mac OS X 10.0 distribution but was included in a later update.

The basic syntax for using sftp is

 sftp [<username>@]<host> 

This syntax opens an interactive sftp session, which works much like a typical interactive ftp session, as shown here:

 brezup:sage Documents $ sftp Connecting to's password: sftp> lcd terminal sftp> cd terminal-misc sftp> put term-display-1.tiff Uploading term-display-1.tiff to /home/miwa/terminal-misc/term-display-1.tiff sftp> ls drwxr-xr-x   2 miwa     class         512 Aug  6 20:56 ./ drwxr-xr-x  21 miwa     class        1024 Aug  6 20:53 ../ -rw-r--r--   1 miwa     class      921862 Aug  6 20:57 term-display-1.tiff sftp> quit 

In this example, an interactive sftp session was used by user sage to transfer the file term-display-1.tiff to user miwa's terminal-misc directory on the remote host The lcd command was used to change to sage's terminal directory on the local machine, brezup, and cd was used on the remote host to change to miwa's terminal-misc directory. Of course, the sftp command could have been issued directly in sage's terminal directory. Like an interactive ftp session, the interactive sftp session can take commands such as cd, ls, and put. As is the case with scp, if you have the same username on both machines, it is not necessary to supply a <username> because the current username is assumed by default.

Table 13.6 shows the syntax and some of the useful options for sftp.

Table 13.6. The Syntax and Some Interesting Options for sftp


Secure file transfer program.

sftp [-1Cv] [-b <batchfile>] [-o <ssh_option>] [-s <subsystem> | <sftp_server>] [-B <buffer_size>] [-F <ssh_config>] [-P <sftp_server path>] [-R <num_requests>] [-S <program>] <host>

sftp [[<user>@]<host>[:<file1> [<file2>]]]

sftp [[<user>@]<host>[:<dir>[/]]]

The first usage initiates an interactive session.


The second usage retrieves files automatically if a noninteractive authentication is used. Otherwise, it retrieves the specified files after interactive authentication.

The third usage causes sftp to start in an interactive session in the specified directory.

-b <batchfile>

Batch mode. Reads a series of commands from an input batch file instead of stdin. Because it lacks user interaction, it should be used in conjunction with noninteractive authentication. sftp aborts if any of the following commands fails: get, put, rename, ln, rm, mkdir, chdir, lchdir, and lmkdir.

-o <ssh_option>

Passes options to ssh in the format used in the ssh configuration file. Useful for specifying options for which there is no separate sftp command-line flag. For example, to specify an alternative port use: sftp -oPort=24.


Enables compression via ssh's -C flag.

-F <ssh_config>

Specifies an alternative per-user configuration file for ssh. Option is passed directly to ssh.


Specifies the use of protocol version 1.

Interactive Commands


cd <path>

Changes remote directory to <path>.

lcd <path>

Changes local directory to <path>.

chgrp <grp> <path>

Changes group of file <path> to <grp>.

chmod <mode> <path>

Changes permissions of file <path> to <mode>.

chown <owner> <path>

Changes owner of file <path> to <owner>.

get [<flags>] <remote-path> [<local-path>]

Retrieves the <remote-path> and stores it on the local machine. If the local pathname is not specified, it is given the same name it has on the remote machine. If the -P flag is specified, the file's full permission and access time are copied as well.


Displays help text.

lls [<ls-options> [<path>]]

Displays local directory listing of either <path> or current directory if <path> is not specified.

lmkdir <path>

Creates local directory specified by <path>.

ln <oldpath> <newpath>

Creates a symbolic link from <oldpath> to <newpath>.


Prints local working directory.

ls [<path>]

Displays remote directory listing of either <path> or current directory if <path> is not specified.

mkdir <path>

Creates remote directory specified by <path>.

put [<flags>] <local-path> [<remote-path>]

Uploads <local-path> and stores it on the remote machine. If the remote pathname is not specified, it is given the same name it has on the local machine. If the -P flag is specified, the file's full permission and access time are copied as well.


Displays remote working directory.


Quits sftp.

rename <oldpath> <newpath>

Renames remote file from <oldpath> to <newpath>.

rmdir <path>

Removes remote directory specified by <path>.

rm <path>

Deletes remote file specified by <path>.

symlink <oldpath> <newpath>

Creates a symbolic link from <oldpath> to <newpath>.

! <command>

Executes command in local shell.

The "Busload of Useful Tricks" Network Client: cURL

cURL, more commonly known simply as curl, is a command-line tool for getting or sending data to network services using URL syntax. The name is a bit of a play on words, being pronounced either as one word as in "kurl," or as two words as in "see URL" (implying the unspoken "do URL" to those with a Unix sense of humor). curl is based on the libcurl library, which has the goal of bringing convenient URL-type data access and transfers to software that needs it.

Philosophically, curl is a very Unix-friendly program, providing a very specific function, while trying not to overlap the functionality of other programs. We are including it among these other more application-like programs because curl makes an excellent assistant program for almost any software that needs network access. As such, it might not fit our definition of an application or application suite, but it does integrate well as a network-access partner for other applications and application suites.

At its simplest, curl syntax is curl [options] <URL>. The complexity and power come from the range of available options. For example, to retrieve the web page using curl, the syntax is simply


The output of this is identical to using lynx -dump -source curl, however, can grab the name of the remote file from the remote server and write the data into a file by that name locally without you having to do a redirect as you would with lynx. Therefore, you could use

 curl -O 

instead of

 lynx -dump -source  > index.html 

curl isn't a replacement for lynx because it isn't a web browser, but, on the other hand, curl is bidirectional and can send data as well as receive it. The -T <file> option directs that the local file named <file> be sent to the remote machine and file or directory name as specified in <URL>.

 brezup:ray Unleashed $ curl -T fig18_.gif   % Total    % Received % Xferd  Average Speed          Time             Curr.                                  Dload  Upload Total    Current  Left    Speed  52  763k    0     0   52  402k      0  27744  0:00:28  0:00:14  0:00:14 34526 

This command results in fig18_.gif being ftped from the current directory to the machine located at and placed in the incoming subdirectory of the ftp directory on that machine. Speed and progress statistics are displayed as the file is transmitted.

These are the variations on curl that most people will use, most frequently, but the range of available options is truly diverse. Table 13.7 shows the syntax and some of the more useful options for curl, including hints on how to access secure servers, track cookie contents, and other useful trivia.

Table 13.7. The Syntax and Some Interesting Options for curl


A utility for getting a URL with FTP, TELNET, LDAP, GOPHER, DICT, FILE, HTTP or HTTPS syntax.

curl [options] [<URL>...]


-a --append

(FTP) When used in an ftp upload, this tells curl to append to the target file instead of overwriting it. If the file doesn't exist, it is created.

-A <agent string> --user-agent <agent string>

(HTTP) Specifies the User-Agent string to send to the HTTPserver. Some badly done CGIs fail if it's not set to "Mozilla/4.0". To encode blanks in the string, surround the string with single quote marks. This can also be set with the -H/--header flag.

-b <name=data> --cookie <name=data>

(HTTP) Passes the data to the HTTP server as a cookie. The datais supposedly the data previously received from the server in a Set-Cookie: line. The data should be in the format NAME1=VALUE1; NAME2=VALUE2, but there's nothing to say you can't change it.


If no = is used in the line, it is treated as a filename to use to read previously stored cookie lines from, which should be used in this session if they match. Using this method also activates the cookie parser, which makes curl record incoming cookies too; that can be handy for using this in combination with the -L/--location option. The file format of the file to read cookies from should be plain HTTP headers or the Netscape cookie file format.


Note that the file specified with -b/--cookie is only used as input. No cookies are stored in the file. To store cookies, save the HTTP headers to a file using -D/--dump-header.

-B --use-ascii

Uses ASCII transfer when getting an FTP file or LDAP info. For FTP, this can also be enforced by using a URL that ends with ;type=A.

--connect-timeout <seconds>

Specifies how long to wait for a server before giving up.

-c --cookie-jar <file name>

Specifies a file where curl should store any cookies it receives. If you set the <file name> to a single - (dash), the cookies are written to STDOUT.

-C <offset>

Continues/resumes a previous file transfer at the given offset.

--continue-at <offset>

The given offset is the exact number of bytes skipped, counted from the beginning of the source file before it is transferred to the destination. If used with uploads, the ftp server command SIZE is not used by curl. Upload resume is for FTP only. HTTP resume is possible only with HTTP/1.1 or later servers.


Causes CR (carriage return) characters to be converted to CRLF (carriage return/line feed) on upload. Specifying --crlf twice forces this option off.

-d <data> --data <data>

(HTTP) Sends the specified data in a POST request to the HTTP server (can be changed to GET by the -G/--get option), in a way that can emulate as if a user has filled in an HTML form and clicked the Submit button. Note that the data is sent exactly as specified with no extra processing (with all newlines cut off). The data is expected to be url-encoded. This causes curl to pass the data to the server using the content-type application/x-www-form-urlencoded. Compare to -F. If more than one -d/--data option is used on the same command line, the data pieces specified are merged together with a separating &-letter. Thus, using-d name=daniel -d skill=lousy generates a post chunk that looks like name=daniel&skill=lousy.

If this option is used several times, the ones following the first append data.

--data-binary <data>

(HTTP) Posts data in a manner similar to --data-ascii, although when using this option the entire context of the posted data is kept as-is. If you want to post a binary file without the strip-newlines feature of the --data-ascii option, this is for you.

-D <file> --dump-header <file>

(HTTP/FTP) Writes the HTTP headers to this <file>. Writes the FTP file info to this <file> if -I/--head is used.


Handy for storing the cookies that an HTTP site sends to you. The cookies could then be read in a second curl invoke by using the -b/--cookie option.

-e <URL> --referer <URL>

(HTTP) Sends the referer page information to the HTTP server. This can also be set with the -H/--header flag. When used with -L/--location you can append ;auto to the referer URL to make curl automatically set the previous URL when it follows a Location: header. The ;auto string can be used alone, even if you don't set an initial referer. This option lets you lie to servers about the page that directed you to a particular link, potentially bypassing "deep linking" safeguards.

-F --form <name=content>

(HTTP) This lets curl emulate a filled-in form in which a user has clicked the Submit button. This causes curl to POST data using the content-type multipart/form-data according to RFC 1867. This enables uploading of binary files and so on. To force the content part to be a file, prefix the filename with an @ sign. To just get the content part from a file, prefix the filename with the character <. The difference between @ and < is that @ makes a file get attached in the post as a file upload, whereas < makes a text field and just gets the contents for that text field from a file.


Use the HTTP GET protocol rather than the POST protocol for sending data.

-h --help

Displays help.

-H --header <header>

(HTTP) Extra header to use when getting a web page. You may specify any number of extra headers. Note that if you should add a custom header that has the same name as one of the internal ones curl would use, your externally set header will be used instead of the internal one. This allows you to make even trickier stuff than curl would normally do. Do not replace internally set headers without knowing perfectly well what you're doing. Replacing an internal header with one without content on the right side of the colon prevents that header from appearing.

-I --include

(HTTP) Includes the HTTP-header in the output. The HTTP-header includes things such as server name, date of the document, HTTP version, and more.

-j --junk-session-cookies

Discard session cookies from any cookie data read from a cookie file. This has the effect of starting a new session. Browsers typically discard session cookies when they quit, but it's useful to keep them around until you want to discard them with curl because it's a single-command, single-connection program.

-k --insecure

Allows curl to perform insecure connections. If this option is not specified, connections to servers with apparently incorrect security certificates will be barred.

-K --config <config file>

Specifies which <config file> to read curl arguments from. The <config file> is a text file in which command-line arguments can be written that then will be used as if they were written on the actual command line. Options and their parameters must be specified on the same line in the file. If the parameter is to contain whitespace, the parameter must be enclosed within quotes. If the first column of a config line is a # character, the rest of the line will be treated as a comment.

Specify the filename as - to make curl read the file from stdin.

-l --list-only

(FTP) When listing an FTP directory, this switch forces a name-only view. This is especially useful if you want to machine-parse the contents of an FTP directory because the normal directory view doesn't use a standard look or format.

--limit-rate <speed>

Throttles the data connection to a maximum of <speed> bytes per second. The transfer speed may also be given in <speed>K to specify kilobytes/second or <speed>M to specify megabytes/second. If you're really optimistic, <speed>G asks for gigabytes/second transfer rates.

-L --location

(HTTP/HTTPS) If the server reports that the requested page has a different location (indicated with the header line Location:), this flag instructs curl to reattempt the get on the new location. If used together with -i or -I, headers from all requested pages are shown. If this flag is used when making an HTTP POST, curl automatically switches to GET after the initial POST is done.

-m --max-time <seconds>

Specifies the maximum time in seconds that you allow the whole operation to take. This is useful for preventing your batch jobs from hanging for hours due to slow networks or links going down. See also the --connect-timeout option.

-M --manual

Manual. Displays the curl man page.

-n --netrc

Makes curl scan the .netrc file in the user's home directory for login name and password. This is typically used for ftp on Unix. If used with http, curl enables user authentication. See neTRc(4) for details on the file format. curl does not complain if that file hasn't the right permissions (it should not be world nor group readable). The environment variable HOME is used to find the home directory.

The basic .neTRc file syntax looks like this:

machine <> login <myname> password <mysecret>

-o --output <file>

Writes output to <file> rather than to stdout. If you are using {} or [] to fetch multiple documents, you can use # followed by a number in the <file> specifier. That variable is replaced with the current string for the URL being fetched.

-O --remote-name

Writes output to a local file named like the remote file we get. (Only the file part of the remote file is used; the path is cut off.)

You may use this option as many times as you have number of URLs.


If used as the first parameter on the command line, the $HOME/.curlrc file will not be read and used as a config file.

-R --remote-time

Attempts to determine the timestamp on the remote file and use that on the local copy.

-s --silent

Silent mode. Doesn't show progress meter or error messages.

-S --show-error

When used with -s, it makes curl show error messages if it fails.

-T --upload-file <file>

Transfers the specified local <file> to the remote server at <URL>. If there is no file part in the specified URL, curl appends the local filename. Note that you must use a trailing / on the last directory to really prove to curl that you aren't providing a filename, or curl thinks that your last directory name is the remote filename to use. That will most likely cause the upload operation to fail. If this is used on an HTTP(S) server, the PUT command is used.

--TRace <file>

Outputs a full diagnostic trace of all data exchanged between the local and remote hosts. A single - (dash) as <file> sends the output to STDOUT.

--trace-ascii <file>

Outputs a diagnostic trace of all information in ASCII format. curl claims this is easier for humans to read.

-u --user <user:password>

Specifies user and password to use when fetching. See README.curl for detailed examples of how to use this. If no password is specified, curl asks for it interactively.

-x --proxy <proxyhost[:port]>

Uses specified proxy. If the port number is not specified, it is assumed at port 1080.

-y --speed-time <time>

If a download is slower than speed-limit bytes per second during a speed-time period, the download will be aborted. If speed-time is used, the default speed-limit will be 1 unless set with -y.

-Y --speed-limit <speed>

Applies a lower limit to the download speed. If a download is slower than this given speed, in bytes per second, for speed-time seconds, it will be aborted. speed-time is set with -Y and defaults to 30 if not set.

-Z --max-redirs <num>

Sets the maximum number of server redirections to follow before giving up.

-3 --sslv3

(HTTPS) Forces curl to use SSL version 3 when negotiating with a remote SSL server.

-2 --sslv2

(HTTPS) Forces curl to use SSL version 2 when negotiating with a remote SSL server.

-# --progress-bar

Displays progress information as a progress bar instead of thedefault statistics.

     < Day Day Up > 

    Mac OS X Tiger Unleashed
    Mac OS X Tiger Unleashed
    ISBN: 0672327465
    EAN: 2147483647
    Year: 2005
    Pages: 251 © 2008-2017.
    If you may any questions please contact us: