Section 16.3. Running Server-Side Examples

16.3. Running Server-Side Examples

Like GUIs, web-based systems are highly interactive, and the best way to get a feel for some of these examples is to test-drive them live. Before we get into some code, let's get set up to run the examples we're going to see.

Running CGI-based programs requires three pieces of software:

The client, to submit requests: a browser or script
The web server that receives the request
The CGI script, which is run by the server to process the request

We'll be writing CGI scripts as we move along, and any web browser can be used as a client (e.g., Firefox or Internet Explorer). As we'll see later, Python's urllib module can also serve as a web client in scripts we write. The only missing piece here is the intermediate web server.

16.3.1. Web Server Options

There are a variety of approaches to running web servers. For example, the open source Apache system provides a complete, production-grade web server, and its mod_python extension discussed later runs Python scripts quickly. Provided you are willing to install and configure, it is a complete solution, which you can run on a machine of your own. Apache usage is beyond our present scope here, though.

If you have access to an account on a web server machine that runs Python, you can also install the HTML and script files we'll see there. For the second edition of this book, for instance, all the web examples were uploaded to an account I had on the "starship" Python server, and were accessed with URLs of this form:

 http://starship.python.net/~lutz/PyInternetDemos.html

If you go this route, replace starship.python.net/~lutz with the names of your own server and account directory path. The downside of using a remote server account is that changing code is more involvedyou will have to either work on the server machine itself, or transfer code back and forth on changes. Moreover, you need access to such a server in the first place, and server configuration details can vary widely. On the starship machine, for example, Python CGI scripts were required to have a .cgi filename extension, executable permission, and the Unix #! line at the top to point the shell to Python.

16.3.2. Running a Local Web Server

To keep things simple, this edition is taking a different approach. All the examples will be run using a simple web server coded in Python itself. Moreover, the web server will be run on the same, local machine as the web browser client. This way, all you have to do to run the server-side examples is start the web server script, and use "localhost" as the server name in all the URLs you will submit or code (see Chapter 13 if you've forgotten why this name means the local machine). For example, to view a web page, use a URL of this form in the address field of your web browser:

 http://localhost/tutor0.html

This also avoids some of the complexity of per-server differences, and it makes changing the code simpleit can be edited on the local machine directly.

For this book's examples, we'll use the web server in Example 16-1. This is essentially the same script introduced in Chapter 2, augmented slightly to allow the working directory and port number to be passed in as command-line arguments (we'll also run this in the root directory of a larger example in the next chapter).

More subtly, here we also add the cgi-bin subdirectory to the module import search path, to mimic the current working directory when scripts are run independently on other platforms (on Windows, they are run in the same process as this server, and the server classes do not change to the CGI's directory). This last point can make a difference if a CGI script must import an application's modules from its own directoryits container directory must be on the module search path, regardless of how it was started.

We won't go into details on all the modules and classes Example 16-1 uses here; see the Python library manual. But as described in Chapter 2, this script implements an HTTP web server, which:

Listens for incoming socket requests from clients on the machine it is run on, and the port number specified in the script or command line (which defaults to 80, that standard HTTP port)
Serves up HTML pages from the webdir directory specified in the script or command line (which defaults to the directory it is launched from)
Runs Python CGI scripts that are located in the cgi-bin (or htbin) subdirectory of the webdir directory, with a .py filename extension

See Chapter 2 for additional background on this web server's operation.

Example 16-1. PP3E\Internet\Web\webserver.py

 ######################################################################### # implement HTTP web server in Python which knows how to serve HTML # pages and run server-side CGI scripts;  serves files/scripts from # the current working dir and port 80, unless command-line args; # Python scripts must be stored in webdir\cgi-bin or webdir\htbin; # more than one of these may be running on the same machine to serve # from different directories, as long as they listen on different ports; ######################################################################### webdir = '.'   # where your HTML files and cgi-bin script directory live port   = 80    # http://servername/ if 80, else use http://servername:xxxx/ import os, sys from BaseHTTPServer import HTTPServer from CGIHTTPServer  import CGIHTTPRequestHandler if len(sys.argv) > 1: webdir = sys.argv[1]             # command-line args if len(sys.argv) > 2: port   = int(sys.argv[2])        # else default ., 80 print 'webdir "%s", port %s' % (webdir, port) # hack for Windows: os.environ not propagated # to subprocess by os.popen2, force in-process if sys.platform[:3] == 'win':     CGIHTTPRequestHandler.have_popen2 = False     CGIHTTPRequestHandler.have_popen3 = False          # emulate path after fork     sys.path.append('cgi-bin')                         # else only adds my dir os.chdir(webdir)                                       # run in HTML root dir srvraddr = ("", port)                                  # my hostname, portnumber srvrobj  = HTTPServer(srvraddr, CGIHTTPRequestHandler) srvrobj.serve_forever( )                                # serve clients till exit

To start the server to run this chapter's examples, simply run this script from the directory the script's file is located in, with no command-line arguments. For instance, from a DOS command line:

 C:\...\PP3E\Internet\Web>webserver.py webdir ".", port 80

On Windows, you can simply click its icon and keep the console window open, or launch it from a DOS command prompt. On Unix it can be run from a command line in the background, or in its own terminal window.

By default, while running locally this way, the script serves up HTML pages requested on "localhost" from the directory it lives in or is launched from, and runs Python CGI scripts from the cgi-bin subdirectory located there; change its webdir variable or pass in a command-line argument to point it to a different directory. Because of this structure, in the examples distribution HTML files are in the same directory as the web server script, and CGI scripts are located in the cgi-bin subdirectory. In other words, to visit web pages and run scripts, we'll be using URLs of these forms, respectively:

 http://localhost/somepage.html http://localhost/cgi-bin/somescript.py

Both map to the directory that contains the web server script (PP3E\Internet\Web) by default. Again, to run the examples on a different server of your own, simply replace the "localhost" and "localhost/cgi-bin" parts of these addresses with your server and directory path details (more on URLs later in this chapter).

The server in Example 16-1 is by no means a production-grade web server, but it can be used to experiment with this book's examples and is viable as way to test your CGI scripts locally before deploying them on a real remote server. If you wish to install and run the examples under a different web server, you'll want to extrapolate the examples for your context. Things like server names and pathnames in URLs, as well as CGI script filename extensions and other conventions, can vary widely; consult your server's documentation for more details. For this chapter and the next, we'll assume that you have the webserver.py script running locally.

16.3.3. The Server-Side Examples Root Page

To confirm that you are set up to run the examples, start the web server script in Example 16-1 and type the following URL in the address field at the top of your web browser:

 http://localhost/PyInternetDemos.html

This address loads a launcher page with links to this chapter's example files (see the examples distribution for this page's HTML source code). The launcher page itself appears as in Figure 16-1, shown running under the open source Firefox web browser (it looks similar in other browsers). Each major example has a link on this page, which runs when clicked.

Figure 16-1. The PyInternetDemos launcher page

It's possible to open some of the examples by clicking on their HTML file directly in your system's file explorer GUI. However, the CGI scripts ultimately invoked by some of the example links must be run by a web server. If you browse such pages directly, your browser will likely display the scripts' source code, instead of running it. To run scripts too, be sure to open the HTML pages by typing their "localhost" URL address into your browser's address field.

Eventually, you probably will want to start using a more powerful web server, so we will study additional CGI installation details later in this chapter, and explore a few custom server options at the end of Chapter 18. Such details can be safely skipped or skimmed if you will not be installing on another server right away. For now, we'll run locally.

16.3.4. Viewing Server-Side Examples and Output

The source code of examples in this part of the book is listed in the text and included in the book's examples distribution. In all cases, if you wish to view the source code of an HTML file, or the HTML generated by a Python CGI script, you can also simply select your browser's View Source menu option while the corresponding web page is displayed.

Keep in mind, though, that your browser's View Source option lets you see the output of a server-side script after it has run, but not the source code of the script itself. There is no automatic way to view the Python source code of the CGI scripts themselves, short of finding them in this book or in its examples distribution.

To address this issue, later in this chapter we'll also write a CGI-based program called getfile, which allows the source code of any file on this book's web site (HTML, CGI script, and so on) to be downloaded and viewed. Simply type the desired file's name into a web page form referenced by the getfile.html link on the Internet demos launcher page, or add it to the end of an explicitly typed URL as a parameter like the following; replace tutor5.py at the end with the name of script whose code you wish to view:

 http://localhost/cgi-bin/getfile.py?filename=cgi-bin/tutor5.py

In response, the server will ship back the text of the named file to your browser. This process requires explicit interface steps, though, and much more knowledge of URLs than we've gained thus far; to learn how and why this magic line works, let's move on to the next section.