Section 13.4. Handling Multiple Clients

13.4. Handling Multiple Clients

The echo client and server programs shown previously serve to illustrate socket fundamentals. But the server model suffers from a fairly major flaw: if multiple clients try to connect to the server, and it takes a long time to process a given client's request, the server will fail. More accurately, if the cost of handling a given request prevents the server from returning to the code that checks for new clients in a timely manner, it won't be able to keep up with all the requests, and some clients will eventually be denied connections.

In real-world client/server programs, it's far more typical to code a server so as to avoid blocking new requests while handling a current client's request. Perhaps the easiest way to do so is to service each client's request in parallelin a new process, in a new thread, or by manually switching (multiplexing) between clients in an event loop. This isn't a socket issue per se, and we already learned how to start processes and threads in Chapter 5. But since these schemes are so typical of socket server programming, let's explore all three ways to handle client requests in parallel here.

13.4.1. Forking Servers

The script in Example 13-4 works like the original echo server, but instead forks a new process to handle each new client connection. Because the handleClient function runs in a new process, the dispatcher function can immediately resume its main loop in order to detect and service a new incoming request.

Example 13-4. PP3E\Internet\Sockets\fork-server.py

 ############################################################################# # Server side: open a socket on a port, listen for a message from a client, # and send an echo reply; forks a process to handle each client connection; # child processes share parent's socket descriptors; fork is less portable # than threads--not yet on Windows, unless Cygwin or similar installed; ############################################################################# import os, time, sys from socket import *                      # get socket constructor and constants myHost = ''                               # server machine, '' means local host myPort = 50007                            # listen on a non-reserved port number sockobj = socket(AF_INET, SOCK_STREAM)           # make a TCP socket object sockobj.bind((myHost, myPort))                   # bind it to server port number sockobj.listen(5)                                # allow 5 pending connects def now( ):                                       # current time on server     return time.ctime(time.time( )) activeChildren = [] def reapChildren( ):                              # reap any dead child processes     while activeChildren:                          # else may fill up system table         pid,stat = os.waitpid(0, os.WNOHANG)       # don't hang if no child exited         if not pid: break         activeChildren.remove(pid) def handleClient(connection):                     # child process: reply, exit     time.sleep(5)                                # simulate a blocking activity     while True:                                  # read, write a client socket         data = connection.recv(1024)             # till eof when socket closed         if not data: break         connection.send('Echo=>%s at %s' % (data, now( )))     connection.close( )     os._exit(0) def dispatcher( ):                                # listen until process killed     while True:                                   # wait for next connection,         connection, address = sockobj.accept( )   # pass to process for service         print 'Server connected by', address,         print 'at', now( )         reapChildren( )                           # clean up exited children now         childPid = os.fork( )                     # copy this process         if childPid == 0:                         # if in child process: handle             handleClient(connection)         else:                                    # else: go accept next connect             activeChildren.append(childPid)      # add to active child pid list dispatcher( )

13.4.1.1. Running the forking server

Parts of this script are a bit tricky, and most of its library calls work only on Unix-like platforms (not Windows). But before we get into too many details, let's start up our server and handle a few client requests. First, notice that to simulate a long-running operation (e.g., database updates, other network traffic), this server adds a five-second time.sleep delay in its client handler function, handleClient. After the delay, the original echo reply action is performed. That means that when we run a server and clients this time, clients won't receive the echo reply until five seconds after they've sent their requests to the server.

To help keep track of requests and replies, the server prints its system time each time a client connect request is received, and adds its system time to the reply. Clients print the reply time sent back from the server, not their ownclocks on the server and client may differ radically, so to compare apples to apples, all times are server times. Because of the simulated delays, we also must usually start each client in its own console window on Windows (on some platforms, clients will hang in a blocked state while waiting for their reply).

But the grander story here is that this script runs one main parent process on the server machine, which does nothing but watch for connections (in dispatcher), plus one child process per active client connection, running in parallel with both the main parent process and the other client processes (in handleClient). In principle, the server can handle any number of clients without bogging down. To test, let's start the server remotely in a Telnet window, and start three clients locally in three distinct console windows:

 [server telnet window] [lutz@starship uploads]$ uname -a  Linux starship ... [lutz@starship uploads]$ python fork-server.py  Server connected by ('38.28.162.194', 1063) at Sun Jun 18 19:37:49 2000 Server connected by ('38.28.162.194', 1064) at Sun Jun 18 19:37:49 2000 Server connected by ('38.28.162.194', 1067) at Sun Jun 18 19:37:50 2000  [client window 1] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net  Client received: 'Echo=>Hello network world at Sun Jun 18 19:37:54 2000'  [client window 2] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net Bruce  Client received: 'Echo=>Bruce at Sun Jun 18 19:37:54 2000'  [client window 3] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net The  Meaning of Life  Client received: 'Echo=>The at Sun Jun 18 19:37:55 2000' Client received: 'Echo=>Meaning at Sun Jun 18 19:37:56 2000' Client received: 'Echo=>of at Sun Jun 18 19:37:56 2000' Client received: 'Echo=>Life at Sun Jun 18 19:37:56 2000'

Again, all times here are on the server machine. This may be a little confusing because four windows are involved. In English, the test proceeds as follows:

The server starts running remotely.
All three clients are started and connect to the server at roughly the same time.
On the server, the client requests trigger three forked child processes, which all immediately go to sleep for five seconds (to simulate being busy doing something useful).
Each client waits until the server replies, which eventually happens five seconds after their initial requests.

In other words, all three clients are serviced at the same time by forked processes, while the main parent process continues listening for new client requests. If clients were not handled in parallel like this, no client could connect until the currently connected client's five-second delay expired.

In a more realistic application, that delay could be fatal if many clients were trying to connect at oncethe server would be stuck in the action we're simulating with time.sleep, and not get back to the main loop to accept new client requests. With process forks per request, all clients can be serviced in parallel.

Notice that we're using the same client script here (echo-client.py), just a different server; clients simply send and receive data to a machine and port and don't care how their requests are handled on the server. Also note that the server is running remotely on a Linux machine. (As we learned in Chapter 5, the fork call is not supported on Windows in standard Python at the time this book was written.) We can also run this test on a Linux server entirely, with two Telnet windows. It works about the same as when clients are started locally, in a DOS console window, but here "local" means a remote machine you're telnetting to locally:

  [one Telnet window] [lutz@starship uploads]$ python fork-server.py &  [1] 3379 Server connected by ('127.0.0.1', 2928) at Sun Jun 18 22:44:50 2000 Server connected by ('127.0.0.1', 2929) at Sun Jun 18 22:45:08 2000 Server connected by ('208.185.174.112', 2930) at Sun Jun 18 22:45:50 2000  [another Telnet window, same machine] [lutz@starship uploads]$ python echo-client.py  Client received: 'Echo=>Hello network world at Sun Jun 18 22:44:55 2000' [lutz@starship uploads]$ python echo-client.py localhost niNiNI  Client received: 'Echo=>niNiNI at Sun Jun 18 22:45:13 2000' [lutz@starship uploads]$ python echo-client.py starship.python.net Say no More!  Client received: 'Echo=>Say at Sun Jun 18 22:45:55 2000' Client received: 'Echo=>no at Sun Jun 18 22:45:55 2000' Client received: 'Echo=>More! at Sun Jun 18 22:45:55 2000'

Now let's move on to the tricky bits. This server script is fairly straightforward as forking code goes, but a few comments about some of the library tools it employs are in order.

13.4.1.2. Forking processes

We met os.fork in Chapter 5, but recall that forked processes are essentially a copy of the process that forks them, and so they inherit file and socket descriptors from their parent process. As a result, the new child process that runs the handleClient function has access to the connection socket created in the parent process. Programs know they are in a forked child process if the fork call returns 0; otherwise, the original parent process gets back the new child's ID.

13.4.1.3. Exiting from children

In earlier fork examples, child processes usually call one of the exec variants to start a new program in the child process. Here, instead, the child process simply calls a function in the same program and exits with os._exit. It's imperative to call os._exit hereif we did not, each child would live on after handleClient returns, and compete for accepting new client requests.

In fact, without the exit call, we'd wind up with as many perpetual server processes as requests servedremove the exit call and do a ps shell command after running a few clients, and you'll see what I mean. With the call, only the single parent process listens for new requests. os._exit is like sys.exit, but it exits the calling process immediately without cleanup actions. It's normally used only in child processes, and sys.exit is used everywhere else.

13.4.1.4. Killing the zombies

Note, however, that it's not quite enough to make sure that child processes exit and die. On systems like Linux, parents must also be sure to issue a wait system call to remove the entries for dead child processes from the system's process table. If we don't do this, the child processes will no longer run, but they will consume an entry in the system process table. For long-running servers, these bogus entries may become problematic.

It's common to call such dead-but-listed child processes zombies: they continue to use system resources even though they've already passed over to the great operating system beyond. To clean up after child processes are gone, this server keeps a list, activeChildren, of the process IDs of all child processes it spawns. Whenever a new incoming client request is received, the server runs its reapChildren to issue a wait for any dead children by issuing the standard Python os.waitpid(0,os.WNOHANG) call.

The os.waitpid call attempts to wait for a child process to exit and returns its process ID and exit status. With a 0 for its first argument, it waits for any child process. With the WNOHANG parameter for its second, it does nothing if no child process has exited (i.e., it does not block or pause the caller). The net effect is that this call simply asks the operating system for the process ID of any child that has exited. If any have, the process ID returned is removed both from the system process table and from this script's activeChildren list.

To see why all this complexity is needed, comment out the reapChildren call in this script, run it on a server, and then run a few clients. On my Linux server, a ps -f full process listing command shows that all the dead child processes stay in the system process table (show as <defunct>):

 [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz      3270  3264  0 22:33 pts/1    00:00:00 -bash lutz      3311  3270  0 22:37 pts/1    00:00:00 python fork-server.py lutz      3312  3311  0 22:37 pts/1    00:00:00 [python <defunct>] lutz      3313  3311  0 22:37 pts/1    00:00:00 [python <defunct>] lutz      3314  3311  0 22:37 pts/1    00:00:00 [python <defunct>] lutz      3316  3311  0 22:37 pts/1    00:00:00 [python <defunct>] lutz      3317  3311  0 22:37 pts/1    00:00:00 [python <defunct>] lutz      3318  3311  0 22:37 pts/1    00:00:00 [python <defunct>] lutz      3322  3270  0 22:38 pts/1    00:00:00 ps -f

When the reapChildren command is reactivated, dead child zombie entries are cleaned up each time the server gets a new client connection request, by calling the Python os.waitpid function. A few zombies may accumulate if the server is heavily loaded, but they will remain only until the next client connection is received:

 [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz      3270  3264  0 22:33 pts/1    00:00:00 -bash lutz      3340  3270  0 22:41 pts/1    00:00:00 python fork-server.py lutz      3341  3340  0 22:41 pts/1    00:00:00 [python <defunct>] lutz      3342  3340  0 22:41 pts/1    00:00:00 [python <defunct>] lutz      3343  3340  0 22:41 pts/1    00:00:00 [python <defunct>] lutz      3344  3270  6 22:41 pts/1    00:00:00 ps -f [lutz@starship uploads]$ Server connected by ('38.28.131.174', 1170) at Sun Jun 18 22:41:43 2000 [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz      3270  3264  0 22:33 pts/1    00:00:00 -bash lutz      3340  3270  0 22:41 pts/1    00:00:00 python fork-server.py lutz      3345  3340  0 22:41 pts/1    00:00:00 [python <defunct>] lutz      3346  3270  0 22:41 pts/1    00:00:00 ps -f

If you type fast enough, you can actually see a child process morph from a real running program into a zombie. Here, for example, a child spawned to handle a new request (process ID 11785) changes to <defunct> on exit. Its process entry will be removed completely when the next request is received:

 [lutz@starship uploads]$ Server connected by ('38.28.57.160', 1106) at Mon Jun 19 22:34:39 2000 [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz     11089 11088  0 21:13 pts/2    00:00:00 -bash lutz     11780 11089  0 22:34 pts/2    00:00:00 python fork-server.py lutz     11785 11780  0 22:34 pts/2    00:00:00 python fork-server.py lutz     11786 11089  0 22:34 pts/2    00:00:00 ps -f [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz     11089 11088  0 21:13 pts/2    00:00:00 -bash lutz     11780 11089  0 22:34 pts/2    00:00:00 python fork-server.py lutz     11785 11780  0 22:34 pts/2    00:00:00 [python <defunct>] lutz     11787 11089  0 22:34 pts/2    00:00:00 ps -f

13.4.1.5. Preventing zombies with signal handlers

On some systems, it's also possible to clean up zombie child processes by resetting the signal handler for the SIGCHLD signal raised by the operating system when a child process exits. If a Python script assigns the SIG_IGN (ignore) action as the SIGCHLD signal handler, zombies will be removed automatically and immediately as child processes exit; the parent need not issue wait calls to clean up after them. Because of that, this scheme is a simpler alternative to manually reaping zombies (on platforms where it is supported).

If you've already read Chapter 5, you know that Python's standard signal module lets scripts install handlers for signalssoftware-generated events. If you haven't read that chapter, here is a brief bit of background to show how this pans out for zombies. The program in Example 13-5 installs a Python-coded signal handler function to respond to whatever signal number you type on the command line.

Example 13-5. PP3E\Internet\Sockets\signal-demo.py

 ############################################################################# # Demo Python's signal module; pass signal number as a command-line arg, # use a "kill -N pid" shell command to send this process a signal; e.g., # on my Linux machine, SIGUSR1=10, SIGUSR2=12, SIGCHLD=17, and SIGCHLD # handler stays in effect even if not restored: all other handlers restored # by Python after caught, but SIGCHLD behavior is left to the platform's # implementation; signal works on Windows but defines only a few signal # types; signals are not very portable in general; ############################################################################# import sys, signal, time def now( ):     return time.ctime(time.time( )) def onSignal(signum, stackframe):                  # Python signal handler     print 'Got signal', signum, 'at', now( )      # most handlers stay in effect     if signum == signal.SIGCHLD:                   # but sigchld handler is not         print 'sigchld caught'         #signal.signal(signal.SIGCHLD, onSignal) signum = int(sys.argv[1]) signal.signal(signum, onSignal)                    # install signal handler while 1: signal.pause( )                          # sleep waiting for signals

To run this script, simply put it in the background and send it signals by typing the kill -signal-number process-id shell command line. Process IDs are listed in the PID column of ps command results. Here is this script in action catching signal numbers 10 (reserved for general use) and 9 (the unavoidable terminate signal):

 [lutz@starship uploads]$ python signal-demo.py 10 & [1] 11297 [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz     11089 11088  0 21:13 pts/2    00:00:00 -bash lutz     11297 11089  0 21:49 pts/2    00:00:00 python signal-demo.py 10 lutz     11298 11089  0 21:49 pts/2    00:00:00 ps -f [lutz@starship uploads]$ kill -10 11297 Got signal 10 at Mon Jun 19 21:49:27 2000 [lutz@starship uploads]$ kill -10 11297 Got signal 10 at Mon Jun 19 21:49:29 2000 [lutz@starship uploads]$ kill -10 11297 Got signal 10 at Mon Jun 19 21:49:32 2000 [lutz@starship uploads]$ kill -9 11297 [1]+  Killed                  python signal-demo.py 10

And here the script catches signal 17, which happens to be SIGCHLD on my Linux server. Signal numbers vary from machine to machine, so you should normally use their names, not their numbers. SIGCHLD behavior may vary per platform as well (see the signal module's library manual entry for more details):

 [lutz@starship uploads]$ python signal-demo.py 17 & [1] 11320 [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz     11089 11088  0 21:13 pts/2    00:00:00 -bash lutz     11320 11089  0 21:52 pts/2    00:00:00 python signal-demo.py 17 lutz     11321 11089  0 21:52 pts/2    00:00:00 ps -f [lutz@starship uploads]$ kill -17 11320 Got signal 17 at Mon Jun 19 21:52:24 2000 [lutz@starship uploads] sigchld caught [lutz@starship uploads]$ kill -17 11320 Got signal 17 at Mon Jun 19 21:52:27 2000 [lutz@starship uploads]$ sigchld caught

Now, to apply all of this to kill zombies, simply set the SIGCHLD signal handler to the SIG_IGN ignore handler action; on systems where this assignment is supported, child processes will be cleaned up when they exit. The forking server variant shown in Example 13-6 uses this trick to manage its children.

Example 13-6. PP3E\Internet\Sockets\fork-server-signal.py

 ############################################################################# # Same as fork-server.py, but use the Python signal module to avoid keeping # child zombie processes after they terminate, not an explicit loop before # each new connection; SIG_IGN means ignore, and may not work with SIG_CHLD # child exit signal on all platforms; on Linux, socket.accept cannot be # interrupted with a signal; ############################################################################# import os, time, sys, signal, signal from socket import *                      # get socket constructor and constants myHost = ''                               # server machine, '' means local host myPort = 50007                            # listen on a non-reserved port number sockobj = socket(AF_INET, SOCK_STREAM)           # make a TCP socket object sockobj.bind((myHost, myPort))                   # bind it to server port number sockobj.listen(5)                                # up to 5 pending connects signal.signal(signal.SIGCHLD, signal.SIG_IGN)    # avoid child zombie processes def now( ):                                       # time on server machine     return time.ctime(time.time( )) def handleClient(connection):                    # child process replies, exits     time.sleep(5)                                # simulate a blocking activity     while True:                                  # read, write a client socket         data = connection.recv(1024)         if not data: break         connection.send('Echo=>%s at %s' % (data, now( )))     connection.close( )     os._exit(0) def dispatcher( ):                                # listen until process killed     while True:                                   # wait for next connection,         connection, address = sockobj.accept( )   # pass to process for service         print 'Server connected by', address,         print 'at', now( )         childPid = os.fork( )                     # copy this process         if childPid == 0:                          # if in child process: handle             handleClient(connection)               # else: go accept next connect dispatcher( )

Where applicable, this technique is:

Much simpler; we don't need to manually track or reap child processes.
More accurate; it leaves no zombies temporarily between client requests.

In fact, only one line is dedicated to handling zombies here: the signal.signal call near the top, to set the handler. Unfortunately, this version is also even less portable than using os.fork in the first place, because signals may work slightly differently from platform to platform. For instance, some platforms may not allow SIG_IGN to be used as the SIGCHLD action at all. On Linux systems, though, this simpler forking server variant works like a charm:

 [lutz@starship uploads]$ Server connected by ('38.28.57.160', 1166) at Mon Jun 19 22:38:29 2000 [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz     11089 11088  0 21:13 pts/2    00:00:00 -bash lutz     11827 11089  0 22:37 pts/2    00:00:00 python fork-server-signal.py lutz     11835 11827  0 22:38 pts/2    00:00:00 python fork-server-signal.py lutz     11836 11089  0 22:38 pts/2    00:00:00 ps -f [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz     11089 11088  0 21:13 pts/2    00:00:00 -bash lutz     11827 11089  0 22:37 pts/2    00:00:00 python fork-server-signal.py lutz     11837 11089  0 22:38 pts/2    00:00:00 ps -f

Notice that in this version, the child process's entry goes away as soon as it exits, even before a new client request is received; no "defunct" zombie ever appears. More dramatically, if we now start up the script we wrote earlier that spawns eight clients in parallel (testecho.py) to talk to this server, all appear on the server while running, but are removed immediately as they exit:

 [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz     11089 11088  0 21:13 pts/2    00:00:00 -bash lutz     11827 11089  0 22:37 pts/2    00:00:00 python fork-server-signal.py lutz     11839 11827  0 22:39 pts/2    00:00:00 python fork-server-signal.py lutz     11840 11827  0 22:39 pts/2    00:00:00 python fork-server-signal.py lutz     11841 11827  0 22:39 pts/2    00:00:00 python fork-server-signal.py lutz     11842 11827  0 22:39 pts/2    00:00:00 python fork-server-signal.py lutz     11843 11827  0 22:39 pts/2    00:00:00 python fork-server-signal.py lutz     11844 11827  0 22:39 pts/2    00:00:00 python fork-server-signal.py lutz     11845 11827  0 22:39 pts/2    00:00:00 python fork-server-signal.py lutz     11846 11827  0 22:39 pts/2    00:00:00 python fork-server-signal.py lutz     11848 11089  0 22:39 pts/2    00:00:00 ps -f [lutz@starship uploads]$ ps -f UID        PID  PPID  C STIME TTY          TIME CMD lutz     11089 11088  0 21:13 pts/2    00:00:00 -bash lutz     11827 11089  0 22:37 pts/2    00:00:00 python fork-server-signal.py lutz     11849 11089  0 22:39 pts/2    00:00:00 ps -f

13.4.2. Threading Servers

The forking model just described works well on Unix-like platforms in general, but it suffers from some potentially significant limitations:

Performance: On some machines, starting a new process can be fairly expensive in terms of time and space resources.
Portability: Forking processes is a Unix technique; as we just noted, the fork call currently doesn't work on non-Unix platforms such as Windows under standard Python. As we learned in Chapter 5, forks can be used in the Cygwin version of Python on Windows, but they may be inefficient and not exactly the same as Unix forks.
Complexity: If you think that forking servers can be complicated, you're not alone. As we just saw, forking also brings with it all the shenanigans of managing zombiescleaning up after child processes that live shorter lives than their parents.

If you read Chapter 5, you know that one solution to all of these dilemmas is to use threads rather than processes. Threads run in parallel and share global (i.e., module and interpreter) memory, but they are usually less expensive to start, and work on both Unix-like machines and Microsoft Windows under standard Python today. Furthermore, some see threads as simpler to programchild threads die silently on exit, without leaving behind zombies to haunt the server.

Example 13-7 is another mutation of the echo server that handles client requests in parallel by running them in threads rather than in processes.

Example 13-7. PP3E\Internet\Sockets\thread-server.py

 ############################################################################# # Server side: open a socket on a port, listen for a message from a client, # and send an echo reply; echoes lines until eof when client closes socket; # spawns a thread to handle each client connection; threads share global # memory space with main thread; this is more portable than fork: threads # work on standard Windows systems, but process forks do not; ############################################################################# import thread, time from socket import *                     # get socket constructor and constants myHost = ''                              # server machine, '' means local host myPort = 50007                           # listen on a non-reserved port number sockobj = socket(AF_INET, SOCK_STREAM)           # make a TCP socket object sockobj.bind((myHost, myPort))                   # bind it to server port number sockobj.listen(5)                                # allow up to 5 pending connects def now( ):     return time.ctime(time.time( ))               # current time on the server def handleClient(connection):                    # in spawned thread: reply     time.sleep(5)                                # simulate a blocking activity     while True:                                  # read, write a client socket         data = connection.recv(1024)         if not data: break         connection.send('Echo=>%s at %s' % (data, now( )))     connection.close( ) def dispatcher( ):                                # listen until process killed     while True:                                    # wait for next connection,         connection, address = sockobj.accept( )   # pass to thread for service         print 'Server connected by', address,         print 'at', now( )         thread.start_new(handleClient, (connection,)) dispatcher( )

This dispatcher delegates each incoming client connection request to a newly spawned thread running the handleClient function. As a result, this server can process multiple clients at once, and the main dispatcher loop can get quickly back to the top to check for newly arrived requests. The net effect is that new clients won't be denied service due to a busy server.

Functionally, this version is similar to the fork solution (clients are handled in parallel), but it will work on any machine that supports threads, including Windows and Linux. Let's test it on both. First, start the server on a Linux machine and run clients on both Linux and Windows:

  [window 1: thread-based server process, server keeps accepting  client connections while threads are servicing prior requests] [lutz@starship uploads]$ /usr/bin/python thread-server.py  Server connected by ('127.0.0.1', 2934) at Sun Jun 18 22:52:52 2000 Server connected by ('38.28.131.174', 1179) at Sun Jun 18 22:53:31 2000 Server connected by ('38.28.131.174', 1182) at Sun Jun 18 22:53:35 2000 Server connected by ('38.28.131.174', 1185) at Sun Jun 18 22:53:37 2000  [window 2: client, but on same server machine] [lutz@starship uploads]$ python echo-client.py  Client received: 'Echo=>Hello network world at Sun Jun 18 22:52:57 2000'  [window 3: remote client, PC] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net  Client received: 'Echo=>Hello network world at Sun Jun 18 22:53:36 2000'  [window 4: client PC] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net Bruce  Client received: 'Echo=>Bruce at Sun Jun 18 22:53:40 2000'  [window 5: client PC] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net The  Meaning of Life  Client received: 'Echo=>The at Sun Jun 18 22:53:42 2000' Client received: 'Echo=>Meaning at Sun Jun 18 22:53:42 2000' Client received: 'Echo=>of at Sun Jun 18 22:53:42 2000' Client received: 'Echo=>Life at Sun Jun 18 22:53:42 2000'

Because this server uses threads rather than forked processes, we can run it portably on both Linux and a Windows PC. Here it is at work again, running on the same local Windows PC as its clients; again, the main point to notice is that new clients are accepted while prior clients are being processed in parallel with other clients and the main thread (in the five-second sleep delay):

  [window 1: server, on local PC] C:\...\PP3E\Internet\Sockets>python thread-server.py  Server connected by ('127.0.0.1', 1186) at Sun Jun 18 23:46:31 2000 Server connected by ('127.0.0.1', 1187) at Sun Jun 18 23:46:33 2000 Server connected by ('127.0.0.1', 1188) at Sun Jun 18 23:46:34 2000  [window 2: client, on local PC] C:\...\PP3E\Internet\Sockets>python echo-client.py  Client received: 'Echo=>Hello network world at Sun Jun 18 23:46:36 2000'  [window 3: client] C:\...\PP3E\Internet\Sockets>python echo-client.py localhost Brian  Client received: 'Echo=>Brian at Sun Jun 18 23:46:38 2000'  [window 4: client] C:\...\PP3E\Internet\Sockets>python echo-client.py localhost Bright side of Life  Client received: 'Echo=>Bright at Sun Jun 18 23:46:39 2000' Client received: 'Echo=>side at Sun Jun 18 23:46:39 2000' Client received: 'Echo=>of at Sun Jun 18 23:46:39 2000' Client received: 'Echo=>Life at Sun Jun 18 23:46:39 2000'

Remember that a thread silently exits when the function it is running returns; unlike the process fork version, we don't call anything like os._exit in the client handler function (and we shouldn'tit may kill all threads in the process!). Because of this, the thread version is not only more portable, but also simpler.

13.4.3. Standard Library Server Classes

Now that I've shown you how to write forking and threading servers to process clients without blocking incoming requests, I should also tell you that there are standard tools in the Python library to make this process easier. In particular, the SocketServer module defines classes that implement all flavors of forking and threading servers that you are likely to be interested in. Simply create the desired kind of imported server object, passing in a handler object with a callback method of your own, as shown in Example 13-8.

Example 13-8. PP3E\Internet\Sockets\class-server.py

 ############################################################################# # Server side: open a socket on a port, listen for a message from a client, # and send an echo reply; this version uses the standard library module # SocketServer to do its work; SocketServer allows us to make a simple # TCPServer, a ThreadingTCPServer, a ForkingTCPServer, and more, and # routes each client connect request to a new instance of a passed-in # request handler object's handle method; SocketServer also supports # UDP and Unix domain sockets; see the library manual for other usage. ############################################################################# import SocketServer, time               # get socket server, handler objects myHost = ''                             # server machine, '' means local host myPort = 50007                          # listen on a non-reserved port number def now( ):     return time.ctime(time.time( )) class MyClientHandler(SocketServer.BaseRequestHandler):     def handle(self):                           # on each client connect         print self.client_address, now( )            # show this client's address         time.sleep(5)                           # simulate a blocking activity         while True:                             # self.request is client socket             data = self.request.recv(1024)      # read, write a client socket             if not data: break             self.request.send('Echo=>%s at %s' % (data, now( )))         self.request.close( ) # make a threaded server, listen/handle clients forever myaddr = (myHost, myPort) server = SocketServer.ThreadingTCPServer(myaddr, MyClientHandler) server.serve_forever( )

This server works the same as the threading server we wrote by hand in the previous section, but instead focuses on service implementation (the customized handle method), not on threading details. It's run the same way, toohere it is processing three clients started by hand, plus eight spawned by the testecho script shown in Example 13-3:

  [window1: server, serverHost='localhost' in echo-client.py] C:\...\PP3E\Internet\Sockets>python class-server.py  ('127.0.0.1', 1189) Sun Jun 18 23:49:18 2000 ('127.0.0.1', 1190) Sun Jun 18 23:49:20 2000 ('127.0.0.1', 1191) Sun Jun 18 23:49:22 2000 ('127.0.0.1', 1192) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1193) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1194) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1195) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1196) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1197) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1198) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1199) Sun Jun 18 23:49:50 2000  [window2: client] C:\...\PP3E\Internet\Sockets>python echo-client.py  Client received: 'Echo=>Hello network world at Sun Jun 18 23:49:23 2000'  [window3: client] C:\...\PP3E\Internet\Sockets>python echo-client.py localhost Robin  Client received: 'Echo=>Robin at Sun Jun 18 23:49:25 2000'  [window4: client] C:\...\PP3E\Internet\Sockets>python echo-client.py localhost Brave Sir Robin  Client received: 'Echo=>Brave at Sun Jun 18 23:49:27 2000' Client received: 'Echo=>Sir at Sun Jun 18 23:49:27 2000' Client received: 'Echo=>Robin at Sun Jun 18 23:49:27 2000' C:\...\PP3E\Internet\Sockets>python testecho.py   [window4: contact remote server instead--times skewed] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net Brave  Sir Robin Client received: 'Echo=>Brave at Sun Jun 18 23:03:28 2000' Client received: 'Echo=>Sir at Sun Jun 18 23:03:28 2000' Client received: 'Echo=>Robin at Sun Jun 18 23:03:29 2000'

To build a forking server instead, just use the class name ForkingTCPServer when creating the server object. The SocketServer module has more power than shown by this example; it also supports synchronous (nonparallel) servers, UDP and Unix sockets, and so on. See Python's library manual for more details. Also see the end of Chapter 18 for more on Python server implementation tools.

For more advanced server needs, Python also comes with standard library tools that allow you to implement a full-blown HTTP (web) server that knows how to run server-side CGI scripts in a few lines of Python code. We'll explore those tools in Chapter 16.

13.4.4. Third-Party Server Tools: Twisted

For other server options, see the open source Twisted system (http://twistedmatrix.com). Twisted is an asynchronous networking framework written in Python that supports TCP, UDP, multicast, SSL/TLS, serial communication, and more. It supports both clients and servers and includes implementations of a number of commonly used network services such as a web server, an IRC chat server, a mail server, a relational database interface, and an object broker.

Although Twisted supports processes and threads for longer-running actions, it also uses an asynchronous, event-driven model to handle clients, which is similar to the event loop of GUI libraries like Tkinter. In fact, it abstracts an event loop, which multiplexes among open socket connectionscoincidentally, the topic of the next section.

13.4.5. Multiplexing Servers with select

So far we've seen how to handle multiple clients at once with both forked processes and spawned threads, and we've looked at a library class that encapsulates both schemes. Under both approaches, all client handlers seem to run in parallel with each other and with the main dispatch loop that continues watching for new incoming requests. Because all of these tasks run in parallel (i.e., at the same time), the server doesn't get blocked when accepting new requests or when processing a long-running client handler.

Technically, though, threads and processes don't really run in parallel, unless you're lucky enough to have a machine with many CPUs. Instead, your operating system performs a juggling actit divides the computer's processing power among all active tasks. It runs part of one, then part of another, and so on. All the tasks appear to run in parallel, but only because the operating system switches focus between tasks so fast that you don't usually notice. This process of switching between tasks is sometimes called time-slicing when done by an operating system; it is more generally known as multiplexing.

When we spawn threads and processes, we rely on the operating system to juggle the active tasks, but there's no reason that a Python script can't do so as well. For instance, a script might divide tasks into multiple stepsdo a step of one task, then one of another, and so on, until all are completed. The script need only know how to divide its attention among the multiple active tasks to multiplex on its own.

Servers can apply this technique to yield yet another way to handle multiple clients at once, a way that requires neither threads nor forks. By multiplexing client connections and the main dispatcher with the select system call, a single event loop can process clients and accept new ones in parallel (or at least close enough to avoid stalling). Such servers are sometimes called asynchronous, because they service clients in spurts, as each becomes ready to communicate. In asynchronous servers, a single main loop run in a single process and thread decides which clients should get a bit of attention each time through. Client requests and the main dispatcher are each given a small slice of the server's attention if they are ready to converse.

Most of the magic behind this server structure is the operating system select call, available in Python's standard select module. Roughly, select is asked to monitor a list of input sources, output sources, and exceptional condition sources and tells us which sources are ready for processing. It can be made to simply poll all the sources to see which are ready; wait for a maximum time period for sources to become ready; or wait indefinitely until one or more sources are ready for processing.

However used, select lets us direct attention to sockets ready to communicate, so as to avoid blocking on calls to ones that are not. That is, when the sources passed to select are sockets, we can be sure that socket calls like accept, recv, and send will not block (pause) the server when applied to objects returned by select. Because of that, a single-loop server that uses select need not get stuck communicating with one client or waiting for new ones while other clients are starved for the server's attention.

13.4.5.1. A select-based echo server

Let's see how all of this translates into code. The script in Example 13-9 implements another echo server, one that can handle multiple clients without ever starting new processes or threads.

Example 13-9. PP3E\Internet\Sockets\select-server.py

 ############################################################################# # Server: handle multiple clients in parallel with select. use the select # module to manually multiplex among a set of sockets: main sockets which # accept new client connections, and input sockets connected to accepted # clients; select can take an optional 4th arg--0 to poll, n.m to wait n.m # seconds, or ommitted to wait till any socket is ready for processing. ############################################################################# import sys, time from select import select from socket import socket, AF_INET, SOCK_STREAM def now(): return time.ctime(time.time( )) myHost = ''                             # server machine, '' means local host myPort = 50007                          # listen on a non-reserved port number if len(sys.argv) == 3:                  # allow host/port as cmdline args too     myHost, myPort = sys.argv[1:] numPortSocks = 2                        # number of ports for client connects # make main sockets for accepting new client requests mainsocks, readsocks, writesocks = [], [], [] for i in range(numPortSocks):     portsock = socket(AF_INET, SOCK_STREAM)   # make a TCP/IP spocket object     portsock.bind((myHost, myPort))           # bind it to server port number     portsock.listen(5)                        # listen, allow 5 pending connects     mainsocks.append(portsock)                # add to main list to identify     readsocks.append(portsock)                # add to select inputs list     myPort += 1                               # bind on consecutive ports # event loop: listen and multiplex until server process killed print 'select-server loop starting' while True:     #print readsocks     readables, writeables, exceptions = select(readsocks, writesocks, [])     for sockobj in readables:         if sockobj in mainsocks:                      # for ready input sockets             # port socket: accept new client             newsock, address = sockobj.accept( )      # accept should not block             print 'Connect:', address, id(newsock)     # newsock is a new socket             readsocks.append(newsock)                  # add to select list, wait         else:             # client socket: read next line             data = sockobj.recv(1024)                # recv should not block             print '\tgot', data, 'on', id(sockobj)             if not data:                             # if closed by the clients                 sockobj.close( )                          # close here and remv from                 readsocks.remove(sockobj)            # del list else reselected             else:                 # this may block: should really select for writes too                 sockobj.send('Echo=>%s at %s' % (data, now( )))

The bulk of this script is the big while event loop at the end that calls select to find out which sockets are ready for processing (these include main port sockets on which clients can connect, and open client connections). It then loops over all such ready sockets, accepting connections on main port sockets and reading and echoing input on any client sockets ready for input. Both the accept and recv calls in this code are guaranteed to not block the server process after select returns; as a result, this server can quickly get back to the top of the loop to process newly arrived client requests and already connected clients' inputs. The net effect is that all new requests and clients are serviced in pseudoparallel fashion.

To make this process work, the server appends the connected socket for each client to the readables list passed to select, and simply waits for the socket to show up in the selected inputs list. For illustration purposes, this server also listens for new clients on more than one porton ports 50007 and 50008, in our examples. Because these main port sockets are also interrogated with select, connection requests on either port can be accepted without blocking either already connected clients or new connection requests appearing on the other port. The select call returns whatever sockets in readables are ready for processingboth main port sockets and sockets connected to clients currently being processed.

13.4.5.2. Running the select server

Let's run this script locally to see how it does its stuff (the client and server can also be run on different machines, as in prior socket examples). First, we'll assume we've already started this server script in one window, and run a few clients to talk to it. The following code is the interaction in two such client windows running on Windows (MS-DOS consoles). The first client simply runs the echo-client script twice to contact the server, and the second also kicks off the testecho script to spawn eight echo-client programs running in parallel. As before, the server simply echoes back whatever text that client sends. Notice that the second client window really runs a script called echo-client-50008 so as to connect to the second port socket in the server; it's the same as echo-client, with a different port number (alas, the original script wasn't designed to input a port):

  [client window 1] C:\...\PP3E\Internet\Sockets>python echo-client.py  Client received: 'Echo=>Hello network world at Sun Aug 13 22:52:01 2000' C:\...\PP3E\Internet\Sockets>python echo-client.py  Client received: 'Echo=>Hello network world at Sun Aug 13 22:52:03 2000'  [client window 2] C:\...\PP3E\Internet\Sockets>python echo-client-50008.py localhost Sir Lancelot  Client received: 'Echo=>Sir at Sun Aug 13 22:52:57 2000' Client received: 'Echo=>Lancelot at Sun Aug 13 22:52:57 2000' C:\...\PP3E\Internet\Sockets>python testecho.py

The next code section is the sort of interaction and output that show up in the window where the server has been started. The first three connections come from echo-client runs; the rest is the result of the eight programs spawned by testecho in the second client window. Notice that for testecho, new client connections and client inputs are multiplexed together. If you study the output closely, you'll see that they overlap in time, because all activity is dispatched by the single event loop in the server.^[*] Also note that the server gets an empty string when the client has closed its socket. We take care to close and delete these sockets at the server right away, or else they would be needlessly reselected again and again, each time through the main loop:

^[*] And the trace output on the server will probably look a bit different every time it runs. Clients and new connections are interleaved almost at random due to timing differences on the host machines.

  [server window] C:\...\PP3E\Internet\Sockets>python select-server.py  select-server loop starting Connect: ('127.0.0.1', 1175) 7965520         got Hello network world on 7965520         got  on 7965520 Connect: ('127.0.0.1', 1176) 7964288         got Hello network world on 7964288         got  on 7964288 Connect: ('127.0.0.1', 1177) 7963920         got Sir on 7963920         got Lancelot on 7963920         got  on 7963920  [testecho results] Connect: ('127.0.0.1', 1178) 7965216         got Hello network world on 7965216         got  on 7965216 Connect: ('127.0.0.1', 1179) 7963968 Connect: ('127.0.0.1', 1180) 7965424         got Hello network world on 7963968 Connect: ('127.0.0.1', 1181) 7962976         got Hello network world on 7965424         got  on 7963968         got Hello network world on 7962976         got  on 7965424         got  on 7962976 Connect: ('127.0.0.1', 1182) 7963648         got Hello network world on 7963648         got  on 7963648 Connect: ('127.0.0.1', 1183) 7966640         got Hello network world on 7966640         got  on 7966640 Connect: ('127.0.0.1', 1184) 7966496         got Hello network world on 7966496         got  on 7966496 Connect: ('127.0.0.1', 1185) 7965888         got Hello network world on 7965888         got  on 7965888

A subtle but crucial point: a time.sleep call to simulate a long-running task doesn't make sense in the server herebecause all clients are handled by the same single loop, sleeping would pause everything (and defeat the whole point of a multiplexing server). Here are a few additional notes before we move on:

select call details

Formally, select is called with three lists of selectable objects (input sources, output sources, and exceptional condition sources), plus an optional timeout. The timeout argument may be a real wait expiration value in seconds (use floating-point numbers to express fractions of a second), a zero value to mean simply poll and return immediately, or omitted to mean wait until at least one object is ready (as done in our server script earlier). The call returns a triple of ready objectssubsets of the first three argumentsany or all of which may be empty if the timeout expired before sources became ready.

select portability

The select call works only for sockets on Windows, but also works for things like files and pipes on Unix and Macintosh. For servers running over the Internet, of course, the primary devices we are interested in are sockets.

Nonblocking sockets

select lets us be sure that socket calls like accept and recv won't block (pause) the caller, but it's also possible to make Python sockets nonblocking in general. Call the setblocking method of socket objects to set the socket to blocking or nonblocking mode. For example, given a call like sock.setblocking(flag), the socket sock is set to nonblocking mode if the flag is zero, and to blocking mode otherwise. All sockets start out in blocking mode initially, so socket calls may always make the caller wait.

But when in nonblocking mode, a socket.error exception is raised if a recv socket call doesn't find any data, or if a send call can't immediately transfer data. A script can catch this exception to determine whether the socket is ready for processing. In blocking mode, these calls always block until they can proceed. Of course, there may be much more to processing client requests than data transfers (requests may also require long-running computations), so nonblocking sockets don't guarantee that servers won't stall in general. They are simply another way to code multiplexing servers. Like select, they are better suited when client requests can be serviced quickly.

The asyncore module framework

If you're interested in using select, you will probably also be interested in checking out the asyncore.py module in the standard Python library. It implements a class-based callback model, where input and output callbacks are dispatched to class methods by a precoded select event loop. As such, it allows servers to be constructed without threads or forks, and is a select-based alternative to the SocketServer threading and forking module we met in the prior sections. We'll discuss this tool again at the end of Chapter 18; see the Python library manual for details and a usage example.

Twisted

The Twisted system, described in the prior section, abstracts away many of the details inherent in an asynchronous server and provides an event-driven model and framework. Twisted's internal event engine is similar in spirit to our select-based server and the asyncore module, but it is regarded as more advanced. Twisted is a third-party system, not a standard library tool.

13.4.6. Choosing a Server Scheme

So when should you use select to build a server, instead of threads or forks? Needs vary per application, of course, but servers based on the select call generally perform very well when client transactions are relatively short and are not CPU-bound. If they are not short, threads or forks may be a better way to split processing among multiple clients. Threads and forks are especially useful if clients require long-running processing above and beyond socket calls. However, combinations are possible toonothing is stopping a select-based polling loop from using threads, too.

It's important to remember that schemes based on select (and nonblocking sockets) are not completely immune to blocking. In the example earlier, for instance, the send call that echoes text back to a client might block too, and hence stall the entire server. We could work around that blocking potential by using select to make sure that the output operation is ready before we attempt it (e.g., use the writesocks list and add another loop to send replies to ready output sockets), albeit at a noticeable cost in program clarity.

In general, though, if we cannot split up the processing of a client's request in such a way that it can be multiplexed with other requests and not block the server's loop, select may not be the best way to construct the server.

Moreover, select also seems more complex than spawning either processes or threads, because we need to manually transfer control among all tasks (for instance, compare the threaded and select versions of this server, even without write selects). As usual, though, the degree of that complexity varies per application. The asyncore standard library module mentioned earlier simplifies some of the tasks of implementing a select-based event-loop socket server.