13.4. Handling Multiple ClientsThe echo client and server programs shown previously serve to illustrate socket fundamentals. But the server model suffers from a fairly major flaw: if multiple clients try to connect to the server, and it takes a long time to process a given client's request, the server will fail. More accurately, if the cost of handling a given request prevents the server from returning to the code that checks for new clients in a timely manner, it won't be able to keep up with all the requests, and some clients will eventually be denied connections. In real-world client/server programs, it's far more typical to code a server so as to avoid blocking new requests while handling a current client's request. Perhaps the easiest way to do so is to service each client's request in parallelin a new process, in a new thread, or by manually switching (multiplexing) between clients in an event loop. This isn't a socket issue per se, and we already learned how to start processes and threads in Chapter 5. But since these schemes are so typical of socket server programming, let's explore all three ways to handle client requests in parallel here. 13.4.1. Forking ServersThe script in Example 13-4 works like the original echo server, but instead forks a new process to handle each new client connection. Because the handleClient function runs in a new process, the dispatcher function can immediately resume its main loop in order to detect and service a new incoming request. Example 13-4. PP3E\Internet\Sockets\fork-server.py
13.4.1.1. Running the forking serverParts of this script are a bit tricky, and most of its library calls work only on Unix-like platforms (not Windows). But before we get into too many details, let's start up our server and handle a few client requests. First, notice that to simulate a long-running operation (e.g., database updates, other network traffic), this server adds a five-second time.sleep delay in its client handler function, handleClient. After the delay, the original echo reply action is performed. That means that when we run a server and clients this time, clients won't receive the echo reply until five seconds after they've sent their requests to the server. To help keep track of requests and replies, the server prints its system time each time a client connect request is received, and adds its system time to the reply. Clients print the reply time sent back from the server, not their ownclocks on the server and client may differ radically, so to compare apples to apples, all times are server times. Because of the simulated delays, we also must usually start each client in its own console window on Windows (on some platforms, clients will hang in a blocked state while waiting for their reply). But the grander story here is that this script runs one main parent process on the server machine, which does nothing but watch for connections (in dispatcher), plus one child process per active client connection, running in parallel with both the main parent process and the other client processes (in handleClient). In principle, the server can handle any number of clients without bogging down. To test, let's start the server remotely in a Telnet window, and start three clients locally in three distinct console windows: [server telnet window] [lutz@starship uploads]$ uname -a Linux starship ... [lutz@starship uploads]$ python fork-server.py Server connected by ('38.28.162.194', 1063) at Sun Jun 18 19:37:49 2000 Server connected by ('38.28.162.194', 1064) at Sun Jun 18 19:37:49 2000 Server connected by ('38.28.162.194', 1067) at Sun Jun 18 19:37:50 2000 [client window 1] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net Client received: 'Echo=>Hello network world at Sun Jun 18 19:37:54 2000' [client window 2] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net Bruce Client received: 'Echo=>Bruce at Sun Jun 18 19:37:54 2000' [client window 3] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net The Meaning of Life Client received: 'Echo=>The at Sun Jun 18 19:37:55 2000' Client received: 'Echo=>Meaning at Sun Jun 18 19:37:56 2000' Client received: 'Echo=>of at Sun Jun 18 19:37:56 2000' Client received: 'Echo=>Life at Sun Jun 18 19:37:56 2000' Again, all times here are on the server machine. This may be a little confusing because four windows are involved. In English, the test proceeds as follows:
In other words, all three clients are serviced at the same time by forked processes, while the main parent process continues listening for new client requests. If clients were not handled in parallel like this, no client could connect until the currently connected client's five-second delay expired. In a more realistic application, that delay could be fatal if many clients were trying to connect at oncethe server would be stuck in the action we're simulating with time.sleep, and not get back to the main loop to accept new client requests. With process forks per request, all clients can be serviced in parallel. Notice that we're using the same client script here (echo-client.py), just a different server; clients simply send and receive data to a machine and port and don't care how their requests are handled on the server. Also note that the server is running remotely on a Linux machine. (As we learned in Chapter 5, the fork call is not supported on Windows in standard Python at the time this book was written.) We can also run this test on a Linux server entirely, with two Telnet windows. It works about the same as when clients are started locally, in a DOS console window, but here "local" means a remote machine you're telnetting to locally: [one Telnet window] [lutz@starship uploads]$ python fork-server.py & [1] 3379 Server connected by ('127.0.0.1', 2928) at Sun Jun 18 22:44:50 2000 Server connected by ('127.0.0.1', 2929) at Sun Jun 18 22:45:08 2000 Server connected by ('208.185.174.112', 2930) at Sun Jun 18 22:45:50 2000 [another Telnet window, same machine] [lutz@starship uploads]$ python echo-client.py Client received: 'Echo=>Hello network world at Sun Jun 18 22:44:55 2000' [lutz@starship uploads]$ python echo-client.py localhost niNiNI Client received: 'Echo=>niNiNI at Sun Jun 18 22:45:13 2000' [lutz@starship uploads]$ python echo-client.py starship.python.net Say no More! Client received: 'Echo=>Say at Sun Jun 18 22:45:55 2000' Client received: 'Echo=>no at Sun Jun 18 22:45:55 2000' Client received: 'Echo=>More! at Sun Jun 18 22:45:55 2000' Now let's move on to the tricky bits. This server script is fairly straightforward as forking code goes, but a few comments about some of the library tools it employs are in order. 13.4.1.2. Forking processesWe met os.fork in Chapter 5, but recall that forked processes are essentially a copy of the process that forks them, and so they inherit file and socket descriptors from their parent process. As a result, the new child process that runs the handleClient function has access to the connection socket created in the parent process. Programs know they are in a forked child process if the fork call returns 0; otherwise, the original parent process gets back the new child's ID. 13.4.1.3. Exiting from childrenIn earlier fork examples, child processes usually call one of the exec variants to start a new program in the child process. Here, instead, the child process simply calls a function in the same program and exits with os._exit. It's imperative to call os._exit hereif we did not, each child would live on after handleClient returns, and compete for accepting new client requests. In fact, without the exit call, we'd wind up with as many perpetual server processes as requests servedremove the exit call and do a ps shell command after running a few clients, and you'll see what I mean. With the call, only the single parent process listens for new requests. os._exit is like sys.exit, but it exits the calling process immediately without cleanup actions. It's normally used only in child processes, and sys.exit is used everywhere else. 13.4.1.4. Killing the zombiesNote, however, that it's not quite enough to make sure that child processes exit and die. On systems like Linux, parents must also be sure to issue a wait system call to remove the entries for dead child processes from the system's process table. If we don't do this, the child processes will no longer run, but they will consume an entry in the system process table. For long-running servers, these bogus entries may become problematic. It's common to call such dead-but-listed child processes zombies: they continue to use system resources even though they've already passed over to the great operating system beyond. To clean up after child processes are gone, this server keeps a list, activeChildren, of the process IDs of all child processes it spawns. Whenever a new incoming client request is received, the server runs its reapChildren to issue a wait for any dead children by issuing the standard Python os.waitpid(0,os.WNOHANG) call. The os.waitpid call attempts to wait for a child process to exit and returns its process ID and exit status. With a 0 for its first argument, it waits for any child process. With the WNOHANG parameter for its second, it does nothing if no child process has exited (i.e., it does not block or pause the caller). The net effect is that this call simply asks the operating system for the process ID of any child that has exited. If any have, the process ID returned is removed both from the system process table and from this script's activeChildren list. To see why all this complexity is needed, comment out the reapChildren call in this script, run it on a server, and then run a few clients. On my Linux server, a ps -f full process listing command shows that all the dead child processes stay in the system process table (show as <defunct>): [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 3270 3264 0 22:33 pts/1 00:00:00 -bash lutz 3311 3270 0 22:37 pts/1 00:00:00 python fork-server.py lutz 3312 3311 0 22:37 pts/1 00:00:00 [python <defunct>] lutz 3313 3311 0 22:37 pts/1 00:00:00 [python <defunct>] lutz 3314 3311 0 22:37 pts/1 00:00:00 [python <defunct>] lutz 3316 3311 0 22:37 pts/1 00:00:00 [python <defunct>] lutz 3317 3311 0 22:37 pts/1 00:00:00 [python <defunct>] lutz 3318 3311 0 22:37 pts/1 00:00:00 [python <defunct>] lutz 3322 3270 0 22:38 pts/1 00:00:00 ps -f When the reapChildren command is reactivated, dead child zombie entries are cleaned up each time the server gets a new client connection request, by calling the Python os.waitpid function. A few zombies may accumulate if the server is heavily loaded, but they will remain only until the next client connection is received: [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 3270 3264 0 22:33 pts/1 00:00:00 -bash lutz 3340 3270 0 22:41 pts/1 00:00:00 python fork-server.py lutz 3341 3340 0 22:41 pts/1 00:00:00 [python <defunct>] lutz 3342 3340 0 22:41 pts/1 00:00:00 [python <defunct>] lutz 3343 3340 0 22:41 pts/1 00:00:00 [python <defunct>] lutz 3344 3270 6 22:41 pts/1 00:00:00 ps -f [lutz@starship uploads]$ Server connected by ('38.28.131.174', 1170) at Sun Jun 18 22:41:43 2000 [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 3270 3264 0 22:33 pts/1 00:00:00 -bash lutz 3340 3270 0 22:41 pts/1 00:00:00 python fork-server.py lutz 3345 3340 0 22:41 pts/1 00:00:00 [python <defunct>] lutz 3346 3270 0 22:41 pts/1 00:00:00 ps -f If you type fast enough, you can actually see a child process morph from a real running program into a zombie. Here, for example, a child spawned to handle a new request (process ID 11785) changes to <defunct> on exit. Its process entry will be removed completely when the next request is received: [lutz@starship uploads]$ Server connected by ('38.28.57.160', 1106) at Mon Jun 19 22:34:39 2000 [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11780 11089 0 22:34 pts/2 00:00:00 python fork-server.py lutz 11785 11780 0 22:34 pts/2 00:00:00 python fork-server.py lutz 11786 11089 0 22:34 pts/2 00:00:00 ps -f [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11780 11089 0 22:34 pts/2 00:00:00 python fork-server.py lutz 11785 11780 0 22:34 pts/2 00:00:00 [python <defunct>] lutz 11787 11089 0 22:34 pts/2 00:00:00 ps -f 13.4.1.5. Preventing zombies with signal handlersOn some systems, it's also possible to clean up zombie child processes by resetting the signal handler for the SIGCHLD signal raised by the operating system when a child process exits. If a Python script assigns the SIG_IGN (ignore) action as the SIGCHLD signal handler, zombies will be removed automatically and immediately as child processes exit; the parent need not issue wait calls to clean up after them. Because of that, this scheme is a simpler alternative to manually reaping zombies (on platforms where it is supported). If you've already read Chapter 5, you know that Python's standard signal module lets scripts install handlers for signalssoftware-generated events. If you haven't read that chapter, here is a brief bit of background to show how this pans out for zombies. The program in Example 13-5 installs a Python-coded signal handler function to respond to whatever signal number you type on the command line. Example 13-5. PP3E\Internet\Sockets\signal-demo.py
To run this script, simply put it in the background and send it signals by typing the kill -signal-number process-id shell command line. Process IDs are listed in the PID column of ps command results. Here is this script in action catching signal numbers 10 (reserved for general use) and 9 (the unavoidable terminate signal): [lutz@starship uploads]$ python signal-demo.py 10 & [1] 11297 [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11297 11089 0 21:49 pts/2 00:00:00 python signal-demo.py 10 lutz 11298 11089 0 21:49 pts/2 00:00:00 ps -f [lutz@starship uploads]$ kill -10 11297 Got signal 10 at Mon Jun 19 21:49:27 2000 [lutz@starship uploads]$ kill -10 11297 Got signal 10 at Mon Jun 19 21:49:29 2000 [lutz@starship uploads]$ kill -10 11297 Got signal 10 at Mon Jun 19 21:49:32 2000 [lutz@starship uploads]$ kill -9 11297 [1]+ Killed python signal-demo.py 10 And here the script catches signal 17, which happens to be SIGCHLD on my Linux server. Signal numbers vary from machine to machine, so you should normally use their names, not their numbers. SIGCHLD behavior may vary per platform as well (see the signal module's library manual entry for more details): [lutz@starship uploads]$ python signal-demo.py 17 & [1] 11320 [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11320 11089 0 21:52 pts/2 00:00:00 python signal-demo.py 17 lutz 11321 11089 0 21:52 pts/2 00:00:00 ps -f [lutz@starship uploads]$ kill -17 11320 Got signal 17 at Mon Jun 19 21:52:24 2000 [lutz@starship uploads] sigchld caught [lutz@starship uploads]$ kill -17 11320 Got signal 17 at Mon Jun 19 21:52:27 2000 [lutz@starship uploads]$ sigchld caught Now, to apply all of this to kill zombies, simply set the SIGCHLD signal handler to the SIG_IGN ignore handler action; on systems where this assignment is supported, child processes will be cleaned up when they exit. The forking server variant shown in Example 13-6 uses this trick to manage its children. Example 13-6. PP3E\Internet\Sockets\fork-server-signal.py
Where applicable, this technique is:
In fact, only one line is dedicated to handling zombies here: the signal.signal call near the top, to set the handler. Unfortunately, this version is also even less portable than using os.fork in the first place, because signals may work slightly differently from platform to platform. For instance, some platforms may not allow SIG_IGN to be used as the SIGCHLD action at all. On Linux systems, though, this simpler forking server variant works like a charm: [lutz@starship uploads]$ Server connected by ('38.28.57.160', 1166) at Mon Jun 19 22:38:29 2000 [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11827 11089 0 22:37 pts/2 00:00:00 python fork-server-signal.py lutz 11835 11827 0 22:38 pts/2 00:00:00 python fork-server-signal.py lutz 11836 11089 0 22:38 pts/2 00:00:00 ps -f [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11827 11089 0 22:37 pts/2 00:00:00 python fork-server-signal.py lutz 11837 11089 0 22:38 pts/2 00:00:00 ps -f Notice that in this version, the child process's entry goes away as soon as it exits, even before a new client request is received; no "defunct" zombie ever appears. More dramatically, if we now start up the script we wrote earlier that spawns eight clients in parallel (testecho.py) to talk to this server, all appear on the server while running, but are removed immediately as they exit: [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11827 11089 0 22:37 pts/2 00:00:00 python fork-server-signal.py lutz 11839 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11840 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11841 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11842 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11843 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11844 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11845 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11846 11827 0 22:39 pts/2 00:00:00 python fork-server-signal.py lutz 11848 11089 0 22:39 pts/2 00:00:00 ps -f [lutz@starship uploads]$ ps -f UID PID PPID C STIME TTY TIME CMD lutz 11089 11088 0 21:13 pts/2 00:00:00 -bash lutz 11827 11089 0 22:37 pts/2 00:00:00 python fork-server-signal.py lutz 11849 11089 0 22:39 pts/2 00:00:00 ps -f 13.4.2. Threading ServersThe forking model just described works well on Unix-like platforms in general, but it suffers from some potentially significant limitations:
If you read Chapter 5, you know that one solution to all of these dilemmas is to use threads rather than processes. Threads run in parallel and share global (i.e., module and interpreter) memory, but they are usually less expensive to start, and work on both Unix-like machines and Microsoft Windows under standard Python today. Furthermore, some see threads as simpler to programchild threads die silently on exit, without leaving behind zombies to haunt the server. Example 13-7 is another mutation of the echo server that handles client requests in parallel by running them in threads rather than in processes. Example 13-7. PP3E\Internet\Sockets\thread-server.py
This dispatcher delegates each incoming client connection request to a newly spawned thread running the handleClient function. As a result, this server can process multiple clients at once, and the main dispatcher loop can get quickly back to the top to check for newly arrived requests. The net effect is that new clients won't be denied service due to a busy server. Functionally, this version is similar to the fork solution (clients are handled in parallel), but it will work on any machine that supports threads, including Windows and Linux. Let's test it on both. First, start the server on a Linux machine and run clients on both Linux and Windows: [window 1: thread-based server process, server keeps accepting client connections while threads are servicing prior requests] [lutz@starship uploads]$ /usr/bin/python thread-server.py Server connected by ('127.0.0.1', 2934) at Sun Jun 18 22:52:52 2000 Server connected by ('38.28.131.174', 1179) at Sun Jun 18 22:53:31 2000 Server connected by ('38.28.131.174', 1182) at Sun Jun 18 22:53:35 2000 Server connected by ('38.28.131.174', 1185) at Sun Jun 18 22:53:37 2000 [window 2: client, but on same server machine] [lutz@starship uploads]$ python echo-client.py Client received: 'Echo=>Hello network world at Sun Jun 18 22:52:57 2000' [window 3: remote client, PC] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net Client received: 'Echo=>Hello network world at Sun Jun 18 22:53:36 2000' [window 4: client PC] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net Bruce Client received: 'Echo=>Bruce at Sun Jun 18 22:53:40 2000' [window 5: client PC] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net The Meaning of Life Client received: 'Echo=>The at Sun Jun 18 22:53:42 2000' Client received: 'Echo=>Meaning at Sun Jun 18 22:53:42 2000' Client received: 'Echo=>of at Sun Jun 18 22:53:42 2000' Client received: 'Echo=>Life at Sun Jun 18 22:53:42 2000' Because this server uses threads rather than forked processes, we can run it portably on both Linux and a Windows PC. Here it is at work again, running on the same local Windows PC as its clients; again, the main point to notice is that new clients are accepted while prior clients are being processed in parallel with other clients and the main thread (in the five-second sleep delay): [window 1: server, on local PC] C:\...\PP3E\Internet\Sockets>python thread-server.py Server connected by ('127.0.0.1', 1186) at Sun Jun 18 23:46:31 2000 Server connected by ('127.0.0.1', 1187) at Sun Jun 18 23:46:33 2000 Server connected by ('127.0.0.1', 1188) at Sun Jun 18 23:46:34 2000 [window 2: client, on local PC] C:\...\PP3E\Internet\Sockets>python echo-client.py Client received: 'Echo=>Hello network world at Sun Jun 18 23:46:36 2000' [window 3: client] C:\...\PP3E\Internet\Sockets>python echo-client.py localhost Brian Client received: 'Echo=>Brian at Sun Jun 18 23:46:38 2000' [window 4: client] C:\...\PP3E\Internet\Sockets>python echo-client.py localhost Bright side of Life Client received: 'Echo=>Bright at Sun Jun 18 23:46:39 2000' Client received: 'Echo=>side at Sun Jun 18 23:46:39 2000' Client received: 'Echo=>of at Sun Jun 18 23:46:39 2000' Client received: 'Echo=>Life at Sun Jun 18 23:46:39 2000' Remember that a thread silently exits when the function it is running returns; unlike the process fork version, we don't call anything like os._exit in the client handler function (and we shouldn'tit may kill all threads in the process!). Because of this, the thread version is not only more portable, but also simpler. 13.4.3. Standard Library Server ClassesNow that I've shown you how to write forking and threading servers to process clients without blocking incoming requests, I should also tell you that there are standard tools in the Python library to make this process easier. In particular, the SocketServer module defines classes that implement all flavors of forking and threading servers that you are likely to be interested in. Simply create the desired kind of imported server object, passing in a handler object with a callback method of your own, as shown in Example 13-8. Example 13-8. PP3E\Internet\Sockets\class-server.py
This server works the same as the threading server we wrote by hand in the previous section, but instead focuses on service implementation (the customized handle method), not on threading details. It's run the same way, toohere it is processing three clients started by hand, plus eight spawned by the testecho script shown in Example 13-3: [window1: server, serverHost='localhost' in echo-client.py] C:\...\PP3E\Internet\Sockets>python class-server.py ('127.0.0.1', 1189) Sun Jun 18 23:49:18 2000 ('127.0.0.1', 1190) Sun Jun 18 23:49:20 2000 ('127.0.0.1', 1191) Sun Jun 18 23:49:22 2000 ('127.0.0.1', 1192) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1193) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1194) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1195) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1196) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1197) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1198) Sun Jun 18 23:49:50 2000 ('127.0.0.1', 1199) Sun Jun 18 23:49:50 2000 [window2: client] C:\...\PP3E\Internet\Sockets>python echo-client.py Client received: 'Echo=>Hello network world at Sun Jun 18 23:49:23 2000' [window3: client] C:\...\PP3E\Internet\Sockets>python echo-client.py localhost Robin Client received: 'Echo=>Robin at Sun Jun 18 23:49:25 2000' [window4: client] C:\...\PP3E\Internet\Sockets>python echo-client.py localhost Brave Sir Robin Client received: 'Echo=>Brave at Sun Jun 18 23:49:27 2000' Client received: 'Echo=>Sir at Sun Jun 18 23:49:27 2000' Client received: 'Echo=>Robin at Sun Jun 18 23:49:27 2000' C:\...\PP3E\Internet\Sockets>python testecho.py [window4: contact remote server instead--times skewed] C:\...\PP3E\Internet\Sockets>python echo-client.py starship.python.net Brave Sir Robin Client received: 'Echo=>Brave at Sun Jun 18 23:03:28 2000' Client received: 'Echo=>Sir at Sun Jun 18 23:03:28 2000' Client received: 'Echo=>Robin at Sun Jun 18 23:03:29 2000' To build a forking server instead, just use the class name ForkingTCPServer when creating the server object. The SocketServer module has more power than shown by this example; it also supports synchronous (nonparallel) servers, UDP and Unix sockets, and so on. See Python's library manual for more details. Also see the end of Chapter 18 for more on Python server implementation tools. For more advanced server needs, Python also comes with standard library tools that allow you to implement a full-blown HTTP (web) server that knows how to run server-side CGI scripts in a few lines of Python code. We'll explore those tools in Chapter 16. 13.4.4. Third-Party Server Tools: TwistedFor other server options, see the open source Twisted system (http://twistedmatrix.com). Twisted is an asynchronous networking framework written in Python that supports TCP, UDP, multicast, SSL/TLS, serial communication, and more. It supports both clients and servers and includes implementations of a number of commonly used network services such as a web server, an IRC chat server, a mail server, a relational database interface, and an object broker. Although Twisted supports processes and threads for longer-running actions, it also uses an asynchronous, event-driven model to handle clients, which is similar to the event loop of GUI libraries like Tkinter. In fact, it abstracts an event loop, which multiplexes among open socket connectionscoincidentally, the topic of the next section. 13.4.5. Multiplexing Servers with selectSo far we've seen how to handle multiple clients at once with both forked processes and spawned threads, and we've looked at a library class that encapsulates both schemes. Under both approaches, all client handlers seem to run in parallel with each other and with the main dispatch loop that continues watching for new incoming requests. Because all of these tasks run in parallel (i.e., at the same time), the server doesn't get blocked when accepting new requests or when processing a long-running client handler. Technically, though, threads and processes don't really run in parallel, unless you're lucky enough to have a machine with many CPUs. Instead, your operating system performs a juggling actit divides the computer's processing power among all active tasks. It runs part of one, then part of another, and so on. All the tasks appear to run in parallel, but only because the operating system switches focus between tasks so fast that you don't usually notice. This process of switching between tasks is sometimes called time-slicing when done by an operating system; it is more generally known as multiplexing. When we spawn threads and processes, we rely on the operating system to juggle the active tasks, but there's no reason that a Python script can't do so as well. For instance, a script might divide tasks into multiple stepsdo a step of one task, then one of another, and so on, until all are completed. The script need only know how to divide its attention among the multiple active tasks to multiplex on its own. Servers can apply this technique to yield yet another way to handle multiple clients at once, a way that requires neither threads nor forks. By multiplexing client connections and the main dispatcher with the select system call, a single event loop can process clients and accept new ones in parallel (or at least close enough to avoid stalling). Such servers are sometimes called asynchronous, because they service clients in spurts, as each becomes ready to communicate. In asynchronous servers, a single main loop run in a single process and thread decides which clients should get a bit of attention each time through. Client requests and the main dispatcher are each given a small slice of the server's attention if they are ready to converse. Most of the magic behind this server structure is the operating system select call, available in Python's standard select module. Roughly, select is asked to monitor a list of input sources, output sources, and exceptional condition sources and tells us which sources are ready for processing. It can be made to simply poll all the sources to see which are ready; wait for a maximum time period for sources to become ready; or wait indefinitely until one or more sources are ready for processing. However used, select lets us direct attention to sockets ready to communicate, so as to avoid blocking on calls to ones that are not. That is, when the sources passed to select are sockets, we can be sure that socket calls like accept, recv, and send will not block (pause) the server when applied to objects returned by select. Because of that, a single-loop server that uses select need not get stuck communicating with one client or waiting for new ones while other clients are starved for the server's attention. 13.4.5.1. A select-based echo serverLet's see how all of this translates into code. The script in Example 13-9 implements another echo server, one that can handle multiple clients without ever starting new processes or threads. Example 13-9. PP3E\Internet\Sockets\select-server.py
The bulk of this script is the big while event loop at the end that calls select to find out which sockets are ready for processing (these include main port sockets on which clients can connect, and open client connections). It then loops over all such ready sockets, accepting connections on main port sockets and reading and echoing input on any client sockets ready for input. Both the accept and recv calls in this code are guaranteed to not block the server process after select returns; as a result, this server can quickly get back to the top of the loop to process newly arrived client requests and already connected clients' inputs. The net effect is that all new requests and clients are serviced in pseudoparallel fashion. To make this process work, the server appends the connected socket for each client to the readables list passed to select, and simply waits for the socket to show up in the selected inputs list. For illustration purposes, this server also listens for new clients on more than one porton ports 50007 and 50008, in our examples. Because these main port sockets are also interrogated with select, connection requests on either port can be accepted without blocking either already connected clients or new connection requests appearing on the other port. The select call returns whatever sockets in readables are ready for processingboth main port sockets and sockets connected to clients currently being processed. 13.4.5.2. Running the select serverLet's run this script locally to see how it does its stuff (the client and server can also be run on different machines, as in prior socket examples). First, we'll assume we've already started this server script in one window, and run a few clients to talk to it. The following code is the interaction in two such client windows running on Windows (MS-DOS consoles). The first client simply runs the echo-client script twice to contact the server, and the second also kicks off the testecho script to spawn eight echo-client programs running in parallel. As before, the server simply echoes back whatever text that client sends. Notice that the second client window really runs a script called echo-client-50008 so as to connect to the second port socket in the server; it's the same as echo-client, with a different port number (alas, the original script wasn't designed to input a port): [client window 1] C:\...\PP3E\Internet\Sockets>python echo-client.py Client received: 'Echo=>Hello network world at Sun Aug 13 22:52:01 2000' C:\...\PP3E\Internet\Sockets>python echo-client.py Client received: 'Echo=>Hello network world at Sun Aug 13 22:52:03 2000' [client window 2] C:\...\PP3E\Internet\Sockets>python echo-client-50008.py localhost Sir Lancelot Client received: 'Echo=>Sir at Sun Aug 13 22:52:57 2000' Client received: 'Echo=>Lancelot at Sun Aug 13 22:52:57 2000' C:\...\PP3E\Internet\Sockets>python testecho.py The next code section is the sort of interaction and output that show up in the window where the server has been started. The first three connections come from echo-client runs; the rest is the result of the eight programs spawned by testecho in the second client window. Notice that for testecho, new client connections and client inputs are multiplexed together. If you study the output closely, you'll see that they overlap in time, because all activity is dispatched by the single event loop in the server.[*] Also note that the server gets an empty string when the client has closed its socket. We take care to close and delete these sockets at the server right away, or else they would be needlessly reselected again and again, each time through the main loop:
[server window] C:\...\PP3E\Internet\Sockets>python select-server.py select-server loop starting Connect: ('127.0.0.1', 1175) 7965520 got Hello network world on 7965520 got on 7965520 Connect: ('127.0.0.1', 1176) 7964288 got Hello network world on 7964288 got on 7964288 Connect: ('127.0.0.1', 1177) 7963920 got Sir on 7963920 got Lancelot on 7963920 got on 7963920 [testecho results] Connect: ('127.0.0.1', 1178) 7965216 got Hello network world on 7965216 got on 7965216 Connect: ('127.0.0.1', 1179) 7963968 Connect: ('127.0.0.1', 1180) 7965424 got Hello network world on 7963968 Connect: ('127.0.0.1', 1181) 7962976 got Hello network world on 7965424 got on 7963968 got Hello network world on 7962976 got on 7965424 got on 7962976 Connect: ('127.0.0.1', 1182) 7963648 got Hello network world on 7963648 got on 7963648 Connect: ('127.0.0.1', 1183) 7966640 got Hello network world on 7966640 got on 7966640 Connect: ('127.0.0.1', 1184) 7966496 got Hello network world on 7966496 got on 7966496 Connect: ('127.0.0.1', 1185) 7965888 got Hello network world on 7965888 got on 7965888 A subtle but crucial point: a time.sleep call to simulate a long-running task doesn't make sense in the server herebecause all clients are handled by the same single loop, sleeping would pause everything (and defeat the whole point of a multiplexing server). Here are a few additional notes before we move on:
13.4.6. Choosing a Server SchemeSo when should you use select to build a server, instead of threads or forks? Needs vary per application, of course, but servers based on the select call generally perform very well when client transactions are relatively short and are not CPU-bound. If they are not short, threads or forks may be a better way to split processing among multiple clients. Threads and forks are especially useful if clients require long-running processing above and beyond socket calls. However, combinations are possible toonothing is stopping a select-based polling loop from using threads, too. It's important to remember that schemes based on select (and nonblocking sockets) are not completely immune to blocking. In the example earlier, for instance, the send call that echoes text back to a client might block too, and hence stall the entire server. We could work around that blocking potential by using select to make sure that the output operation is ready before we attempt it (e.g., use the writesocks list and add another loop to send replies to ready output sockets), albeit at a noticeable cost in program clarity. In general, though, if we cannot split up the processing of a client's request in such a way that it can be multiplexed with other requests and not block the server's loop, select may not be the best way to construct the server. Moreover, select also seems more complex than spawning either processes or threads, because we need to manually transfer control among all tasks (for instance, compare the threaded and select versions of this server, even without write selects). As usual, though, the degree of that complexity varies per application. The asyncore standard library module mentioned earlier simplifies some of the tasks of implementing a select-based event-loop socket server. |