13.1. "Tune In, Log On, and Drop Out"
Over the last decade, the Internet has virtually exploded onto the mainstream stage. It has rapidly grown from a simple communication device used primarily by academics and researchers into a medium that is now nearly as pervasive as the television and telephone. Social observers have likened the Internet's cultural impact to that of the printing press, and technical observers have suggested that all new software development of interest occurs only on the Internet. Naturally, time will be the final arbiter for such claims, but there is little doubt that the Internet is a major force in society, and one of the main application contexts for modern software systems.
The Internet also happens to be one of the primary application domains for the Python programming language. It has been a decade since the first edition of this book was written as well, and in that time the Internet's growth has strongly influenced Python's tool set and roles. Given Python and a computer with a socket-based Internet connection today, we can write Python scripts to read and send email around the world, fetch web pages from remote sites, transfer files by FTP, program interactive web sites, parse HTML and XML files, and much more, simply by using the Internet modules that ship with Python as standard tools.
In fact, companies all over the world do: Google, Yahoo!, Walt Disney, Hewlett-Packard, JPL, and many others rely on Python's standard tools to power their web sites. For example, the Google search enginewidely credited with making the web usablemakes extensive use of Python code. And the BitTorrent peer-to-peer file transfer systemwritten in Python and already downloaded by tens of millions of usersleverages Python's networking skills to share files among clients and remove server bottlenecks.
Many also build and manage their sites with the Zope web application server, which is itself written and customizable in Python. Others build sites with the Plone content management system, which is built upon Zope and delegates site content to its users. Still others use Python to script Java web applications with Jython (formerly known as JPython)a system that compiles Python programs to Java bytecode, exports Java libraries for use in Python scripts, and allows Python code to serve as web applets downloaded and run in a browser.
More recently, XML-RPC and SOAP interfaces for Python, such as xmlrpclib and SOAPy, have enabled web service programming; frameworks such as CherryPy, Webware, TurboGears, and Django have emerged as convenient tools for constructing web sites; the new XML package in Python's standard library provides a suite of XML processing tools; and the new IronPython implementation promises to provide seamless .NET/Mono integration for Python code.
As the Internet has grown, so too has Python's role as an Internet tool. Python has proven to be well suited to Internet scripting for some of the very same reasons that make it ideal in other domains. Its modular design and rapid turnaround mix well with the intense demands of Internet development. In this part of the book, we'll find that Python does more than simply support Internet scripts; it also fosters qualities such as productivity and maintainability that are essential to Internet projects of all shapes and sizes.
13.1.1. Internet Scripting Topics
Internet programming entails many topics, so to make the presentation easier to digest, I've split this subject over the next six chapters of this book. This chapter introduces Internet fundamentals and explores sockets, the underlying communications mechanism of the Internet. From there, later chapters move on to discuss the client, the server, web site construction, and more advanced topics.
Each chapter assumes you've read the previous one, but you can generally skip around, especially if you have any experience in the Internet domain. Since these chapters represent a substantial portion of this book at large, the following sections go into a few more details about what we'll be studying.
126.96.36.199. What we will cover
In conceptual terms, the Internet can roughly be thought of as being composed of multiple functional layers:
In this chapter and in Chapter 14, our main focus is on programming the second and third layers: sockets and higher-level protocols. We'll start this chapter at the bottom, learning about the socket model of network programming. Sockets aren't strictly tied to Internet scripting, but they are presented here because this is their primary role. As we'll see, most of what happens on the Internet happens through sockets, whether you notice or not.
After introducing sockets, the next two chapters make their way up to Python's client-side interfaces to higher-level protocolsthings like email and FTP transfers, which run on top of sockets. It turns out that a lot can be done with Python on the client alone, and Chapters 14 and 15 will sample the flavor of Python client-side scripting. The next two chapters then go on to present server-side scriptingprograms that run on a server computer and are usually invoked by a web browser. Finally, the last chapter in this part, Chapter 18, briefly introduces even higher-level tools such as Jython and Zope.
Along the way, we will also put to work some of the operating-system and GUI interfaces we studied earlier (e.g., processes, threads, signals, and Tkinter), and we'll investigate some of the design choices and challenges that the Internet presents.
That last statement merits a few more words. Internet scripting, like GUIs, is one of the sexier application domains for Python. As in GUI work, there is an intangible but instant gratification in seeing a Python Internet program ship information all over the world. On the other hand, by its very nature, network programming imposes speed overheads and user interface limitations. Though it may not be a fashionable stance these days, some applications are still better off not being deployed on the Net. In this part of the book, we will take an honest look at the Net's trade-offs as they arise.
The Internet is also considered by many to be something of an ultimate proof of concept for open source tools. Indeed, much of the Net runs on top of a large number of such tools, such as Python, Perl, the Apache web server, the sendmail program, MySQL, and Linux.[*] Moreover, new tools and technologies for programming the Web sometimes seem to appear faster than developers can absorb them.
The good news is that Python's integration focus makes it a natural in such a heterogeneous world. Today, Python programs can be installed as client-side and server-side tools; used as applets and servlets in Java applications; mixed into distributed object systems like CORBA, SOAP, and XML-RPC; integrated with XML-coded objects; and more. In more general terms, the rationale for using Python in the Internet domain is exactly the same as in any other: Python's emphasis on productivity, portability, and integration makes it ideal for writing Internet programs that are open, maintainable, and delivered according to the ever-shrinking schedules in this field.
188.8.131.52. What we won't cover
Now that I've told you what we will cover in this book, I should also mention what we won't cover. Like Tkinter, the Internet is a vast topic, and this part of the book is mostly an introduction to core concepts and representative tasks, not an exhaustive reference. There are simply too many Python Internet modules to include each in this text, but the examples here should help you understand the library manual entries for modules we don't have time to cover.
I also want to point out that higher-level tools like Jython and Zope are large systems in their own right, and they are best dealt with in more focused documents. Because books on both topics are available, we'll merely scratch their surfaces here. Moreover, this book says almost nothing about lower-level networking layers such as TCP/IP. If you're curious about what happens on the Internet at the bit-and-wire level, consult a good networking text for more details.
184.108.40.206. Running examples in this part of the book
Internet scripts generally imply execution contexts that earlier examples in this book have not. That is, it usually takes a bit more to run programs that talk over networks. Here are a few pragmatic notes about this part's examples, up front:
When a Python script opens an Internet connection (with the socket or protocol modules), Python will happily use whatever Internet link exists on your machine, be that a dedicated T1 line, a DSL line, or a simple modem. For instance, opening a socket on a Windows PC automatically initiates processing to create a dial-up connection to your Internet Service Provider (ISP) if needed (on a Windows laptop without a broadband connection, a modem connection dialog automatically pops up). In other words, if you have a way to connect to the Net, you likely can run programs in this chapter.
Moreover, as long as your machine supports sockets, you probably can run many of the examples here even if you have no Internet connection at all. As we'll see, a machine name localhost or "" (an empty string) usually means the local computer itself. This allows you to test both the client and the server sides of a dialog on the same computer without connecting to the Net. For example, you can run both socket-based clients and servers locally on a Windows PC without ever going out to the Net.
Some later examples assume that a particular kind of server is running on a server machine (e.g., FTP, POP, SMTP), but client-side scripts work on any Internet-aware machine with Python installed. Server-side examples in Chapter 16 and Chapter 17 require more: to develop CGI scripts, you'll need to either have a web server account, or run a web server program locally on your own computer (which is easier than you may thinkwe'll learn how to code one in Python in Chapter 16). Advanced third-party systems like Jython and Zope must be downloaded separately; we'll peek at some of these briefly in the advanced topics chapter.