Section 3.1.

3.1. "The os.path to Knowledge"

This chapter begins our in-depth look at ways to apply Python to real programming tasks. In this and the following chapters, you'll see how to use Python to write system tools, GUIs, database applications, Internet scripts, web sites, and more. Along the way, we'll also study larger Python programming concepts in action: code reuse, maintainability, object-oriented programming (OOP), and so on.

In this first part of the book, we begin our Python programming tour by exploring the systems application domain scripts that deal with files, programs, and the general environment surrounding a program. Although the examples in this domain focus on particular kinds of tasks, the techniques they employ will prove to be useful in later parts of the book as well. In other words, you should begin your journey here, unless you are already a Python systems programming wizard.

3.1.1. Why Python Here?

Python's system interfaces span application domains, but for the next five chapters, most of our examples fall into the category of system toolsprograms sometimes called command-line utilities, shell scripts, and other permutations of such words. Regardless of their title, you are probably already familiar with this sort of script; these scripts accomplish such tasks as processing files in a directory, launching test scripts, and so on. Such programs historically have been written in nonportable and syntactically obscure shell languages such as DOS batch files, csh, and awk.

Even in this relatively simple domain, though, some of Python's better attributes shine brightly. For instance, Python's ease of use and extensive built-in library make it simple (and even fun) to use advanced system tools such as threads, signals, forks, sockets, and their kin; such tools are much less accessible under the obscure syntax of shell languages and the slow development cycles of compiled languages. Python's support for concepts like code clarity and OOP also help us write shell tools that can be read, maintained, and reused. When using Python, there is no need to start every new script from scratch.

Moreover, we'll find that Python not only includes all the interfaces we need in order to write system tools, but also fosters script portability. By employing Python's standard library, most system scripts written in Python are automatically portable to all major platforms. For instance, you can usually run in Linux a Python directory-processing script written in Windows without changing its source code at allsimply copy over the source code. If used well, Python is the only system scripting tool you need to know.

3.1.2. The Next Five Chapters

To make this part of the book easier to study, I have broken it down into five chapters:

• In this chapter, I'll introduce the main system-related modules in overview fashion, and then use them to illustrate core system programming concepts: streams, command-line arguments, environment variables, and so on.

• In Chapter 4, we'll focus on the tools Python provides for processing files and directories, as well as focusing on directory trees.

• Chapter 5 moves on to cover Python's standard tools for parallel processingprocesses, threads, queues, pipes, signals, and more.

• Chapters 6 and 7 wrap up by presenting larger and more realistic examples that use the tools introduced in the prior three chapters. Chapter 6 is a collection of general system scripts, and Chapter 7 focuses on scripts for processing directories of files.

Especially in the two example chapters at the end of this part of the book, we will be concerned as much with system interfaces as with general Python development concepts. We'll see non-object-oriented and object-oriented versions of some examples along the way, for instance, to help illustrate the benefits of thinking in more strategic ways.

"Batteries Included"

This chapter, and those that follow, deal with both the Python language and its standard librarya collection of precoded modules written in Python and C that are installed with the Python interpreter. Although Python itself provides an easy-to-use scripting language, much of the real action in Python development involves this vast library of programming tools (a few hundred modules at last count) that ship with the Python package.

In fact, the standard library is so powerful that it is not uncommon to hear Python described with the phrase "batteries included"a phrase generally credited to Frank Stajano meaning that most of what you need for real day-to-day work is already there for importing.

As we'll see, the standard library forms much of the challenge in Python programming. Once you've mastered the core language, you'll find that you'll spend most of your time applying the built-in functions and modules that come with the system. On the other hand, libraries are where most of the fun happens. In practice, programs become most interesting when they start using services external to the language interpreter: networks, files, GUIs, databases, and so on. All of these are supported in the Python standard library. Beyond the standard library, there is an additional collection of third-party packages for Python that must be fetched and installed separately. As of this writing, you can find most of these third-party extensions via searches and links at http://www.python.org and at the PyPI and Vaults of Parnassus Python sites (also reachable from links at http://www.python.org). Some third-party extensions are large systems in their own rightNumPy and VPython, for instance, add vector processing and visualization, respectively.

If you have to do something special with Python, chances are good that you can find a free and open source module that will help. Most of the tools we'll employ in this text are a standard part of Python, but I'll be careful to point out things that must be installed separately.