Current Working Directory

The notion of the current working directory (CWD) turns out to be a key concept in some scripts' execution: it's always the implicit place where files processed by the script are assumed to reside, unless their names have absolute directory paths. As we saw earlier, os.getcwd lets a script fetch the CWD name explicitly, and os.chdir allows a script to move to a new CWD.

Keep in mind, though, that filenames without full pathnames map to the CWD, and have nothing to do with your PYTHONPATH setting. Technically, the CWD is always where a script is launched from, not the directory containing the script file. Conversely, imports always first search the directory containing the script, not the CWD (unless the script happens to also be located in the CWD). Since this distinction is subtle and tends to trip up beginners, let's explore it in more detail.

2.7.1 CWD, Files, and Import Paths

When you run a Python script by typing a shell command line like python dir1dir2file.py, the CWD is the directory you were in when you typed this command, not dir1dir2. On the other hand, Python automatically adds the identity of the script's home directory to the front of the module search path, such that file.py can always import other files in dir1dir2, no matter where it is run from. To illustrate, let's write a simple script to echo both its CWD and module search path:

C:PP2ndEdexamplesPP2ESystem>type whereami.py
import os, sys
print 'my os.getcwd =>', os.getcwd( ) # show my cwd execution dir
print 'my sys.path =>', sys.path[:6] # show first 6 import paths
raw_input( ) # wait for keypress if clicked

Now, running this script in the directory in which it resides sets the CWD as expected, and adds an empty string ('') to the front of the module search path, to designate the CWD (we met the sys.path module search path earlier):

C:PP2ndEdexamplesPP2ESystem>set PYTHONPATH=C:PP2ndEdexamples
C:PP2ndEdexamplesPP2ESystem>python whereami.py
my os.getcwd => C:PP2ndEdexamplesPP2ESystem
my sys.path => ['', 'C:\PP2ndEd\examples', 'C:\Program Files\Python
\Lib\plat-win', 'C:\Program Files\Python\Lib', 'C:\Program Files\
Python\DLLs', 'C:\Program Files\Python\Lib\lib-tk']

But if we run this script from other places, the CWD moves with us (it's the directory where we type commands), and Python adds a directory to the front of the module search path that allows the script to still see files in its own home directory. For instance, when running from one level up (".."), the "System" name added to the front of sys.path will be the first directory Python searches for imports within whereami.py ; it points imports back to the directory containing the script run. Filenames without complete paths, though, will be mapped to the CWD (C:PP2ndEdexamplesPP2E ), not the System subdirectory nested there:

C:PP2ndEdexamplesPP2ESystem>cd .. 
C:PP2ndEdexamplesPP2E>python Systemwhereami.py 
my os.getcwd => C:PP2ndEdexamplesPP2E
my sys.path => ['System', 'C:\PP2ndEd\examples', ...  rest same... ]

C:PP2ndEdexamplesPP2E>cd .. 
C:PP2ndEdexamples>python PP2ESystemwhereami.py 
my os.getcwd => C:PP2ndEdexamples
my sys.path => ['PP2E\System', 'C:\PP2ndEd\examples', ...  rest same...  ]

C:PP2ndEdexamplesPP2ESystem>cd PP2ESystemApp 
C:PP2ndEdexamplesPP2ESystemApp>python ..whereami.py 
my os.getcwd => C:PP2ndEdexamplesPP2ESystemApp
my sys.path => ['..', 'C:\PP2ndEd\examples', ...  rest same...  ]

The net effect is that filenames without directory paths in a script will be mapped to the place where the command was typed (os.getcwd), but imports still have access to the directory of the script being run (via the front of sys.path). Finally, when a file is launched by clicking its icon, the CWD is just the directory that contains the clicked file. The following output, for example, appears in a new DOS console box, when whereami.py is double-clicked in Windows explorer:

my os.getcwd => C:PP2ndEdexamplesPP2ESystem
my sys.path => ['C:\PP2NDED\EXAMPLES\PP2E\SYSTEM', 'C:\PP2ndEd\examples',
'C:\Program Files\Python\Lib\plat-win', 'C:\Program Files\Python\Lib',
'C:\Program Files\Python\DLLs']

In this case, both the CWD used for filenames and the first import search directory are the directory containing the script file. This all usually works out just as you expect, but there are two pitfalls to avoid:

• Filenames might need to include complete directory paths if scripts cannot be sure from where they will be run.
• Command-line scripts cannot use the CWD to gain import visibility to files not in their own directories; instead, use PYTHONPATH settings and package import paths to access modules in other directories.

For example, files in this book can always import other files in their own home directories without package path imports, regardless of how they are run (import filehere) but must go through the PP2E package root to find files anywhere else in the examples tree (from PP2E.dir1.dir2 import filethere) even if they are run from the directory containing the desired external module. As usual for modules, the PP2Edir1dir2 directory name could also be added to PYTHONPATH to make filethere visible everywhere without package path imports (though adding more directories to PYTHONPATH increases the likelihood of name clashes). In either case, though, imports are always resolved to the script's home directory or other Python search path settings, not the CWD.

2.7.2 CWD and Command Lines

This distinction between the CWD and import search paths explains why many scripts in this book designed to operate in the current working directory (instead of one whose name is passed in) are run with command lines like this:

C:	emp>python %X%PyToolscleanpyc-py.py   process cwd

In this example, the Python script file itself lives in the directory C:PP2ndEdexamplesPP2EPyTools, but because it is run from C: emp, it processes the files located in C: emp (i.e., in the CWD, not in the script's home directory). To process files elsewhere with such a script, simply cd to the directory to be processed to change the CWD:

C:	emp>cd C:PP2nEdexamples 
C:PP2ndEdexamples>python %X%PyToolscleanpyc-py.py   process cwd

Because the CWD is always implied, a cd tells the script which directory to process in no less certain terms that passing a directory name to the script explicitly like this:

C:...PP2EPyTools>python find.py *.py C:	emp   process named dir

In this command line, the CWD is the directory containing the script to be run (notice that the script filename has no directory path prefix); but since this script processes a directory named explicitly on the command line (C: emp), the CWD is irrelevant. Finally, if we want to run such a script located in some other directory to process files located in some other directory, we can simply give directory paths to both:

C:	emp>python %X%PyToolsfind.py *.cxx C:PP2ndEdexamplesPP2E

Here, the script has import visibility to files in its PP2EPyTools home directory and processes files in the PP2E root, but the CWD is something else entirely (C: emp). This last form is more to type, of course, but watch for a variety of CWD and explicit script-path command lines like these in this book.

Whenever you see a %X% in command lines like those in the preceding examples, it refers to the value of the shell environment variable named X. It's just a shorthand for the full directory pathname of the PP2E book examples package root directory, which I use to point to scripts' files. On my machines, it is preset in my PP2EConfig setup-pp* files like this:

set X=C:PP2ndEdexamplesPP2E --DOS
setenv X /home/mark/PP2ndEd/examples/PP2E --Unix/csh

That is, it is assigned and expanded to the directory where PP2E lives on the system. See the Configsetup-pp* files for more details, and see later in this chapter for more on shell variables. You can instead type full paths everywhere you see %X% in this book, but your fingers and your keyboard are probably both better off if you set X to your examples root.