Shell Environment Variables

Shell variables, sometimes known as environment variables, are made available to Python scripts as os.environ, a Python dictionary-like object with one entry per variable setting in the shell. Shell variables live outside the Python system; they are often set at your system prompt or within startup files, and typically serve as systemwide configuration inputs to programs.

In fact, by now you should be familiar with a prime example: the PYTHONPATH module search path setting is a shell variable used by Python to import modules. By setting it once in your system startup files, its value is available every time a Python program is run. Shell variables can also be set by programs to serve as inputs to other programs in an application; because their values are normally inherited by spawned programs, they can be used as a simple form of interprocess communication.

2.9.1 Fetching Shell Variables

In Python, the surrounding shell environment becomes a simple preset object, not special syntax. Indexing os.environ by the desired shell variable's name string (e.g., os.environ['USER']) is the moral equivalent of adding a dollar sign before a variable name in most Unix shells (e.g., $USER), using surrounding percent signs on DOS (%USER%), and calling getenv("USER") in a C program. Let's start up an interactive session to experiment:

>>> import os
>>> os.environ.keys( )
['WINBOOTDIR', 'PATH', 'USER', 'PP2HOME', 'CMDLINE', 'PYTHONPATH', 'BLASTER', 
'X', 'TEMP', 'COMSPEC', 'PROMPT', 'WINDIR', 'TMP']
>>> os.environ['TEMP']
'C:\windows\TEMP'

Here, the keys method returns a list of variables set, and indexing fetches the value of shell variable TEMP on Windows. This works the same on Linux, but other variables are generally preset when Python starts up. Since we know about PYTHONPATH, let's peek at its setting within Python to verify its content:[6]

[6] For color, these results reflect an old path setting used during development; this variable now contains just the single directory containing the PP2E root.

>>> os.environ['PYTHONPATH']
'C:\PP2ndEd\examples\Part3;C:\PP2ndEd\examples\Part2;C:\PP2ndEd\
examples\Part2\Gui;C:\PP2ndEd\examples'
>>>
>>> import string
>>> for dir in string.split(os.environ['PYTHONPATH'], os.pathsep):
... print dir
...
C:PP2ndEdexamplesPart3
C:PP2ndEdexamplesPart2
C:PP2ndEdexamplesPart2Gui
C:PP2ndEdexamples

PYTHONPATH is a string of directory paths separated by whatever character is used to separate items in such paths on your platform (e.g., ";" on DOS/Window, ":" on Unix and Linux). To split it into its components, we pass string.split a delimiter os.pathsep, a portable setting that gives the proper separator for the underlying machine.

2.9.2 Changing Shell Variables

Like normal dictionaries, the os.environ object supports both key indexing and assignment. As usual, assignments change the value of the key:

>>> os.environ['TEMP'] = r'c:	emp'
>>> os.environ['TEMP']
'c:\temp'

But something extra happens here. In recent Python releases, values assigned to os.environ keys in this fashion are automatically exported to other parts of the application. That is, key assignments change both the os.environ object in the Python program as well as the associated variable in the enclosing shell environment of the running program's process. Its new value becomes visible to the Python program, all linked-in C modules, and any programs spawned by the Python process. Internally, key assignments to os.environ call os.putenv -- a function that changes the shell variable outside the boundaries of the Python interpreter. To demonstrate this how this works, we need a couple scripts that set and fetch shell variables; the first is shown in Example 2-4.

Example 2-4. PP2ESystemEnvironmentsetenv.py

import os
print 'setenv...',
print os.environ['USER'] # show current shell variable value

os.environ['USER'] = 'Brian' # runs os.putenv behind the scenes
os.system('python echoenv.py')

os.environ['USER'] = 'Arthur' # changes passed to spawned programs
os.system('python echoenv.py') # and linked-in C library modules

os.environ['USER'] = raw_input('?') 
print os.popen('python echoenv.py').read( ) 

This setenv.py script simply changes a shell variable, USER, and spawns another script that echoes this variable's value, shown in Example 2-5.

Example 2-5. PP2ESystemEnvironmentechoenv.py

import os
print 'echoenv...', 
print 'Hello,', os.environ['USER']

No matter how we run echoenv.py, it displays the value of USER in the enclosing shell; when run from the command line, this value comes from whatever we've set the variable to in the shell itself:

C:...PP2ESystemEnvironment>set USER=Bob

C:...PP2ESystemEnvironment>python echoenv.py
echoenv... Hello, Bob

When spawned by another script like setenv.py, though, echoenv.py gets whatever USER settings its parent program has made:

C:...PP2ESystemEnvironment>python setenv.py
setenv... Bob
echoenv... Hello, Brian
echoenv... Hello, Arthur
?Gumby
echoenv... Hello, Gumby

C:...PP2ESystemEnvironment>echo %USER%
Bob

This works the same way on Linux. In general terms, a spawned program always inherits environment settings from its parents. "Spawned" programs are programs started with Python tools such as os.spawnv on Windows, the os.fork/exec combination on Unix and Linux, and os.popen and os.system on a variety of platforms -- all programs thus launched get the environment variable settings that exist in the parent at launch time.[7]

[7] This is by default. Some program-launching tools also let scripts pass environment settings different from their own to child programs. For instance, the os.spawnve call is like os.spawnv, but accepts a dictionary argument representing the shell environment to be passed to the started program. Some os.exec* variants (ones with an "e" at the end of their names) similarly accept explicit environments; see the os.exec call formats in Chapter 3, for more details.

Setting shell variables like this before starting a new program is one way to pass information into the new program. For instance, a Python configuration script might tailor the PYTHONPATH variable to include custom directories, just before launching another Python script; the launched script will have the custom search path because shell variables are passed down to children (in fact, watch for such a launcher script to appear at the end of Chapter 4).

Notice the last command in the preceding example, though -- the USER variable is back to its original value after the top-level Python program exits. Assignments to os.environ keys are passed outside the interpreter and down the spawned programs chain, but never back up to parent program processes (including the system shell). This is also true in C programs that use the putenv library call, and isn't a Python limitation per se. It's also likely to be a nonissue if a Python script is at the top of your application. But keep in mind that shell settings made within a program only endure for that program's run, and that of its spawned children.