Section 5.8. Other Ways to Start Programs
5.8. Other Ways to Start ProgramsSuppose, just for a moment, that you've been asked to write a big Python book and you want to provide a way for readers to easily start the book's examples on just about any platform that Python runs on. Books are nice, but it's awfully fun to be able to click on demos right away. That is, you want to write a general and portable launcher program in Python for starting other Python programs. What to do? In this chapter, we've seen how to portably spawn threads, but these are simply parallel functions, not external programs. We've also learned how to go about starting new, independently running programs, with both the fork/exec combination and with tools for launching shell commands such as os.popen and os.system. Along the way, though, I've also been careful to point out numerous times that the os.fork call doesn't work on Windows today. This constraint may be improved by the time you read this book, but it still is a limitation as I write these words. Moreover, for reasons we'll explore later, the os.popen call is prone to blocking (pausing) its caller in some scenarios and requires a potentially platform-specific command-line string. Luckily, there are other ways to start programs in the Python standard library, some of which are more platform neutral than others:
We won't talk about the PyWin32 extensions package in this chapter, but the other tools available in the standard library merit a quick look here. 5.8.1. The os.spawn CallsThe os.spawn family of calls execute a program named by a command line in a new process, on both Windows and Unix-like systems. In basic operation, they are similar to the fork/exec call combination on Unix and can be used as alternatives to the system and popen calls we've already learned. In the following interaction, for instance, we start a Python program with a command line in two traditional ways (the second also reads its output): >>> print open('makewords.py').read( ) print 'spam' print 'eggs' print 'ham' >>> os.system('python makewords.py') spam eggs ham 0 >>> result = os.popen('python makewords.py').read( ) >>> print result spam eggs ham The equivalent os.spawn calls achieve the same effect, with a slightly more complex call signature that provides more control over the way the program is launched: >>> os.spawnv(os.P_WAIT, r'C:\Python24\python', ('python', 'makewords.py')) spam eggs ham 0 >>> os.spawnl(os.P_NOWAIT, r'C:\Python24\python', 'python', 'makewords.py') 1820 >>> spam eggs ham The spawn calls are also much like forking programs in Unix. They don't actually copy the calling process (so shared descriptor operations won't work), but they can be used to start a program running completely independent of the calling program, even on Windows. The script in Example 5-23 makes the similarity to Unix programming patterns more obvious. It launches a program with a fork/exec combination in Linux, or an os.spawnv call on Windows. Example 5-23. PP3E\System\Processes\spawnv.py
To make sense of these examples, you have to understand the arguments being passed to the spawn calls. In this script, we call os.spawnv with a process mode flag, the full directory path to the Python interpreter, and a tuple of strings representing the DOS command line with which to start a new program. The path to the Python interpreter executable program running a script is available as sys.executable in recent Python releases. In general, the process mode flag is taken from these predefined values:
In fact, there are eight different calls in the spawn family, which all start a program but vary slightly in their call signatures. In their names, an "l" means you list arguments individually, "p" means the executable file is looked up on the system path, and "e" means a dictionary is passed in to provide the shelled environment of the spawned program: the os.spawnve call, for example, works the same way as os.spawnv but accepts an extra fourth dictionary argument to specify a different shell environment for the spawned program (which, by default, inherits all of the parent's settings): os.spawnl(mode, path, ...) os.spawnle(mode, path, ..., env) os.spawnlp(mode, file, ...) # Unix only os.spawnlpe(mode, file, ..., env) # Unix only os.spawnv(mode, path, args) os.spawnve(mode, path, args, env) os.spawnvp(mode, file, args) # Unix only os.spawnvpe(mode, file, args, env) # Unix only Because these calls mimic the names and call signatures of the os.exec variants, see the section "The fork/exec Combination," earlier in this chapter, for more details on the differences between these call forms. Unlike the os.exec calls, only half of the os.spawn formsthose without system path checking (and hence without a "p" in their names)are currently implemented on Windows. All the process mode flags are supported on Windows, but detach and overlay modes are not available on Unix. To see which are present, read the library manual or ask Python: >>> import sys, os >>> sys.platform 'win32' >>> [x for x in dir(os) if x.startswith('spawn')] ['spawnl', 'spawnle', 'spawnv', 'spawnve'] >>> [x for x in dir(os) if x.startswith('P_')] ['P_DETACH', 'P_NOWAIT', 'P_NOWAITO', 'P_OVERLAY', 'P_WAIT'] Run a few tests or see the Python library manual for more details; things such as standard stream connection policies vary between the P_DETACH and P_NOWAIT modes in subtle ways. Here is the script in Example 5-23 at work on Windows, spawning 10 independent copies of the child.py Python program we met earlier in this chapter: C:\...\PP3E\System\Processes>type child.py import os, sys print 'Hello from child', os.getpid( ), sys.argv[1] C:\...\PP3E\System\Processes>python spawnv.py Hello from child -583587 0 Hello from child -558199 2 Hello from child -586755 1 Hello from child -562171 3 Main process exiting. Hello from child -581867 6 Hello from child -588651 5 Hello from child -568247 4 Hello from child -563527 7 Hello from child -543163 9 Hello from child -587083 8 Notice that the copies print their output in random order, and the parent program exits before all children do; all of these programs are really running in parallel on Windows. Also observe that the child program's output shows up in the console box where spawnv.py was run; when using P_NOWAIT, standard output comes to the parent's console, but it seems to go nowhere when using P_DETACH (which is most likely a feature when spawning GUI programs). 5.8.2. Launching Programs on WindowsThe os.system, os.popen, and os.spawn calls can be used to start command lines on Windows just as on Unix-like platforms (but with the handful of caveats mentioned earlier). On Windows, though, the DOS start command combined with os.system provides an easy way for scripts to launch any file on the system, using Windows filename associations. Starting a program file this way makes it run as independently as its starter. Example 5-24 demonstrates these launch techniques. Example 5-24. PP3E\System\Processes\dosstart.py
Uncomment one of the lines in this script's for loop to experiment with these schemes on your computer. On mine, when run with either of the first two calls in the loop uncommented, I get the following sort of outputthe text printed by five spawned Python programs: C:\...\PP3E\System\Processes>python dosstart.py Hello from child -582331 0 Hello from child -547703 1 Hello from child -547703 2 Hello from child -547651 3 Hello from child -547651 4 Main process exiting. The os.system call usually blocks its caller until the spawned program exits; reading the output of an os.popen call has the same blocking effect (the reader waits for the spawned program's output to be complete). But with either of the last two statements in the loop uncommented, I get output that simply looks like this: C:\...\PP3E\System\Processes>python dosstart.py Main process exiting. In both cases, I also see five new and completely independent DOS console windows appear on my display; when the third line in the loop is uncommented, all of the DOS boxes go away right after they appear; when the last line in the loop is active, they remain on the screen after the dosstart program exits because the child-wait script pauses for input before exiting. 5.8.2.1. Using the DOS start commandTo understand why, first you need to know how the DOS start command works in general. Roughly, a DOS command line of the form start command works as if command were typed in the Windows Run dialog box available in the Start button menu. If command is a filename, it is opened exactly as if its name had been double-clicked in the Windows Explorer file selector GUI. For instance, the following three DOS commands automatically start Internet Explorer on a file named index.html, my registered image viewer program on a file named uk-1.jpg, and my sound media player program on a file named sousa.au. Windows simply opens the file with whatever program is associated to handle filenames of that form. Moreover, all three of these programs run independently of the DOS console box where the command is typed: C:\temp>start c:\stuff\website\public_html\index.html C:\temp>start c:\stuff\website\public_html\uk-1.jpg C:\...\PP3E\System\Processes>start ..\..\Internet\Ftp\sousa.au Now, because the start command can run any file and command line, there is no reason it cannot also be used to start an independently running Python program: C:\...\PP3E\System\Processes>start child.py 1 Because Python is registered to open names ending in .py when it is installed, this really does work. The script child.py is launched independently of the DOS console window even though we didn't provide the name or path of the Python interpreter program. Because child.py simply prints a message and exits, though, the result isn't exactly satisfying: a new DOS window pops up to serve as the script's standard output, and it immediately goes away when the child exits (it's that Windows "flash feature" described earlier!). To do better, add a raw_input call at the bottom of the program file to wait for a key press before exiting: C:\...\PP3E\System\Processes>type child-wait.py import os, sys print 'Hello from child', os.getpid( ), sys.argv[1] raw_input("Press <Enter>") # don't flash on Windows C:\...\PP3E\System\Processes>start child-wait.py 2 Now the child's DOS window pops up and stays up after the start command has returned. Pressing the Enter key in the pop-up DOS window makes it go away. 5.8.2.2. Using start in Python scriptsSince we know that Python's os.system and os.popen can be called by a script to run any command line that can be typed at a DOS shell prompt, we can also start independently running programs from a Python script by simply running a DOS start command line. For instance: C:\...\PP3E>python >>> import os >>> >>> cmd = r'start c:\stuff\website\public_html\index.html' # start IE browser >>> os.system(cmd) # runs independent 0 >>> file = r'gui\gifs\pythonPowered.gif' # start image viewer >>> os.system('start ' + file) # IE opens .gif for me 0 >>> os.system('start ' + 'Gui/gifs/PythonPowered.gif') # fwd slashes work too 0 >>> os.system(r'start Internet\Ftp\sousa.au') # start media player 0 The four Python os.system calls here start whatever web-page browser, image viewer, and sound player are registered on your machine to open .html, .gif, and .au files (unless these programs are already running). The launched programs run completely independent of the Python sessionwhen running a DOS start command, os.system does not wait for the spawned program to exit. For instance, Figure 5-1 shows the .gif file handler in action on my machine, generated by both the second and the third os.system calls in the preceding code. Figure 5-1. Started image viewer (Internet Explorer) Now, since we also know that a Python program can be started from a command line, this yields two ways to launch Python programs: C:\...\PP3E>python >>> os.system(r'python Gui\TextEditor\textEditor.pyw') # start and wait 0 >>> os.system(r'start Gui\TextEditor\textEditor.pyw') # start, go on 0 When running a python command, the os.system call waits (blocks) for the command to finish. When running a start command, it does not; the launched Python program (here, PyEdit, a text editor GUI we'll meet in Chapter 12) runs independent of the os.system caller. And finally, that's why the following call in dosstart.py generates a new, independent instance of child-wait.py: C:\...\PP3E\System\Processes>python >>> os.system('start child-wait.py 1') 0 When run, this call pops up a new, independent DOS console window to serve as the standard input and output streams of the child-wait program. It truly is independentin fact, it keeps running if we exit both this Python interpreter session and the DOS console box where the command was typed.[*] An os.popen call can launch a start command too; but since it normally starts commands independently anyhow, the only obvious advantages of start here are the pop-up DOS box and the fact that Python need not be in the system search path setting:
>>> file = os.popen('start child-wait.py 1') # versus: python child-wait... >>> file.read( ) 'Hello from child -413849 1\012Press <Enter>' Which scheme to use, then? Using os.system or os.popen to run a python command works fine, but only if your users have added the python.exe directory to their system search path setting. Running a DOS start command is often a simpler alternative to both running python commands and calling the os.spawnv function, since filename associations are automatically installed along with Python and os.spawnv requires a full directory path to the Python interpreter program (python.exe). On the other hand, running start commands with os.system calls can fail on Windows for very long command-line strings: >>> os.system('start child-wait.py ' + 'Z'*425) # OK- 425 Zs in dos pop up 0 >>> os.system('start child-wait.py ' + 'Z'*450) # fails- msg, not exception Access is denied. 0 >>> os.popen('python child-wait.py ' + 'Z'*500).read( ) # works if PATH set >>> os.system('python child-wait.py ' + 'Z'*500) # works if PATH set >>> pypath = r'C:\program files\python\python.exe' # this works too >>> os.spawnv(os.P_NOWAIT, pypath, ('python', 'child-wait.py', 'Z'*500)) As a rule of thumb, use one of the os.spawn variants if your commands are (or may be) long. For instance, we'll meet a script in Chapter 6 that launches web browsers to view HTML files; even though a start command applied to an HTML file will automatically start a browser program, this script instead must use os.spawnv to accommodate potentially long directory paths in HTML filenames. 5.8.2.3. The os.startfile callOne more Windows twist: as mentioned previously in this book, recent Python releases also include an os.startfile call, which is essentially the same as spawning a DOS start command with os.system and works as though the named file were double-clicked. The following calls, for instance, have a similar effect: >>> os.startfile('README.txt') >>> os.system('start README.txt') Both pop up the README.txt file in Notepad on my Windows XP computer. Unlike the second of these calls, though, os.startfile provides no option to wait for the application to close (the DOS start command's /WAIT option does) and no way to retrieve the application's exit status (returned from os.system). On recent versions of Windows, the following has a similar effect too, because the registry is used at the command line (though this form pauses until the file's viewer is closedlike using start /WAIT): >>> os.system('README.txt') # 'start' is optional today A subtle thing: under the IDLE GUI in Python 2.4, the os.startfile call also does not pop up an intermediate DOS console window, whereas running a start command with os.system does, both from IDLE's interactive window and from within a script launched in IDLE. This seems to be just an artifact of the IDLE system, though; neither call pops up a DOS console outside of IDLE, whether typed interactively or run by a program that is launched from a system command line or icon click. If you start a Python script with any of these call forms, what happens also depends upon the name of the file: a .py file pops up a console window for os.startfile, but a .pyw file will not (exactly as when a file icon is clicked). The os.system call may map the script's output to the interactive session window unless start is used in the command, in which case we get a console pop up again. This is complex and seems like the sort of small nonlanguage detail that may vary over time, so try the variations that you care about on your own to see the DOS pop-up differences among these three call forms The os.startfile call works only on Windows, because it uses the Windows registry to know how to open the file. If you want to be more platform neutral, consider using os.popen or os.spawnv. For more information on other Windows-specific program launcher tools, see O'Reilly's Python Programming on Win32, by Mark Hammond and Andy Robinson. Other schemes are less standard than those shown here but are given excellent coverage in that text. 5.8.3. Other Program Launch OptionsFor a slightly higher-level interface for launching shell commands, see also the standard library commands module, a Unix-only module which is roughly just a wrapper around the os.popen call. In Python 2.4 and later, the subprocess module also allows you to spawn new processes, connect to their input, output, and error streams, and obtain their return codes. This module can be used to replace several older modules and functions, including os.system, os.spawn*, os.popen*, and commands.*. It provides lower-level control over spawned programs and is generally portable, but it can be more complex to code in some cases. Some advanced roles are made simpler by this module, however. For instance, to emulate shell-level command chaining with pipes, the following Python code: from subprocess import Popen p1 = Popen(["dmesg"], stdout=PIPE) p2 = Popen(["grep", "hda"], stdin=p1.stdout) output = p2.communicate( )[0] is equivalent to this Unix shell language command: output='dmesg | grep hda' |
