Section 9.7. File System

9.7. File System

Access to your file system occurs mostly through the Python os module. This module serves as the primary interface to your operating system facilities and services from Python. The os module is actually a front-end to the real module that is loaded, a module that is clearly operating systemdependent. This "real" module may be one of the following: posix (Unix-based, i.e., Linux, MacOS X, *BSD, Solaris, etc.), nt (Win32), mac (old MacOS), dos (DOS), os2 (OS/2), etc. You should never import those modules directly. Just import os and the appropriate module will be loaded, keeping all the underlying work hidden from sight. Depending on what your system supports, you may not have access to some of the attributes, which may be available in other operating system modules.

In addition to managing processes and the process execution environment, the os module performs most of the major file system operations that the application developer may wish to take advantage of. These features include removing and renaming files, traversing the directory tree, and managing file accessibility. Table 9.5 lists some of the more common file or directory operations available to you from the os module.

Table 9.5. os Module File/Directory Access Functions

Function	Description
File Processing
mkfifo()/mknod()[a]	Create named pipe/create filesystem node
remove()/unlink()	Delete file
rename()/renames()[b]	Rename file
*stat [c]( )	Return file statistics
symlink()	Create symbolic link
utime()	Update timestamp
tmpfile()	Create and open ('w+b') new temporary file
walk()[a]	Generate filenames in a directory tree
Directories/Folders
chdir()/fchdir()[a]	Change working directory/via a file descriptor
chroot()[d]	Change root directory of current process
listdir()	List files in directory
Directories/Folders
getcwd()/getcwdu()[a]	Return current working directory/same but in Unicode
mkdir()/makedirs()	Create directory(ies)
rmdir()/removedirs()	Remove directory(ies)
Access/Permissions
access()	Verify permission modes
chmod()	Change permission modes
chown()/lchown()[a]	Change owner and group ID/same, but do not follow links
umask()	Set default permission modes
File Descriptor Operations
open()	Low-level operating system open [for files, use the standard open() built-in functions
read()/write()	Read/write data to a file descriptor
dup()/dup2()	Duplicate file descriptor/same but to another FD
Device Numbers
makedev()[a]	Generate raw device number from major and minor device numbers
major()[a]/minor()[a]	Extract major/minor device number from raw device number

^[a] New in Python 2.3.

^[b] New in Python 1.5.2.

^[c] Includes stat(), lstat(), xstat().

^[d] New in Python 2.2.

A second module that performs specific pathname operations is also available. The os.path module is accessible through the os module. Included with this module are functions to manage and manipulate file pathname components, obtain file or directory information, and make file path inquiries. Table 9.6 outlines some of the more common functions in os.path.

Table 9.6. os.path Module Pathname Access Functions

Function	Description
Separation
basename()	Remove directory path and return leaf name
dirname()	Remove leaf name and return directory path
join()	Join separate components into single pathname
split()	Return (dirname(), basename()) tuple
splitdrive()	Return (drivename, pathname) tuple
splitext()	Return (filename, extension) tuple
Information
getatime()	Return last file access time
getctime()	Return file creation time
getmtime()	Return last file modification time
getsize()	Return file size (in bytes)
Inquiry
exists()	Does pathname (file or directory) exist?
isabs()	Is pathname absolute?
isdir()	Does pathname exist and is a directory?
isfile()	Does pathname exist and is a file?
islink()	Does pathname exist and is a symbolic link?
ismount()	Does pathname exist and is a mount point?
samefile()	Do both pathnames point to the same file?

These two modules allow for consistent access to the file system regardless of platform or operating system. The program in Example 9.1 (ospathex.py) test drives some of these functions from the os and os.path modules.

Example 9.1. os & os.path Modules Example (ospathex.py)

This code exercises some of the functionality found in the os and os.path modules. It creates a test file, populates a small amount of data in it, renames the file, and dumps its contents. Other auxiliary file operations are performed as well, mostly pertaining to directory tree traversal and file pathname manipulation.

1 #!/usr/bin/env python 2 3 import os 4 for tmpdir in ('/tmp', r'c:\temp'): 5     if os.path.isdir(tmpdir): 6         break 7 else: 8     print 'no temp directory available' 9     tmpdir = '' 10 11 if tmpdir: 12     os.chdir(tmpdir) 13     cwd = os.getcwd() 14     print '*** current temporary directory' 15     print cwd 16 17     print '*** creating example directory...' 18     os.mkdir('example') 19     os.chdir('example') 20     cwd = os.getcwd() 21     print '*** new working directory:' 22     print cwd 23     print '*** original directory listing:' 24     print os.listdir(cwd) 25 26     print '*** creating test file...' 27     fobj = open('test', 'w') 28     fobj.write('foo\n') 29     fobj.write('bar\n') 30     fobj.close() 31     print '*** updated directory listing:' 32     print os.listdir(cwd) 33 34     print "*** renaming 'test' to 'filetest.txt'" 35     os.rename('test', 'filetest.txt') 36     print '*** updated directory listing:' 37     print os.listdir(cwd) 38 39     path = os.path.join(cwd, os.listdir (cwd)[0]) 40     print '*** full file pathname' 41     print path 42     print '*** (pathname, basename) ==' 43     print os.path.split(path) 44     print '*** (filename, extension) ==' 45     print os.path.splitext(os.path.basename(path)) 46 47     print '*** displaying file contents:' 48     fobj = open(path) 49     for eachLine in fobj: 50         print eachLine, 51     fobj.close() 52 53     print '*** deleting test file' 54     os.remove(path) 55     print '*** updated directory listing:' 56     print os.listdir(cwd) 57     os.chdir(os.pardir) 58     print '*** deleting test directory' 59     os.rmdir('example') 60     print '*** DONE'

The os.path submodule to os focuses more on file pathnames. Some of the more commonly used attributes are found in Table 9.6.

Running this program on a Unix platform, we get the following output:

$ ospathex.py *** current temporary directory /tmp *** creating example directory... *** new working directory: /tmp/example *** original directory listing: [] *** creating test file... *** updated directory listing: ['test'] *** renaming 'test' to 'filetest.txt' *** updated directory listing: ['filetest.txt'] *** full file pathname: /tmp/example/filetest.txt *** (pathname, basename) == ('/tmp/example', 'filetest.txt') *** (filename, extension) == ('filetest', '.txt') *** displaying file contents: foo bar *** deleting test file *** updated directory listing: [] *** deleting test directory *** DONE

Running this example from a DOS window results in very similar execution:

C:\>python ospathex.py *** current temporary directory c:\windows\temp *** creating example directory... *** new working directory: c:\windows\temp\example *** original directory listing: [] *** creating test file... *** updated directory listing: ['test'] *** renaming 'test' to 'filetest.txt' *** updated directory listing: ['filetest.txt'] *** full file pathname: c:\windows\temp\example\filetest.txt *** (pathname, basename) == ('c:\\windows\\temp\\example', 'filetest.txt') *** (filename, extension) == ('filetest', '.txt') *** displaying file contents: foo bar *** deleting test file *** updated directory listing: [] *** deleting test directory *** DONE

Rather than providing a line-by-line explanation here, we will leave it to the reader as an exercise. However, we will walk through a similar interactive example (including errors) to give you a feel for what it is like to execute this script one step at a time. We will break into the code every now and then to describe the code we just encountered.

>>> import os >>> os.path.isdir('/tmp') True >>> os.chdir('/tmp') >>> cwd = os.getcwd() >>> cwd '/tmp'

This first block of code consists of importing the os module (which also grabs the os.path module). We verify that '/tmp' is a valid directory and change to that temporary directory to do our work. When we arrive, we call the getcwd() method to tell us where we are.

>>> os.mkdir('example') >>> os.chdir('example') >>> cwd = os.getcwd() >>> cwd '/tmp/example' >>> >>> os.listdir() # oops, forgot name Traceback (innermost last):   File "<stdin>", line 1, in ? TypeError: function requires at least one argument >>> >>> os.listdir(cwd) # that's better :) []

Next, we create a subdirectory in our temporary directory, after which we will use the listdir() method to confirm that the directory is indeed empty (since we just created it). The problem with our first call to listdir() was that we forgot to give the name of the directory we want to list. That problem is quickly remedied on the next line of input.

>>> fobj = open('test', 'w') >>> fobj.write('foo\n') >>> fobj.write('bar\n') >>> fobj.close() >>> os.listdir(cwd) ['test']

We then create a test file with two lines and verify that the file has been created by listing the directory again afterward.

>>> os.rename('test', 'filetest.txt') >>> os.listdir(cwd) ['filetest.txt'] >>> >>> path = os.path.join(cwd, os.listdir(cwd)[0]) >>> path '/tmp/example/filetest.txt' >>> >>> os.path.isfile(path) True >>> os.path.isdir(path) False >>> >>> os.path.split(path) ('/tmp/example', 'filetest.txt') >>> >>> os.path.splitext(os.path.basename(path)) ('filetest', '.ext')

This section is no doubt an exercise of os.path functionality, testing join(), isfile(), isdir() which we have seen earlier, split(), basename(), and splitext(). We also call the rename() function from os. Next, we display the file, and finally, we delete the temporary files and directories:

>>> fobj = open(path) >>> for eachLine in fobj: ...     print eachLine, ... foo bar >>> fobj.close() >>> os.remove(path) >>> os.listdir(cwd) [] >>> os.chdir(os.pardir) >>> os.rmdir('example')

Core Module(S): os (and os.path)

As you can tell from our lengthy discussion above, the os and os.path modules provide different ways to access the file system on your computer. Although our study in this chapter is restricted to file access only, the os module can do much more. It lets you manage your process environment, contains provisions for low-level file access, allows you to create and manage new processes, and even enables your running Python program to "talk" directly to another running program. You may find yourself a common user of this module in no time. Read more about the os module in Chapter 14.