Section 12.8. Other Features of Modules


12.8. Other Features of Modules

12.8.1. Auto-Loaded Modules

When the Python interpreter starts up in standard mode, some modules are loaded by the interpreter for system use. The only one that affects you is the __builtin__ module, which normally gets loaded in as the __builtins__ module.

The sys.modules variable consists of a dictionary of modules that the interpreter has currently loaded (in full and successfully) into the interpreter. The module names are the keys, and the location from which they were imported are the values.

For example, in Windows, the sys.modules variable contains a large number of loaded modules, so we will shorten the list by requesting only the module names. This is accomplished by using the dictionary's keys() method:

>>> import sys >>> sys.modules.keys() ['os.path', 'os', 'exceptions', '__main__', 'ntpath', 'strop', 'nt', 'sys',  '__builtin__', 'site', 'signal', 'UserDict', 'string', 'stat']


The loaded modules for Unix are quite similar:

>>> import sys >>> sys.modules.keys() ['os.path', 'os', 'readline', 'exceptions', '__main__', 'posix', 'sys', '__builtin__', 'site', 'signal', 'UserDict', 'posixpath', 'stat']


12.8.2. Preventing Attribute Import

If you do not want module attributes imported when a module is imported with "from module import *", prepend an underscore ( _ ) to those attribute names (you do not want imported). This minimal level of data hiding does not apply if the entire module is imported or if you explicitly import a "hidden" attribute, e.g., import foo._bar.

12.8.3. Case-Insensitive Import

There are various operating systems with case-insensitive file systems. Prior to version 2.1, Python attempted to "do the right thing" when importing modules on the various supported platforms, but with the growing popularity of the MacOS X and Cygwin platforms, certain deficiencies could no longer be ignored, and support needed to be cleaned up.

The world was pretty clean-cut when it was just Unix (case-sensitive) and Win32 (case-insensitive), but these new case-insensitive systems coming online were not ported with the case-insensitive features. PEP 235, which specifies this feature, attempts to address this weakness as well as taking away some "hacks" that had existed for other systems to make importing modules more consistent.

The bottom line is that for case-insensitive imports to work properly, an environment variable named PYTHONCASEOK must be defined. Python will then import the first module name that is found (in a case-insensitive manner) that matches. Otherwise Python will perform its native case-sensitive module name matching and import the first matching one it finds.

12.8.4. Source Code Encoding

Starting in Python 2.3, it is now possible to create your Python module file in a native encoding other than 7-bit ASCII. Of course ASCII is the default, but with an additional encoding directive at the top of your Python modules, it will enable the importer to parse your modules using the specified encoding and designate natively encoded Unicode strings correctly so you do not have to worry about editing your source files in a plain ASCII text editor and have to individually "Unicode-tag" each string literal.

An example directive specifying a UTF-8 file can be declared like this:

#!/usr/bin/env python # -*- coding: UTF-8 -*-


If you execute or import modules that contain non-ASCII Unicode string literals and do not have an encoding directive at the top, this will result in a DeprecationWarning in Python 2.3 and a syntax error starting in 2.5. You can read more about source code encoding in PEP 263.

12.8.5. Import Cycles

Working with Python in real-life situations, you discover that it is possible to have import loops. If you have ever worked on any large Python project, you are likely to have run into this situation.

Let us take a look at an example. Assume we have a very large product with a very complex command-line interface (CLI). There are a million commands for your product, and as a result, you have an overly massive handler (OMH) set. Every time a new feature is added, from one to three new commands must be added to support the new feature. This will be our omh4cli.py script:

from cli4vof import cli4vof # command line interface utility function def cli_util():     pass # overly massive handlers for the command line interface def omh4cli():          :     cli4vof()          : omh4cli()


You can pretend that the (empty) utility function is a very popular piece of code that most handlers must use. The overly massive handlers for the command-line interface are all in the omh4cli() function. If we have to add a new command, it would be called from here.

Now, as this module grows in a boundless fashion, certain smarter engineers decide to split off their new commands into a separate module and just provide hooks in the original module to access the new stuff. Therefore, the code is easier to maintain, and if bugs were found in the new stuff, one would not have to search through a one-megabyte-plus-sized Python file.

In our case, we have an excited product manager asking us to add a "very outstanding feature" (VOF). Instead of integrating our stuff into omh4cli.py, we create a new script, cli4vof.py:

import omh4cli # command-line interface for a very outstanding feature def cli4vof():     omh4cli.cli_util()


As mentioned before, the utility function is a must for every command, and because we do not want to cut and paste its code from the main handler, we import the main module and call it that way. To finish off our integration, we add a call to our handler into the main overly massive handler, omh4cli().

The problem occurs when the main handler omh4cli imports our new little module cli4vof (to get the new command function) because cli4vof imports omh4cli (to get the utility function). Our module import fails because Python is trying to import a module that was not previously fully imported the first time:

$ python omh4cli.py Traceback (most recent call last):   File "omh4cli.py", line 3, in ?     from cli4vof import cli4vof   File "/usr/prod/cli4vof.py", line 3, in ?     import omh4cli   File "/usr/prod/omh4cli.py", line 3, in ?     from cli4vof import cli4vof ImportError: cannot import name cli4vof


Notice the circular import of cli4vof in the traceback. The problem is that in order to call the utility function, cli4vof has to import omh4cli. If it did not have to do that, then omh4cli would have completed its import of cli4vof successfully and there would be no problem. The issue is that when omh4cli is attempting to import cli4vof, cli4vof is trying to import omh4cli. No one finishes an import, hence the error. This is just one example of an import cycle. There are much more complicated ones out in the real world.

The workaround for this problem is almost always to move one of the import statements, e.g., the offending one. You will commonly see import statements at the bottom of modules. As a beginning Python programmer, you are used to seeing them in the beginning, but if you ever run across import statements at the end of modules, you will now know why. In our case, we cannot move the import of omh4cli to the end, because if cli4vof() is called, it will not have the omh4cli name loaded yet:

$ python omh4cli.py Traceback (most recent call last):   File "omh4cli.py", line 3, in ?     from cli4vof import cli4vof   File "/usr/prod/cli4vof.py", line 7, in ?     import omh4cli   File "/usr/prod/omh4cli.py", line 13, in ?     omh4cli()   File "/usr/prod/omh4cli.py", line 11, in omh4cli     cli4vof()   File "/usr/prod/cli4vof.py", line 5, in cli4vof     omh4cli.cli_util() NameError: global name 'omh4cli' is not defined


No, our solution here is to just move the import statement into the cli4vof() function declaration:

def cli4vof():     import omh4cli     omh4cli.cli_util()


This way, the import of the cli4vof module from omh4cli completes successfully, and on the tail end, calling the utility function is successful because the omh4cli name is imported before it is called. As far as execution goes, the only difference is that from cli4vof, the import of omh4cli is performed when cli4vof.cli4vof() is called and not when the cli4vof module is imported.

12.8.6. Module Execution

There are many ways to execute a Python module: script invocation via the command-line or shell, execfile(), module import, interpreter -m option, etc. These are out of the scope of this chapter. We refer you to Chapter 14, "Execution Environment," which covers all of these features in full detail.



Core Python Programming
Core Python Programming (2nd Edition)
ISBN: 0132269937
EAN: 2147483647
Year: 2004
Pages: 334
Authors: Wesley J Chun

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net