Section 22.7. Wrapping C Environment Calls

22.7. Wrapping C Environment Calls

Let's move on to a more useful application of C extension modules. The handcoded C file in Example 22-9 integrates the standard C library's getenv and putenv shell environment variable calls for use in Python scripts.

Example 22-9. PP3E\Integrate\Extend\CEnviron\cenviron.c

 /******************************************************************  * A C extension module for Python, called "cenviron".  Wraps the  * C library's getenv/putenv routines for use in Python programs.  ******************************************************************/ #include <Python.h> #include <stdlib.h> #include <string.h> /***********************/ /* 1) module functions */ /***********************/ static PyObject *                                   /* returns object */ wrap_getenv(PyObject *self, PyObject *args)         /* self not used */ {                                                   /* args from python */     char *varName, *varValue;     PyObject *returnObj = NULL;                         /* null=exception */     if (PyArg_Parse(args, "s", &varName)) {             /* Python -> C */         varValue = getenv(varName);                     /* call C getenv */         if (varValue != NULL)             returnObj = Py_BuildValue("s", varValue);   /* C -> Python */         else             PyErr_SetString(PyExc_SystemError, "Error calling getenv");     }     return returnObj; } static PyObject * wrap_putenv(PyObject *self, PyObject *args) {     char *varName, *varValue, *varAssign;     PyObject *returnObj = NULL;     if (PyArg_Parse(args, "(ss)", &varName, &varValue))     {         varAssign = malloc(strlen(varName) + strlen(varValue) + 2);         sprintf(varAssign, "%s=%s", varName, varValue);         if (putenv(varAssign) == 0) {             Py_INCREF(Py_None);                   /* C call success */             returnObj = Py_None;                  /* reference None */         }         else             PyErr_SetString(PyExc_SystemError, "Error calling putenv");     }     return returnObj; } /**************************/ /* 2) registration table  */ /**************************/ static struct PyMethodDef cenviron_methods[] = {     {"getenv", wrap_getenv},     {"putenv", wrap_putenv},        /* method name, address */     {NULL, NULL} }; /*************************/ /* 3) module initializer */ /*************************/ void initcenviron( )                  /* called on first import */ {     (void) Py_InitModule("cenviron", cenviron_methods);   /* mod name, table */ } 

This example is less useful now than it was in the first edition of this bookas we learned in Part II, not only can you fetch shell environment variables by indexing the os.environ table, but assigning to a key in this table automatically calls C's putenv to export the new setting to the C code layer in the process. That is, os.environ['key'] fetches the value of the shell variable 'key', and os.environ['key']=value assigns a variable both in Python and in C.

The second actionpushing assignments out to Cwas added to Python releases after the first edition of this book was published. Besides demonstrating additional extension coding techniques, though, this example still serves a practical purpose: even today, changes made to shell variables by the C code linked into a Python process are not picked up when you index os.environ in Python code. That is, once your program starts, os.environ reflects only subsequent changes made by Python code.

Moreover, although Python now has both a putenv and a getenv call in its os module, their integration seems incomplete. Changes to os.environ call os.putenv, but direct calls to os.putenv do not update os.environ, so the two can become out of sync. And os.getenv today simply translates to an os.environ fetch, and hence will not pick up environment changes made in the process outside of Python code after startup time. This may rarely, if ever, be an issue for you, but this C extension module is not completely without purpose; to truly interface environment variables with linked-in C code, we need to call the C library routines directly.^[*]

^[*] This code is also open to customization (e.g., it can limit the set of shell variables read and written by checking names), but you could do the same by wrapping os.environ. In fact, because os.environ is simply a Python UserDict subclass that preloads shell variables on startup, you could almost add the required getenv call to load C layer changes by simply wrapping os.environ accesses in a Python class whose _ _getitem_ _ calls gentenv before passing the access off to os.environ. But you still need C's getenv call in the first place, and it's not directly available in os today.

The cenviron.c C file in Example 22-9 creates a Python module called cenviron that does a bit more than the prior examplesit exports two functions, sets some exception descriptions explicitly, and makes a reference count call for the Python None object (it's not created anew, so we need to add a reference before passing it to Python). As before, to add this code to Python, compile and link into an object file; the Cygwin makefile in Example 22-10 builds the C source code for dynamic binding.

Example 22-10. PP3E\Integrate\Extend\Cenviron\makefile.cenviron

 ################################################################## # Compile cenviron.c into cenviron.dll--a shareable object file # on Cygwin, which is loaded dynamically when first imported. ################################################################## PYLIB = /usr/bin PYINC = /usr/include/python2.4 cenviron.dll: cenviron.c         gcc cenviron.c -g -I$(PYINC) -shared  -L$(PYLIB) -lpython2.4 -o $@ clean:         rm -f *.pyc cenviron.dll 

To build, type make -f makefile.cenviron at your shell. To run, make sure the .dll file is in a directory on Python's module path (a period [.] works too):

 .../PP3E/Integrate/Extend/Cenviron$ python >>> import cenviron >>> cenviron.getenv('USER')                # like os.environ[key] but refetched 'mark' >>> cenviron.putenv('USER', 'gilligan')    # like os.environ[key]=value >>> cenviron.getenv('USER')                # C sees the changes too 'gilligan' 

As before, cenviron is a bona fide Python module object after it is imported, with all the usual attached information:

 >>> dir(cenviron) ['_ _doc_ _', '_ _file_ _', '_ _name_ _', 'getenv', 'putenv'] >>> cenviron._ _file_ _ './cenviron.dll' >>> cenviron._ _name_ _ 'cenviron' >>> cenviron.getenv <built-in function getenv> >>> cenviron <module 'cenviron' from 'cenviron.dll'> >>> cenviron.getenv('PYTHONPATH') '/cygdrive/c/Mark/PP3E-cd/Examples' 

Here is an example of the problem this module addresses (but you have to pretend that some of these calls are made by linked-in C code, not by Python):

 .../PP3E/Integrate/Extend/Cenviron$ python >>> import os >>> os.environ['USER']                      # initialized from the shell 'skipper' >>> from cenviron import getenv, putenv     # direct C library call access >>> getenv('USER') 'skipper' >>> putenv('USER', 'gilligan')              # changes for C but not Python >>> getenv('USER') 'gilligan' >>> os.environ['USER']                      # oops--does not fetch values again 'skipper' >>> os.getenv('USER')                       # ditto 'skipper' 

22.7.1. Adding Wrapper Classes to Flat Libraries

As is, the C extension module exports a function-based interface, but you can wrap its functions in Python code that makes the interface look any way you like. For instance, Example 22-11 makes the functions accessible by dictionary indexing and integrates with the os.environ objectit guarantees that the object will stay in sync with fetches and changes made by calling our C extension functions.

Example 22-11. PP3E\Integrate\Extend\Cenviron\envmap.py

 import os from cenviron import getenv, putenv       # get C module's methods class EnvMapping:                         # wrap in a Python class     def _ _setitem_ _(self, key, value):         os.environ[key] = value           # on writes: Env[key]=value         putenv(key, value)                # put in os.environ too     def _ _getitem_ _(self, key):         value = getenv(key)               # on reads: Env[key]         os.environ[key] = value           # integrity check         return value Env = EnvMapping( )                        # make one instance 

To use this module, clients may import its Env object using Env['var'] dictionary syntax to refer to environment variables. And Example 22-12 exports the functions as qualified attribute names rather than as callsvariables are referenced with Env.var attribute syntax. The main point to notice here is that you can graft many different sorts of interface models on top of extension functions by providing Python wrappers on top of the extension's C wrappers.

Example 22-12. PP3E\Integrate\Extend\Cenviron\envattr.py

 import os from cenviron import getenv, putenv       # get C module's methods class EnvWrapper:                         # wrap in a Python class     def _ _setattr_ _(self, name, value):         os.environ[name] = value          # on writes: Env.name=value         putenv(name, value)               # put in os.environ too     def _ _getattr_ _(self, name):         value = getenv(name)              # on reads: Env.name         os.environ[name] = value          # integrity check         return value Env = EnvWrapper( )                        # make one instance 

22.7.2. But Don't Do That EitherSWIG

You can manually code extension modules like we just did, but you don't necessarily have to. Because this example really just wraps functions that already exist in standard C libraries, the entire cenviron.c C code file in Example 22-9 can be replaced with a simple SWIG input file that looks like Example 22-13.

Example 22-13. PP3E\Integrate\Extend\Swig\Environ\environ.i

 /***************************************************************  * Swig module description file, to generate all Python wrapper  * code for C lib getenv/putenv calls: "swig -python environ.i".  ***************************************************************/ %module environ extern char * getenv(const char *varname); extern int    putenv(char *assignment); 

And you're done. Well, almost; you still need to run this file through SWIG and compile its output. As before, simply add a SWIG step to your makefile and compile its output file into a shareable object, and you're in business. Example 22-14 is a Cygwin makefile that does the job.

Example 22-14. PP3E\Integrate\Extend\Swig\Environ\makefile.environ-swig

 # build environ extension from SWIG generated code PYLIB = /usr/bin PYINC = /usr/include/python2.4 _environ.dll: environ_wrap.c         gcc environ_wrap.c -g -I$(PYINC) -L$(PYLIB) -lpython2.4 -shared -o $@ environ_wrap.c: environ.i         swig -python environ.i clean:         rm -f *.o *.dll *.pyc core environ_wrap.c environ.py 

When run on environ.i, SWIG generates two files and two modulesenviron.py (the Python interface module we import) and environ_wrap.c (the lower-level glue code module file we compile). Because the functions being wrapped here live in standard linked-in C libraries, there is nothing to combine with the generated code; this makefile simply runs SWIG and compiles the wrapper file into a C extension module, ready to be imported:

 .../PP3E/Integrate/Extend/Swig/Environ$ make -f makefile.environ-swig swig -python environ.i gcc environ_wrap.c -g -I/usr/include/python2.4 -L/usr/bin -lpython2.4     -shared -o _environ.dll 

And now you're really done. The resulting C extension module is linked when imported, and it's used as before (except that SWIG handled all the gory bits):

 .../PP3E/Integrate/Extend/Swig/Environ$ ls _environ.dll  environ.py      makefile.environ-swig environ.i     environ_wrap.c .../PP3E/Integrate/Extend/Swig/Environ$ python >>> import environ >>> environ.getenv('USER') 'Mark Lutz' >>> temp = 'USER=gilligan'                # use C lib call pattern now >>> environ.putenv(temp)                  # temp required in Cygwin 0 >>> environ.getenv('USER') 'gilligan' >>> environ._ _name_ _, environ._ _file_ _, environ ('environ', 'environ.py', <module 'environ' from 'environ.py'>) >>> dir(environ) [ ... '_environ', 'getenv', 'putenv' ... ]