Recipe17.2.Implementing a Simple Extension Type with Pyrex

Recipe 17.2. Implementing a Simple Extension Type with Pyrex

Credit: Alex Martelli

Problem

You want to code and build an extension type for Python with a minimal amount of hard work.

Solution

The Pyrex language is the simplest and fastest way to build Python extensions. Once we have Pyrex installed, the next step is to create a setup.py file to use the distutils package to build and install our module:

from distutils.core import setup, Extension from Pyrex.Distutils import build_ext setup(name = "elemlist",       version = "1.0",       maintainer = "Alex Martelli",       maintainer_email = "amcx@aleax.it",       description = "Simple Python extension module in Pyrex",       ext_modules = [Extension('elemlist',sources=['elemlist.pyx'])],       cmdclass = {'build_ext': build_ext}, )

Then, we need a file elemlist.pyx with our module's source code:

cdef class cons:     cdef public object car, cdr     def _ _init_ _(self, car, cdr):         self.car = car         self.cdr = cdr     def _ _repr_ _(self):         return 'cons(%r, %r)' % (self.car, self.cdr)

Discussion

Pyrex is a language for writing Python extension modules. It was developed by Greg Ewing and is freely downloadable and installable. The Pyrex language is a large subset of Python, with the addition of a few language constructs to allow easy generation of fast C code. In this recipe, the only special Pyrex construct we use is the cdef statement, which lets us express C-level declarations.

This module is roughly equivalent to the Python-coded module:

class cons(object):     _ _slots_ _ = ('car', 'cdr')     def _ _init_ _(self, car, cdr):         self.car = car         self.cdr = cdr     def _ _repr_ _(self):         return 'cons(%r, %r)' % (self.car, self.cdr)

As you see, Pyrex code is very similar to Python code. Pyrex code gets compiled into C, which in turn gets compiled to machine code, while Python code is compiled into bytecode instead. For such a simple type as cons, the Pyrex version is not much faster than the Python version, but a pure C version, such as the one that I demonstrated previously in Recipe 17.1, despite having 25 times as much code, wouldn't be any faster either.

Building a compiled extension module is just as simple when it's coded in Pyrex as when it's coded directly in C, thanks to the distutils package, which does all of the hard work. (You need to have Pyrex installed.) When you build a Pyrex-coded module, you may get warnings from your C compiler about symbols with names starting with _ _pyx or _ _pyx that are defined but not used, or declared but not defined. Do not let these warning messages worry you: your C compiler is running with the highest possible level of warnings, and the little anomalies it's warning about are just perfectly normal and innocuous artifacts in the C sources that Pyrex automatically generates. Pyrex is not quite finished yet (the Pyrex version at the time of writing is 0.9.3), so no attention has yet been spent on purely cosmetic warts. (By the time you read this, a mature version of Pyrex may have been released, with all i's dotted and all t's crossed. Nevertheless, I would recommend Pyrex even if the latest version still causes numerous warnings.)

Installing Pyrex

To use Pyrex, you need to download and install it (http://nz.cosc.canterbury.ac.nz/~greg/python/Pyrex/), and you also need to have a C compiler. Pyrex translates your .pyx source into C source and then uses your C compiler to make from that C source a machine-code Python extension module (a .pyd file on Windows, a .so file on Linux, a .dynlib file on the Mac, etc.). Installing Pyrex itself is a snap: unpack the .tar.gz file, cd with the shell of your choice into the directory thus made, and at the shell prompt type the usual command to install any Python module: python setup.py install. Just as for any other Python module, you may need "root" or "administrator" privileges to install Pyrex, depending on your platform and on the details of your Python installation. In the directory where you unpacked Pyrex's .tar.gz archive, you will also find abundant documentation and examples, particularly in subdirectories Doc and Demos.

Recipe17.2.Implementing a Simple Extension Type with Pyrex