Recipe20.17.Solving Metaclass Conflicts

Recipe 20.17. Solving Metaclass Conflicts

Credit: Michele Simionato, David Mertz, Phillip J. Eby, Alex Martelli, Anna Martelli Ravenscroft

Problem

You need to multiply inherit from several classes that may come from several metaclasses, so you need to generate automatically a custom metaclass to solve any possible metaclass conflicts.

Solution

First of all, given a sequence of metaclasses, we want to filter out "redundant" onesthose that are already implied by others, being duplicates or superclasses. This job nicely factors into a general-purpose generator yielding the unique, nonredundant items of an iterable, and a function using inspect.getmro to make the set of all superclasses of the given classes (since superclasses are redundant):

# support 2.3, too try: set except NameError: from sets import Set as set # support classic classes, to some extent import types def uniques(sequence, skipset):     for item in sequence:         if item not in skipset:             yield item             skipset.add(item) import inspect def remove_redundant(classes):     redundant = set([types.ClassType])   # turn old-style classes to new     for c in classes:         redundant.update(inspect.getmro(c)[1:])     return tuple(uniques(classes, redundant))

Using the remove_redundant function, we can generate a metaclass that can resolve metatype conflicts (given a sequence of base classes, and other metaclasses to inject both before and after those implied by the base classes). It's important to avoid generating more than one metaclass to solve the same potential conflicts, so we also keep a "memoization" mapping:

memoized_metaclasses_map = {  } def _get_noconflict_metaclass(bases, left_metas, right_metas):      # make tuple of needed metaclasses in specified order      metas = left_metas + tuple(map(type, bases)) + right_metas      needed_metas = remove_redundant(metas)      # return existing confict-solving meta, if any      try: return memoized_metaclasses_map[needed_metas]      except KeyError: pass      # compute, memoize and return needed conflict-solving meta      if not needed_metas:         # whee, a trivial case, happy us          meta = type      elif len(needed_metas) == 1: # another trivial case          meta = needed_metas[0]      else:                        # nontrivial, darn, gotta work...          # ward against non-type root metatypes          for m in needed_metas:              if not issubclass(m, type):                  raise TypeError( 'Non-type root metatype %r' % m)          metaname = '_' + ''.join([m._ _name_ _ for m in needed_metas])          meta = classmaker( )(metaname, needed_metas, {  })      memoized_metaclasses_map[needed_metas] = meta      return meta def classmaker(left_metas=( ), right_metas=( )):      def make_class(name, bases, adict):          metaclass = _get_noconflict_metaclass(bases, left_metas, right_metas)          return metaclass(name, bases, adict)      return make_class

The internal _get_noconflict_metaclass function, which returns (and, if needed, builds) the conflict-resolution metaclass, and the public classmaker closure must be mutually recursive for a rather subtle reason. If _get_noconflict_metaclass just built the metaclass with the reasonably common idiom:

         meta = type(metaname, needed_metas, {  })

it would work in all ordinary cases, but it might get into trouble when the metaclasses involved have custom metaclasses themselves! Just like "little fleas have lesser fleas," so, potentially, metaclasses can have meta-metaclasses, and so onfortunately not "ad infinitum," pace Augustus De Morgan, so the mutual recursion does eventually terminate.

The recipe offers minimal support for old-style (i.e., classic) classes, with the simple expedient of initializing the set redundant to contain the metaclass of old-style classes, types.ClassType. In practice, this recipe imposes automatic conversion to new-style classes. Trying to offer more support than this for classic classes, which are after all a mere legacy feature, would be overkill, given the confused and confusing situation of metaclass use for old-style classes.

In all of our code outside of this noconflict.py module, we will only use noconflict.classmaker, optionally passing it metaclasses we want to inject, left and right, to obtain a callable that we can then use just like a metaclass to build new class objects given names, bases, and dictionary, but with the assurance that metatype conflicts cannot occur. Phew. Now that was worth it, wasn't it?!

Discussion

Here is the simplest case in which a metatype conflict can occur: multiply inheriting from two classes with independent metaclasses. In a pedagogically simplified toy-level example, that could be, say:

>>> class Meta_A(type): pass ...  >>> class Meta_B(type): pass ...  >>> class A: _ _metaclass_ _ = Meta_A ...  >>> class B: _ _metaclass_ _ = Meta_B ...  >>> class C(A, B): pass Traceback (most recent call last):   File "<stdin>", line 1, in ? TypeError: Error when calling the metaclass bases     metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases >>>

A class normally inherits its metaclass from its bases, but when the bases have distinct metaclasses, the metatype constraint that Python expresses so tersely in this error message applies. So, we need to build a new metaclass, say Meta_C, which inherits from both Meta_A and Meta_B. For a demonstration of this need, see the book that's so aptly considered the bible of metaclasses: Ira R. Forman and Scott H. Danforth, Putting Metaclasses to Work: A New Dimension in Object-Oriented Programming (Addison-Wesley).

Python does not do magic: it does not automatically create the required Meta_C. Rather, Python raises a TypeError to ensure that the programmer is aware of the problem. In simple cases, the programmer can solve the metatype conflict by hand, as follows:

>>> class Meta_C(Meta_A, Meta_B): pass >>> class C(A, B): _ _metaclass_ _ = Meta_C

In this case, everything works smoothly.

The key point of this recipe is to show an automatic way to resolve metatype conflicts, rather than having to do it by hand every time. Having saved all the code from this recipe's Solution into noconflict.py somewhere along your Python sys.path, you can make class C with automatic conflict resolution, as follows:

>>> import noconflict >>> class C(A, B): _ _metaclass_ _ = noconflict.classmaker( )

The call to the noconflict.classmaker closure returns a function that, when Python calls it, obtains the proper metaclass and uses it to build the class object. It cannot yet return the metaclass itself, but that's OKyou can assign anything you want to the _ _metaclass_ _ attribute of your class, as long as it's callable with the (name, bases, dict) arguments and nicely builds the class object. Once again, Python's signature-based polymorphism serves us well and unobtrusively.

Automating the resolution of the metatype conflict has many pluses, even in simple cases. Thanks to the "memoizing" technique used in noconflict.py, the same conflict-resolving metaclass is used for any occurrence of a given sequence of conflicting metaclasses. Moreover, with this approach you may also explicitly inject other metaclasses, beyond those you get from your base classes, and again you can avoid conflicts. Consider:

>>> class D(A): _ _metaclass_ _ = Meta_B Traceback (most recent call last):   File "<stdin>", line 1, in ? TypeError: Error when calling the metaclass bases     metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

This metatype conflict is resolved just as easily as the former one:

>>> class D(A): _ _metaclass_ _ = noconflict.classmaker((Meta_B,))

The code presented in this recipe's Solution takes pains to avoid any subclassing that is not strictly necessary, and it also uses mutual recursion to avoid any meta-level of meta-meta-type conflicts. You might never meet higher-order-meta conflicts anyway, but if you adopt the code presented in this recipe, you need not even worry about them.

Thanks to David Mertz for help in polishing the original version of the code. This version has benefited immensely from discussions with Phillip J. Eby. Alex Martelli and Anna Martelli Ravenscroft did their best to make the recipe's code and discussion as explicit and understandable as they could. The functionality in this recipe is not absolutely complete: for example, it supports old-style classes only in a rather backhanded way, and it does not really cover such exotica as nontype metatype roots (such as Zope 2's old ExtensionClass). These limitations are there primarily to keep this recipe as understandable as possible. You may find a more complete implementation of metatype conflict resolution at Phillip J. Eby's PEAK site, http://peak.telecommunity.com/, in the peak.util.Meta module of the PEAK framework.

Recipe20.17.Solving Metaclass Conflicts