The copy_reg Module

The copy_reg module provides a registry that you can use to register your own extension types. The pickle and copy modules use this registry to figure out how to process non-standard types.

For example, the standard pickle implementation cannot deal with Python code objects, as shown in the following example:

File: copy-reg-example-1.py

import pickle

CODE = """
print 'good evening'
"""

code = compile(CODE, "", "exec")

exec code
exec pickle.loads(pickle.dumps(code))

good evening
Traceback (innermost last):
...
pickle.PicklingError: can't pickle 'code' objects

We can work around this by registering a code object handler. Such a handler consists of two parts: a pickler, which takes the code object and returns a tuple that can only contain simple datatypes, and an unpickler, which takes the contents of such a tuple as its arguments. Example 4-14 demonstrates this.

Example 4-14. Using the copy_reg Module to Enable Pickling of Code Objects

File: copy-reg-example-2.py

import copy_reg
import pickle, marshal, types

#
# register a pickle handler for code objects

def code_unpickler(data):
 return marshal.loads(data)

def code_pickler(code):
 return code_unpickler, (marshal.dumps(code),)

copy_reg.pickle(types.CodeType, code_pickler, code_unpickler)

#
# try it out

CODE = """
print "suppose he's got a pointed stick"
"""

code = compile(CODE, "", "exec")

exec code
exec pickle.loads(pickle.dumps(code))

suppose he's got a pointed stick
suppose he's got a pointed stick

If you're transferring the pickled data across a network or to another program, the custom unpickler must be available at the receiving end as well.

For the really adventurous, Example 4-15 shows a version that allows you to pickle open file objects.

Example 4-15. Using the copy_reg Module to Enable Pickling of File Objects

File: copy-reg-example-3.py

import copy_reg
import pickle, types
import StringIO

#
# register a pickle handler for file objects

def file_unpickler(position, data):
 file = StringIO.StringIO(data)
 file.seek(position)
 return file

def file_pickler(code):
 position = file.tell()
 file.seek(0)
 data = file.read()
 file.seek(position)
 return file_unpickler, (position, data)

copy_reg.pickle(types.FileType, file_pickler, file_unpickler)

#
# try it out

file = open("samples/sample.txt", "rb")

print file.read(120),
print "",
print pickle.loads(pickle.dumps(file)).read()

We will perhaps eventually be writing only small
modules, which are identified by name as they are
used to build larger  ones, so that devices like
indentation, rather than delimiters, might become
feasible for expressing local structure in the
source language.
 -- Donald E. Knuth, December 1974