Ask Your Question

pickling extension classes

asked 2011-10-13 17:47:27 -0500

Hartmut Monien gravatar image

updated 2011-10-13 17:49:39 -0500

What is the right way to pickle a cython extension class? Consider the example

cdef class Stuff:
 def __init__(self):


a = Stuff()
a == loads(dumps(a))

gives the result False because the address of the new object generated by loads is different from the original object a.

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted

answered 2011-10-13 19:56:31 -0500

Simon King gravatar image

Even when you had a Python class,

would not evaluate as true, because the objects loads(dumps(a)) is a new object, thus, id(loads(dumps(a))) will differ from id(a).

So, if one sees your problem as a Python/Cython problem, then there are two parts of your problem (which are mostly answered by the documentation of Python, by the way). If you work in Sage, some related tools are available, so that your problem has a third part.

The first part is comparison of objects. For that purpose, Python (and Cython) use methods such as __cmp__ and __richcmp__ (see Python docs). If they are not provided, comparison will simply compare id(...).

The second part is pickling/serialisation. Python offers different serialisation protocols (again: see Python docs).

If you have Python classes, you may use the first protocol, which relies on methods like __getstate__, __setstate__, __getinitargs__, __getnewargs__. But for extension classes you have to use the second protocol: You need to provide a __reduce__ method, returning a function f and arguments args to that function, so that f(*args) will return (a copy of) the original object.

Here is the third part of your problem: You work in Sage. You know that they say "we don't re-invent the wheel, we build the car". Thus, it makes sense to use Sage's infrastructure, rather than programming everything from scratch.

For example, when you want to create a class whose instances are "unique" (meaning: A loads(dumps(...)) copy of the instance will not only be equal but identical with the instance), you can simply inherit from sage.structure.unique_representation.UniqueRepresentation. In more complicated cases (or with Cython classes), you can also use sage.structure.factory.UniqueFactory, which does not seem to appear in the reference manual, but you can of course still browse its documentation in a Sage session.

If your intension is to implement algebraic structures, you are likely to derive your classes from sage.structure.element.Element or sage.structure.parent.Parent (or from existing sub-classes; also see this worksheet). In that case, you should look into the source code, near to the __cmp__ and __richcmp__ methods: You will find comments that explain what methods you should provide for comparison.

edit flag offensive delete link more


Well said. Just as a rule of thumb, you always need __reduce__ for Cython classes. And if your class does not inherit comparison, you need _cmp_ as well.

Volker Braun gravatar imageVolker Braun ( 2011-10-14 05:01:37 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools


Asked: 2011-10-13 17:47:27 -0500

Seen: 556 times

Last updated: Oct 13 '11