ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Fri, 14 Oct 2011 12:01:37 +0200pickling extension classeshttps://ask.sagemath.org/question/8376/pickling-extension-classes/What is the right way to pickle a cython extension class? Consider the example
cdef class Stuff:
def __init__(self):
pass
then
a = Stuff()
a == loads(dumps(a))
gives the result **False** because the address of the new object generated by *loads* is different from the original object *a*.
Fri, 14 Oct 2011 00:47:27 +0200https://ask.sagemath.org/question/8376/pickling-extension-classes/Answer by Simon King for <p>What is the right way to pickle a cython extension class? Consider the example</p>
<pre><code>cdef class Stuff:
def __init__(self):
pass
</code></pre>
<p>then </p>
<pre><code>a = Stuff()
a == loads(dumps(a))
</code></pre>
<p>gives the result <strong>False</strong> because the address of the new object generated by <em>loads</em> is different from the original object <em>a</em>.</p>
https://ask.sagemath.org/question/8376/pickling-extension-classes/?answer=12755#post-id-12755Even when you had a Python class, <pre>loads(dumps(a))==a</pre> would not evaluate as true, because the objects `loads(dumps(a))` is a new object, thus, `id(loads(dumps(a)))` will differ from `id(a)`.
So, if one sees your problem as a Python/Cython problem, then there are two parts of your problem (which are mostly answered by the documentation of Python, by the way). If you work in Sage, some related tools are available, so that your problem has a third part.
The first part is comparison of objects. For that purpose, Python (and Cython) use methods such as `__cmp__` and `__richcmp__` (see [Python docs](http://docs.python.org/reference/datamodel.html "Python data model")). If they are not provided, comparison will simply compare `id(...)`.
The second part is pickling/serialisation. Python offers different serialisation protocols (again: see [Python docs](http://docs.python.org/library/pickle.html#the-pickle-protocol "Pickling")).
If you have Python classes, you may use the [first protocol](http://docs.python.org/library/pickle.html#pickling-and-unpickling-normal-class-instances "Pickling with pure Python"), which relies on methods like `__getstate__`, `__setstate__`, `__getinitargs__`, `__getnewargs__`. But for extension classes you have to use the [second protocol](http://docs.python.org/library/pickle.html#pickling-and-unpickling-extension-types "Pickling of extension types"): You need to provide a `__reduce__` method, returning a function `f` and arguments `args` to that function, so that `f(*args)` will return (a copy of) the original object.
Here is the third part of your problem: You work in Sage. You know that they say "we don't re-invent the wheel, we build the car". Thus, it makes sense to use Sage's infrastructure, rather than programming everything from scratch.
For example, when you want to create a class whose instances are "unique" (meaning: A `loads(dumps(...))` copy of the instance will not only be equal but identical with the instance), you can simply inherit from [`sage.structure.unique_representation.UniqueRepresentation`](http://www.sagemath.org/doc/reference/sage/structure/unique_representation.html "Unique representation"). In more complicated cases (or with Cython classes), you can also use `sage.structure.factory.UniqueFactory`, which does not seem to appear in the reference manual, but you can of course still browse its documentation in a Sage session.
If your intension is to implement algebraic structures, you are likely to derive your classes from `sage.structure.element.Element` or `sage.structure.parent.Parent` (or from existing sub-classes; also see [this worksheet](http://flask.sagenb.org/home/pub/82/ "How to implement algebraic structures in Sage")). In that case, you should look into the source code, near to the `__cmp__` and `__richcmp__` methods: You will find comments that explain what methods you should provide for comparison. Fri, 14 Oct 2011 02:56:31 +0200https://ask.sagemath.org/question/8376/pickling-extension-classes/?answer=12755#post-id-12755Comment by Volker Braun for <p>Even when you had a Python class, </p><pre>loads(dumps(a))==a</pre> would not evaluate as true, because the objects <code>loads(dumps(a))</code> is a new object, thus, <code>id(loads(dumps(a)))</code> will differ from <code>id(a)</code>.<p></p>
<p>So, if one sees your problem as a Python/Cython problem, then there are two parts of your problem (which are mostly answered by the documentation of Python, by the way). If you work in Sage, some related tools are available, so that your problem has a third part.</p>
<p>The first part is comparison of objects. For that purpose, Python (and Cython) use methods such as <code>__cmp__</code> and <code>__richcmp__</code> (see <a href="http://docs.python.org/reference/datamodel.html" title="Python data model">Python docs</a>). If they are not provided, comparison will simply compare <code>id(...)</code>.</p>
<p>The second part is pickling/serialisation. Python offers different serialisation protocols (again: see <a href="http://docs.python.org/library/pickle.html#the-pickle-protocol" title="Pickling">Python docs</a>).</p>
<p>If you have Python classes, you may use the <a href="http://docs.python.org/library/pickle.html#pickling-and-unpickling-normal-class-instances" title="Pickling with pure Python">first protocol</a>, which relies on methods like <code>__getstate__</code>, <code>__setstate__</code>, <code>__getinitargs__</code>, <code>__getnewargs__</code>. But for extension classes you have to use the <a href="http://docs.python.org/library/pickle.html#pickling-and-unpickling-extension-types" title="Pickling of extension types">second protocol</a>: You need to provide a <code>__reduce__</code> method, returning a function <code>f</code> and arguments <code>args</code> to that function, so that <code>f(*args)</code> will return (a copy of) the original object.</p>
<p>Here is the third part of your problem: You work in Sage. You know that they say "we don't re-invent the wheel, we build the car". Thus, it makes sense to use Sage's infrastructure, rather than programming everything from scratch.</p>
<p>For example, when you want to create a class whose instances are "unique" (meaning: A <code>loads(dumps(...))</code> copy of the instance will not only be equal but identical with the instance), you can simply inherit from <a href="http://www.sagemath.org/doc/reference/sage/structure/unique_representation.html" title="Unique representation"><code>sage.structure.unique_representation.UniqueRepresentation</code></a>. In more complicated cases (or with Cython classes), you can also use <code>sage.structure.factory.UniqueFactory</code>, which does not seem to appear in the reference manual, but you can of course still browse its documentation in a Sage session.</p>
<p>If your intension is to implement algebraic structures, you are likely to derive your classes from <code>sage.structure.element.Element</code> or <code>sage.structure.parent.Parent</code> (or from existing sub-classes; also see <a href="http://flask.sagenb.org/home/pub/82/" title="How to implement algebraic structures in Sage">this worksheet</a>). In that case, you should look into the source code, near to the <code>__cmp__</code> and <code>__richcmp__</code> methods: You will find comments that explain what methods you should provide for comparison. </p>
https://ask.sagemath.org/question/8376/pickling-extension-classes/?comment=21125#post-id-21125Well said. Just as a rule of thumb, you always need __reduce__ for Cython classes. And if your class does not inherit comparison, you need _cmp_ as well.Fri, 14 Oct 2011 12:01:37 +0200https://ask.sagemath.org/question/8376/pickling-extension-classes/?comment=21125#post-id-21125