ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Sun, 19 Sep 2010 18:06:52 +0200What is the standard pickle jar for and how do I update ithttps://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/I started Trac #9907, which basically consists of a relocation of some general decorators into `sage.misc.decorators`. However, after applying the patch, the doctest for `sage.structure.sage_object.unpickle_all` fails. It seems to be because there is a standard pickle jar which still contains the information that the decorators are located in `sage.plot.misc`.
Is it correct that there is such a pickle jar? In that case, what for, and how do I get about updating it and fixing my patch?
Cheers, JohanFri, 17 Sep 2010 05:32:18 +0200https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/Answer by niles for <p>I started Trac #9907, which basically consists of a relocation of some general decorators into <code>sage.misc.decorators</code>. However, after applying the patch, the doctest for <code>sage.structure.sage_object.unpickle_all</code> fails. It seems to be because there is a standard pickle jar which still contains the information that the decorators are located in <code>sage.plot.misc</code>.</p>
<p>Is it correct that there is such a pickle jar? In that case, what for, and how do I get about updating it and fixing my patch?</p>
<p>Cheers, Johan</p>
https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?answer=11659#post-id-11659Have you looked at the examples section of [`unpickle_all`][1]? I noticed the following there:
> We unpickle the standard pickle jar. This doctest tests that all standard pickles unpickle. Every so often the standard pickle jar should be updated by running the doctest suite with the environment variable `SAGE_PICKLE_JAR` set, then copying the files from `SAGE_ROOT/tmp/pickle_jar*` into the standard pickle jar.
But I don't know much about pickling, so I don't know if this actually answers your question! Anyway, I hope it's helpful :)
----
EDIT: Judging by the docstring, I think the "standard pickle jar" is the contents of `SAGE_DATA/extcode/pickle_jar/pickle_jar.tar.bz2`:
sage: std = os.environ['SAGE_DATA'] + '/extcode/pickle_jar/pickle_jar.tar.bz2'
sage: std
'/Applications/sage/data/extcode/pickle_jar/pickle_jar.tar.bz2'
testing this:
<pre>
$ cd $SAGE_ROOT/devel/sage
$ export SAGE_PICKLE_JAR="pickle"
$ sage -t sage/rings/power_series_ring*
sage -t "devel/sage-main/sage/rings/power_series_ring.py"
[4.3 s]
sage -t "devel/sage-main/sage/rings/power_series_ring_element.pyx"
[3.3 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 7.7 seconds
$ ls ../../tmp/pickle_jar*
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.txt
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.txt
_type__sage_rings_power_series_poly_PowerSeries_poly__.sobj
_type__sage_rings_power_series_poly_PowerSeries_poly__.txt
</pre>
And this looks a lot like the contents of `pickle_jar.tar.bz2`.
So I guess all you have to do is add whatever new things are generated by your code to the pickle jar file (or maybe replace existing things with the new things).
[1]: http://sagemath.org/doc/reference/sage/structure/sage_object.html#sage.structure.sage_object.unpickle_all
[2]: http://sagemath.org/doc/reference/sage/structure/sage_object.html#sage.structure.sage_object.unpickle_allFri, 17 Sep 2010 08:39:49 +0200https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?answer=11659#post-id-11659Comment by Mike Hansen for <p>Have you looked at the examples section of <a href="http://sagemath.org/doc/reference/sage/structure/sage_object.html#sage.structure.sage_object.unpickle_all"><code>unpickle_all</code></a>? I noticed the following there:</p>
<blockquote>
<p>We unpickle the standard pickle jar. This doctest tests that all standard pickles unpickle. Every so often the standard pickle jar should be updated by running the doctest suite with the environment variable <code>SAGE_PICKLE_JAR</code> set, then copying the files from <code>SAGE_ROOT/tmp/pickle_jar*</code> into the standard pickle jar.</p>
</blockquote>
<p>But I don't know much about pickling, so I don't know if this actually answers your question! Anyway, I hope it's helpful :)</p>
<hr/>
<p>EDIT: Judging by the docstring, I think the "standard pickle jar" is the contents of <code>SAGE_DATA/extcode/pickle_jar/pickle_jar.tar.bz2</code>:</p>
<pre><code>sage: std = os.environ['SAGE_DATA'] + '/extcode/pickle_jar/pickle_jar.tar.bz2'
sage: std
'/Applications/sage/data/extcode/pickle_jar/pickle_jar.tar.bz2'
</code></pre>
<p>testing this:</p>
<pre>$ cd $SAGE_ROOT/devel/sage
$ export SAGE_PICKLE_JAR="pickle"
$ sage -t sage/rings/power_series_ring*
sage -t "devel/sage-main/sage/rings/power_series_ring.py"
[4.3 s]
sage -t "devel/sage-main/sage/rings/power_series_ring_element.pyx"
[3.3 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 7.7 seconds
$ ls ../../tmp/pickle_jar*
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.txt
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.txt
_type__sage_rings_power_series_poly_PowerSeries_poly__.sobj
_type__sage_rings_power_series_poly_PowerSeries_poly__.txt
</pre>
<p>And this looks a lot like the contents of <code>pickle_jar.tar.bz2</code>.</p>
<p>So I guess all you have to do is add whatever new things are generated by your code to the pickle jar file (or maybe replace existing things with the new things).</p>
https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?comment=22626#post-id-22626See my answer -- you don't want to make a new pickle jar unless you explicitly decide that you need to break old pickles. This should not be done lightly, especially when there is a trival fix.Sun, 19 Sep 2010 18:06:52 +0200https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?comment=22626#post-id-22626Comment by jsrn for <p>Have you looked at the examples section of <a href="http://sagemath.org/doc/reference/sage/structure/sage_object.html#sage.structure.sage_object.unpickle_all"><code>unpickle_all</code></a>? I noticed the following there:</p>
<blockquote>
<p>We unpickle the standard pickle jar. This doctest tests that all standard pickles unpickle. Every so often the standard pickle jar should be updated by running the doctest suite with the environment variable <code>SAGE_PICKLE_JAR</code> set, then copying the files from <code>SAGE_ROOT/tmp/pickle_jar*</code> into the standard pickle jar.</p>
</blockquote>
<p>But I don't know much about pickling, so I don't know if this actually answers your question! Anyway, I hope it's helpful :)</p>
<hr/>
<p>EDIT: Judging by the docstring, I think the "standard pickle jar" is the contents of <code>SAGE_DATA/extcode/pickle_jar/pickle_jar.tar.bz2</code>:</p>
<pre><code>sage: std = os.environ['SAGE_DATA'] + '/extcode/pickle_jar/pickle_jar.tar.bz2'
sage: std
'/Applications/sage/data/extcode/pickle_jar/pickle_jar.tar.bz2'
</code></pre>
<p>testing this:</p>
<pre>$ cd $SAGE_ROOT/devel/sage
$ export SAGE_PICKLE_JAR="pickle"
$ sage -t sage/rings/power_series_ring*
sage -t "devel/sage-main/sage/rings/power_series_ring.py"
[4.3 s]
sage -t "devel/sage-main/sage/rings/power_series_ring_element.pyx"
[3.3 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 7.7 seconds
$ ls ../../tmp/pickle_jar*
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.txt
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.txt
_type__sage_rings_power_series_poly_PowerSeries_poly__.sobj
_type__sage_rings_power_series_poly_PowerSeries_poly__.txt
</pre>
<p>And this looks a lot like the contents of <code>pickle_jar.tar.bz2</code>.</p>
<p>So I guess all you have to do is add whatever new things are generated by your code to the pickle jar file (or maybe replace existing things with the new things).</p>
https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?comment=22638#post-id-22638Thanks for the reference Niles, but yes, I have looked at that example, but I didn't really understand it; first off, I don't know where the standard pickle jar is :)Fri, 17 Sep 2010 10:11:19 +0200https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?comment=22638#post-id-22638Comment by niles for <p>Have you looked at the examples section of <a href="http://sagemath.org/doc/reference/sage/structure/sage_object.html#sage.structure.sage_object.unpickle_all"><code>unpickle_all</code></a>? I noticed the following there:</p>
<blockquote>
<p>We unpickle the standard pickle jar. This doctest tests that all standard pickles unpickle. Every so often the standard pickle jar should be updated by running the doctest suite with the environment variable <code>SAGE_PICKLE_JAR</code> set, then copying the files from <code>SAGE_ROOT/tmp/pickle_jar*</code> into the standard pickle jar.</p>
</blockquote>
<p>But I don't know much about pickling, so I don't know if this actually answers your question! Anyway, I hope it's helpful :)</p>
<hr/>
<p>EDIT: Judging by the docstring, I think the "standard pickle jar" is the contents of <code>SAGE_DATA/extcode/pickle_jar/pickle_jar.tar.bz2</code>:</p>
<pre><code>sage: std = os.environ['SAGE_DATA'] + '/extcode/pickle_jar/pickle_jar.tar.bz2'
sage: std
'/Applications/sage/data/extcode/pickle_jar/pickle_jar.tar.bz2'
</code></pre>
<p>testing this:</p>
<pre>$ cd $SAGE_ROOT/devel/sage
$ export SAGE_PICKLE_JAR="pickle"
$ sage -t sage/rings/power_series_ring*
sage -t "devel/sage-main/sage/rings/power_series_ring.py"
[4.3 s]
sage -t "devel/sage-main/sage/rings/power_series_ring_element.pyx"
[3.3 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 7.7 seconds
$ ls ../../tmp/pickle_jar*
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.txt
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.txt
_type__sage_rings_power_series_poly_PowerSeries_poly__.sobj
_type__sage_rings_power_series_poly_PowerSeries_poly__.txt
</pre>
<p>And this looks a lot like the contents of <code>pickle_jar.tar.bz2</code>.</p>
<p>So I guess all you have to do is add whatever new things are generated by your code to the pickle jar file (or maybe replace existing things with the new things).</p>
https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?comment=22635#post-id-22635I guess I don't know what you should put up with the patch . . . you're probably right, and reviewers can ask you to upload the file if they want. On the other hand, you have to make the file anyway to test your code, so you might as well upload it and make life a little easier for everyone :)Fri, 17 Sep 2010 17:12:42 +0200https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?comment=22635#post-id-22635Comment by jsrn for <p>Have you looked at the examples section of <a href="http://sagemath.org/doc/reference/sage/structure/sage_object.html#sage.structure.sage_object.unpickle_all"><code>unpickle_all</code></a>? I noticed the following there:</p>
<blockquote>
<p>We unpickle the standard pickle jar. This doctest tests that all standard pickles unpickle. Every so often the standard pickle jar should be updated by running the doctest suite with the environment variable <code>SAGE_PICKLE_JAR</code> set, then copying the files from <code>SAGE_ROOT/tmp/pickle_jar*</code> into the standard pickle jar.</p>
</blockquote>
<p>But I don't know much about pickling, so I don't know if this actually answers your question! Anyway, I hope it's helpful :)</p>
<hr/>
<p>EDIT: Judging by the docstring, I think the "standard pickle jar" is the contents of <code>SAGE_DATA/extcode/pickle_jar/pickle_jar.tar.bz2</code>:</p>
<pre><code>sage: std = os.environ['SAGE_DATA'] + '/extcode/pickle_jar/pickle_jar.tar.bz2'
sage: std
'/Applications/sage/data/extcode/pickle_jar/pickle_jar.tar.bz2'
</code></pre>
<p>testing this:</p>
<pre>$ cd $SAGE_ROOT/devel/sage
$ export SAGE_PICKLE_JAR="pickle"
$ sage -t sage/rings/power_series_ring*
sage -t "devel/sage-main/sage/rings/power_series_ring.py"
[4.3 s]
sage -t "devel/sage-main/sage/rings/power_series_ring_element.pyx"
[3.3 s]
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 7.7 seconds
$ ls ../../tmp/pickle_jar*
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_domain__.txt
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.sobj
_class__sage_rings_power_series_ring_PowerSeriesRing_over_field__.txt
_type__sage_rings_power_series_poly_PowerSeries_poly__.sobj
_type__sage_rings_power_series_poly_PowerSeries_poly__.txt
</pre>
<p>And this looks a lot like the contents of <code>pickle_jar.tar.bz2</code>.</p>
<p>So I guess all you have to do is add whatever new things are generated by your code to the pickle jar file (or maybe replace existing things with the new things).</p>
https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?comment=22637#post-id-22637Ok, that would be great. So in an actual patch, I would also post a whole new piclee_jar.tar.bz2 file with the patch? Or I would leave a note with the patch for reviewers and the release manager that this picle_jar.tar.bz2 should be remade? Probably the latter...Fri, 17 Sep 2010 16:51:27 +0200https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?comment=22637#post-id-22637Answer by Mike Hansen for <p>I started Trac #9907, which basically consists of a relocation of some general decorators into <code>sage.misc.decorators</code>. However, after applying the patch, the doctest for <code>sage.structure.sage_object.unpickle_all</code> fails. It seems to be because there is a standard pickle jar which still contains the information that the decorators are located in <code>sage.plot.misc</code>.</p>
<p>Is it correct that there is such a pickle jar? In that case, what for, and how do I get about updating it and fixing my patch?</p>
<p>Cheers, Johan</p>
https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?answer=11670#post-id-11670The pickle jar contains old pickles of objects that we want to ensure work in the future. For example, if you made the changes at [9907][1] as the are, then possibly many pickles out in the wild would break. Instead of trying to update the pickle jar, the appropriate thing to do would be to add the following to `sage/plot/misc.py`
#For backward compatibility -- see #9907.
from sage.misc.decorators import options, suboptions, rename_keyword
That way, the old pickles will still work as they will still be able to find the decorators in `sage.plot.misc`.
[1]: http://trac.sagemath.org/sage_trac/ticket/9907Sun, 19 Sep 2010 17:43:31 +0200https://ask.sagemath.org/question/7692/what-is-the-standard-pickle-jar-for-and-how-do-i-update-it/?answer=11670#post-id-11670