ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Sat, 19 Aug 2023 05:04:44 +0200Problems with memory usage with bliss algorithmhttps://ask.sagemath.org/question/72676/problems-with-memory-usage-with-bliss-algorithm/I am trying to run a very long (weeks long) process that constructs a large collection of graphs.
The collection is too large to keep in memory and so as each graph is found it is stored on disk (rather than in an array).
However when I set it running, I noticed that the amount of RAM used by the process was continuously increasing until the rest of the machine ground to a halt. My computer has 64Gb RAM and it takes a bit more than a day for the process to grow to that size.
I have tried to isolate the part of the program that was causing the memory usage to grow, and I think that I have isolated it to the canonical labelling algorithm "bliss".
Here is a simple program that just creates random graphs and canonically labels them (but does nothing more):
for it in range(1000000):
g = graphs.RandomGNP(20,0.5)
gc = g.canonical_label(algorithm="bliss")
if it % 1000 == 0:
print(it)
When I run this program the memory consumed grows and grows.
If I change the program to use the built-in Sage method,
for it in range(1000000):
g = graphs.RandomGNP(20,0.5)
gc = g.canonical_label(algorithm="sage")
if it % 1000 == 0:
print(it)
then the program runs to completion with no increase in memory used - the memory is correctly being recovered as it goes out of scope each time through the loop.
What should I do in this situation?
Fri, 18 Aug 2023 04:24:53 +0200https://ask.sagemath.org/question/72676/problems-with-memory-usage-with-bliss-algorithm/Answer by rburing for <p>I am trying to run a very long (weeks long) process that constructs a large collection of graphs.</p>
<p>The collection is too large to keep in memory and so as each graph is found it is stored on disk (rather than in an array).</p>
<p>However when I set it running, I noticed that the amount of RAM used by the process was continuously increasing until the rest of the machine ground to a halt. My computer has 64Gb RAM and it takes a bit more than a day for the process to grow to that size.</p>
<p>I have tried to isolate the part of the program that was causing the memory usage to grow, and I think that I have isolated it to the canonical labelling algorithm "bliss".</p>
<p>Here is a simple program that just creates random graphs and canonically labels them (but does nothing more):</p>
<pre><code>for it in range(1000000):
g = graphs.RandomGNP(20,0.5)
gc = g.canonical_label(algorithm="bliss")
if it % 1000 == 0:
print(it)
</code></pre>
<p>When I run this program the memory consumed grows and grows.</p>
<p>If I change the program to use the built-in Sage method,</p>
<pre><code>for it in range(1000000):
g = graphs.RandomGNP(20,0.5)
gc = g.canonical_label(algorithm="sage")
if it % 1000 == 0:
print(it)
</code></pre>
<p>then the program runs to completion with no increase in memory used - the memory is correctly being recovered as it goes out of scope each time through the loop.</p>
<p>What should I do in this situation?</p>
https://ask.sagemath.org/question/72676/problems-with-memory-usage-with-bliss-algorithm/?answer=72683#post-id-72683Which version of SageMath are you using? I remember [investigating this over a year ago](https://groups.google.com/g/sage-devel/c/mTqUg93Ei1I/m/951jcu1WIAAJ) and the issue [#33010](https://github.com/sagemath/sage/issues/33010) should be fixed by [#35344](https://github.com/sagemath/sage/pull/35344) which is in 10.1.beta5 and later.
Please try the latest pre-release (10.1.rc0 at the time of writing) and check that the issue is fixed.Fri, 18 Aug 2023 08:48:01 +0200https://ask.sagemath.org/question/72676/problems-with-memory-usage-with-bliss-algorithm/?answer=72683#post-id-72683Comment by Gordon for <p>Which version of SageMath are you using? I remember <a href="https://groups.google.com/g/sage-devel/c/mTqUg93Ei1I/m/951jcu1WIAAJ">investigating this over a year ago</a> and the issue <a href="https://github.com/sagemath/sage/issues/33010">#33010</a> should be fixed by <a href="https://github.com/sagemath/sage/pull/35344">#35344</a> which is in 10.1.beta5 and later.</p>
<p>Please try the latest pre-release (10.1.rc0 at the time of writing) and check that the issue is fixed.</p>
https://ask.sagemath.org/question/72676/problems-with-memory-usage-with-bliss-algorithm/?comment=72733#post-id-72733Excellent - 10.1 does not display the memory leak for bliss, so I'll restart the program and see if it can catch up with the running version using the Sage algorithm. Thanks.Sat, 19 Aug 2023 05:04:44 +0200https://ask.sagemath.org/question/72676/problems-with-memory-usage-with-bliss-algorithm/?comment=72733#post-id-72733Comment by Gordon for <p>Which version of SageMath are you using? I remember <a href="https://groups.google.com/g/sage-devel/c/mTqUg93Ei1I/m/951jcu1WIAAJ">investigating this over a year ago</a> and the issue <a href="https://github.com/sagemath/sage/issues/33010">#33010</a> should be fixed by <a href="https://github.com/sagemath/sage/pull/35344">#35344</a> which is in 10.1.beta5 and later.</p>
<p>Please try the latest pre-release (10.1.rc0 at the time of writing) and check that the issue is fixed.</p>
https://ask.sagemath.org/question/72676/problems-with-memory-usage-with-bliss-algorithm/?comment=72684#post-id-72684I use the most recent MacOs X binary version as provided at https://github.com/3-manifolds/Sage_macOS/releases which, by Murphy's Law is 10.0. They also supply a pre-release version, currently 10.1, so I will try that and see what happens.Fri, 18 Aug 2023 09:09:46 +0200https://ask.sagemath.org/question/72676/problems-with-memory-usage-with-bliss-algorithm/?comment=72684#post-id-72684