ASKSAGE: Sage Q&A Forum - Individual question feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Fri, 22 Mar 2019 14:09:13 -0500Memory usage strictly increasing on Sage interactive shellhttps://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/
Hello,
I executed the following script on the Sage interactive shell on Ubuntu 18.04 (on SageMath version 8.7.beta5):
sage: import gc
sage: t = 100000
sage: l = [n for n in xrange(t,t+100)]
sage: ll = []
sage: for x in l:
....: s = 0
....: for n in xrange(0,x):
....: s += cos(RDF(2)*RDF(pi)*RDF(n)/RDF(x))
....: ll.append([x, s])
....: del(s)
....: print "memory usage: " + str(get_memory_usage()) + ", gc: " + str(gc.collect())
....:
memory usage: 5824.5546875, gc: 277
memory usage: 5828.3046875, gc: 0
memory usage: 5832.0546875, gc: 0
memory usage: 5836.0546875, gc: 0
:
memory usage: 6203.5546875, gc: 0
memory usage: 6207.5546875, gc: 0
As we can see the memory usages, it increases by about 4 MB per each step. Thus, after completed, it increased around 400 MB in total. It is frustrating since it eventually eats up all memories in my system as the size of the input list grows and gc.collect() does not seem to help.
I suspect that there were memory leaks or Sage(or Python) stored all the values of cos() and never freed them for some reasons while the shell not killed.
Can someone please explain what was going on and suggest how to avoid this unwanted memory consumption? Thank you in advance.Thu, 21 Mar 2019 16:57:45 -0500https://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/Comment by rburing for <p>Hello, </p>
<p>I executed the following script on the Sage interactive shell on Ubuntu 18.04 (on SageMath version 8.7.beta5):</p>
<pre><code>sage: import gc
sage: t = 100000
sage: l = [n for n in xrange(t,t+100)]
sage: ll = []
sage: for x in l:
....: s = 0
....: for n in xrange(0,x):
....: s += cos(RDF(2)*RDF(pi)*RDF(n)/RDF(x))
....: ll.append([x, s])
....: del(s)
....: print "memory usage: " + str(get_memory_usage()) + ", gc: " + str(gc.collect())
....:
memory usage: 5824.5546875, gc: 277
memory usage: 5828.3046875, gc: 0
memory usage: 5832.0546875, gc: 0
memory usage: 5836.0546875, gc: 0
:
memory usage: 6203.5546875, gc: 0
memory usage: 6207.5546875, gc: 0
</code></pre>
<p>As we can see the memory usages, it increases by about 4 MB per each step. Thus, after completed, it increased around 400 MB in total. It is frustrating since it eventually eats up all memories in my system as the size of the input list grows and gc.collect() does not seem to help. </p>
<p>I suspect that there were memory leaks or Sage(or Python) stored all the values of cos() and never freed them for some reasons while the shell not killed. </p>
<p>Can someone please explain what was going on and suggest how to avoid this unwanted memory consumption? Thank you in advance.</p>
https://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/?comment=45872#post-id-45872If you wait long enough (or run it on a faster computer) then it will work without leaks with `gp(pi)` as well. I will report the memory leak and post that as an answer.Fri, 22 Mar 2019 14:06:32 -0500https://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/?comment=45872#post-id-45872Comment by jb for <p>Hello, </p>
<p>I executed the following script on the Sage interactive shell on Ubuntu 18.04 (on SageMath version 8.7.beta5):</p>
<pre><code>sage: import gc
sage: t = 100000
sage: l = [n for n in xrange(t,t+100)]
sage: ll = []
sage: for x in l:
....: s = 0
....: for n in xrange(0,x):
....: s += cos(RDF(2)*RDF(pi)*RDF(n)/RDF(x))
....: ll.append([x, s])
....: del(s)
....: print "memory usage: " + str(get_memory_usage()) + ", gc: " + str(gc.collect())
....:
memory usage: 5824.5546875, gc: 277
memory usage: 5828.3046875, gc: 0
memory usage: 5832.0546875, gc: 0
memory usage: 5836.0546875, gc: 0
:
memory usage: 6203.5546875, gc: 0
memory usage: 6207.5546875, gc: 0
</code></pre>
<p>As we can see the memory usages, it increases by about 4 MB per each step. Thus, after completed, it increased around 400 MB in total. It is frustrating since it eventually eats up all memories in my system as the size of the input list grows and gc.collect() does not seem to help. </p>
<p>I suspect that there were memory leaks or Sage(or Python) stored all the values of cos() and never freed them for some reasons while the shell not killed. </p>
<p>Can someone please explain what was going on and suggest how to avoid this unwanted memory consumption? Thank you in advance.</p>
https://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/?comment=45871#post-id-45871You're absolutely right! Thank you very much for your comment. If I use `pari(pi)`, it works without memory leaks. However, if `gp(pi)` used, it just hangs. I still want to know how the leaks happen; I guess that this issue is related with `gp('znprimroot(10007)')` and `pari('znprimroot(10007)')` shown on [Interfaces--Sage Tutorial v86](http://doc.sagemath.org/html/en/tutorial/interfaces.html). By the way, if you put it in Answer, I will mark yours as an answer. Best, jb.Fri, 22 Mar 2019 13:40:02 -0500https://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/?comment=45871#post-id-45871Comment by rburing for <p>Hello, </p>
<p>I executed the following script on the Sage interactive shell on Ubuntu 18.04 (on SageMath version 8.7.beta5):</p>
<pre><code>sage: import gc
sage: t = 100000
sage: l = [n for n in xrange(t,t+100)]
sage: ll = []
sage: for x in l:
....: s = 0
....: for n in xrange(0,x):
....: s += cos(RDF(2)*RDF(pi)*RDF(n)/RDF(x))
....: ll.append([x, s])
....: del(s)
....: print "memory usage: " + str(get_memory_usage()) + ", gc: " + str(gc.collect())
....:
memory usage: 5824.5546875, gc: 277
memory usage: 5828.3046875, gc: 0
memory usage: 5832.0546875, gc: 0
memory usage: 5836.0546875, gc: 0
:
memory usage: 6203.5546875, gc: 0
memory usage: 6207.5546875, gc: 0
</code></pre>
<p>As we can see the memory usages, it increases by about 4 MB per each step. Thus, after completed, it increased around 400 MB in total. It is frustrating since it eventually eats up all memories in my system as the size of the input list grows and gc.collect() does not seem to help. </p>
<p>I suspect that there were memory leaks or Sage(or Python) stored all the values of cos() and never freed them for some reasons while the shell not killed. </p>
<p>Can someone please explain what was going on and suggest how to avoid this unwanted memory consumption? Thank you in advance.</p>
https://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/?comment=45868#post-id-45868I think the problem is not with `cos` but rather with `pi`:
import gc
for x in xrange(10^5,10^5+100):
s = sum(RDF(pi) for n in xrange(x))
del(s)
print "memory usage: " + str(get_memory_usage()) + ", gc: " + str(gc.collect())
The memory leak goes away when one replaces `RDF(pi)` by e.g. `RDF(3.14)` or `RDF.pi()`. Something must be going wrong when calling `RDF(pi)`. The same memory leak occurs when replacing the `pi` in `RDF(pi)` with any of the other [mathematical constants](http://doc.sagemath.org/html/en/reference/constants/sage/symbolic/constants.html).Fri, 22 Mar 2019 07:14:16 -0500https://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/?comment=45868#post-id-45868Answer by rburing for <p>Hello, </p>
<p>I executed the following script on the Sage interactive shell on Ubuntu 18.04 (on SageMath version 8.7.beta5):</p>
<pre><code>sage: import gc
sage: t = 100000
sage: l = [n for n in xrange(t,t+100)]
sage: ll = []
sage: for x in l:
....: s = 0
....: for n in xrange(0,x):
....: s += cos(RDF(2)*RDF(pi)*RDF(n)/RDF(x))
....: ll.append([x, s])
....: del(s)
....: print "memory usage: " + str(get_memory_usage()) + ", gc: " + str(gc.collect())
....:
memory usage: 5824.5546875, gc: 277
memory usage: 5828.3046875, gc: 0
memory usage: 5832.0546875, gc: 0
memory usage: 5836.0546875, gc: 0
:
memory usage: 6203.5546875, gc: 0
memory usage: 6207.5546875, gc: 0
</code></pre>
<p>As we can see the memory usages, it increases by about 4 MB per each step. Thus, after completed, it increased around 400 MB in total. It is frustrating since it eventually eats up all memories in my system as the size of the input list grows and gc.collect() does not seem to help. </p>
<p>I suspect that there were memory leaks or Sage(or Python) stored all the values of cos() and never freed them for some reasons while the shell not killed. </p>
<p>Can someone please explain what was going on and suggest how to avoid this unwanted memory consumption? Thank you in advance.</p>
https://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/?answer=45873#post-id-45873I reported this memory leak as [trac ticket #27536](https://trac.sagemath.org/ticket/27536).
A workaround is to replace `RDF(pi)` by `RDF.pi()`.Fri, 22 Mar 2019 14:09:13 -0500https://ask.sagemath.org/question/45863/memory-usage-strictly-increasing-on-sage-interactive-shell/?answer=45873#post-id-45873