ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Tue, 10 Sep 2013 14:18:00 +0200Out of memory while enumerating vectorshttps://ask.sagemath.org/question/10527/out-of-memory-while-enumerating-vectors/I am trying to enumerate all vectors of a certain dimension with entries less then some prescribed values and then check some conditions on each of them. This is the code I was thinking about:
`A = [(a,b,c,d) for a in range(100) for b in range (100) for c in range (100) for d in range (10)]`
`for f in A:`
`if Condition(f):`
`print f`
The problem is that when the product of my ranges is larger than $10^7$ or so I get a `MemoryError`.
Is there a more efficient way of enumerating vectors or avoiding the `MemoryError`?
Thank you.Mon, 09 Sep 2013 18:42:56 +0200https://ask.sagemath.org/question/10527/out-of-memory-while-enumerating-vectors/Answer by ndomes for <p>I am trying to enumerate all vectors of a certain dimension with entries less then some prescribed values and then check some conditions on each of them. This is the code I was thinking about: </p>
<p><code>A = [(a,b,c,d) for a in range(100) for b in range (100) for c in range (100) for d in range (10)]</code></p>
<p><code>for f in A:</code>
<code>if Condition(f):</code>
<code>print f</code></p>
<p>The problem is that when the product of my ranges is larger than $10^7$ or so I get a <code>MemoryError</code>.</p>
<p>Is there a more efficient way of enumerating vectors or avoiding the <code>MemoryError</code>?</p>
<p>Thank you.</p>
https://ask.sagemath.org/question/10527/out-of-memory-while-enumerating-vectors/?answer=15435#post-id-15435You may use generators instead of lists.
See: https://wiki.python.org/moin/Generators
For example:
def test_generator(n1,n2,n3,n4):
for a in range(n1):
for b in range(n2):
for c in range(n3):
for d in range(n4):
yield (a,b,c,d)
G = test_generator(100,100,100,100)
for k in range(100^4):
if sum(G.next()) == 10: print kMon, 09 Sep 2013 19:13:41 +0200https://ask.sagemath.org/question/10527/out-of-memory-while-enumerating-vectors/?answer=15435#post-id-15435Answer by tmonteil for <p>I am trying to enumerate all vectors of a certain dimension with entries less then some prescribed values and then check some conditions on each of them. This is the code I was thinking about: </p>
<p><code>A = [(a,b,c,d) for a in range(100) for b in range (100) for c in range (100) for d in range (10)]</code></p>
<p><code>for f in A:</code>
<code>if Condition(f):</code>
<code>print f</code></p>
<p>The problem is that when the product of my ranges is larger than $10^7$ or so I get a <code>MemoryError</code>.</p>
<p>Is there a more efficient way of enumerating vectors or avoiding the <code>MemoryError</code>?</p>
<p>Thank you.</p>
https://ask.sagemath.org/question/10527/out-of-memory-while-enumerating-vectors/?answer=15437#post-id-15437In your code, by just replacing `[]` by `()` and `range()` by `xrange()`, you build a generator, that will loop in real time, without storing all values in a list, saving a lot of memory:
sage: A = ((a,b,c,d) for a in xrange(100) for b in xrange (100) for c in xrange (100) for d in xrange (100))
sage: A
<generator object <genexpr> at 0x5d269b0>
sage: for f in A:
....: print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...
By the way, if all prescribed values are the same, you can also do:
sage: B = cartesian_product_iterator([xrange(100) for i in range(4)])
sage: for f in B:
....: print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...
Tue, 10 Sep 2013 05:54:56 +0200https://ask.sagemath.org/question/10527/out-of-memory-while-enumerating-vectors/?answer=15437#post-id-15437Comment by vdelecroix for <p>In your code, by just replacing <code>[]</code> by <code>()</code> and <code>range()</code> by <code>xrange()</code>, you build a generator, that will loop in real time, without storing all values in a list, saving a lot of memory:</p>
<pre><code>sage: A = ((a,b,c,d) for a in xrange(100) for b in xrange (100) for c in xrange (100) for d in xrange (100))
sage: A
<generator object <genexpr> at 0x5d269b0>
sage: for f in A:
....: print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...
</code></pre>
<p>By the way, if all prescribed values are the same, you can also do:</p>
<pre><code>sage: B = cartesian_product_iterator([xrange(100) for i in range(4)])
sage: for f in B:
....: print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...
</code></pre>
https://ask.sagemath.org/question/10527/out-of-memory-while-enumerating-vectors/?comment=17025#post-id-17025It is better to use product from itertools then the cartesian_product_iterator (speed reason) as in: product(xrange(100), repeat=4).Tue, 10 Sep 2013 14:18:00 +0200https://ask.sagemath.org/question/10527/out-of-memory-while-enumerating-vectors/?comment=17025#post-id-17025