Ask Your Question
0

Out of memory while enumerating vectors

asked 2013-09-09 11:42:56 -0500

Alexandru Papiu gravatar image

I am trying to enumerate all vectors of a certain dimension with entries less then some prescribed values and then check some conditions on each of them. This is the code I was thinking about:

A = [(a,b,c,d) for a in range(100) for b in range (100) for c in range (100) for d in range (10)]

for f in A: if Condition(f): print f

The problem is that when the product of my ranges is larger than $10^7$ or so I get a MemoryError.

Is there a more efficient way of enumerating vectors or avoiding the MemoryError?

Thank you.

edit retag flag offensive close merge delete

2 answers

Sort by ยป oldest newest most voted
1

answered 2013-09-09 22:54:56 -0500

tmonteil gravatar image

updated 2013-09-09 23:05:15 -0500

In your code, by just replacing [] by () and range() by xrange(), you build a generator, that will loop in real time, without storing all values in a list, saving a lot of memory:

sage: A = ((a,b,c,d) for a in xrange(100) for b in xrange (100) for c in xrange (100) for d in xrange (100))

sage: A
<generator object <genexpr> at 0x5d269b0>

sage: for f in A:
....:     print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...

By the way, if all prescribed values are the same, you can also do:

sage: B = cartesian_product_iterator([xrange(100) for i in range(4)])

sage: for f in B:
....:     print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...
edit flag offensive delete link more

Comments

It is better to use product from itertools then the cartesian_product_iterator (speed reason) as in: product(xrange(100), repeat=4).

vdelecroix gravatar imagevdelecroix ( 2013-09-10 07:18:00 -0500 )edit
1

answered 2013-09-09 12:13:41 -0500

ndomes gravatar image

You may use generators instead of lists. See: https://wiki.python.org/moin/Generators

For example:

def test_generator(n1,n2,n3,n4):
    for a in range(n1):
        for b in range(n2):
            for c in range(n3):
                for d in range(n4):
                    yield (a,b,c,d)

G = test_generator(100,100,100,100)

for k in range(100^4):
    if sum(G.next()) == 10: print k
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2013-09-09 11:42:56 -0500

Seen: 156 times

Last updated: Sep 09 '13