Out of memory while enumerating vectors

asked 2013-09-09 18:42:56 +0100

Alexandru Papiu
15 ●1 ●1 ●5

I am trying to enumerate all vectors of a certain dimension with entries less then some prescribed values and then check some conditions on each of them. This is the code I was thinking about:

A = [(a,b,c,d) for a in range(100) for b in range (100) for c in range (100) for d in range (10)]

for f in A: if Condition(f): print f

The problem is that when the product of my ranges is larger than $10^7$ or so I get a MemoryError.

Is there a more efficient way of enumerating vectors or avoiding the MemoryError?

Thank you.

edit retag flag offensive close merge delete

add a comment

2 Answers

Sort by » oldest newest most voted

answered 2013-09-10 05:54:56 +0100

tmonteil
27423 ●31 ●204 ●517 http://wiki.sagemath.o...

updated 2013-09-10 06:05:15 +0100

In your code, by just replacing [] by () and range() by xrange(), you build a generator, that will loop in real time, without storing all values in a list, saving a lot of memory:

sage: A = ((a,b,c,d) for a in xrange(100) for b in xrange (100) for c in xrange (100) for d in xrange (100))

sage: A
<generator object <genexpr> at 0x5d269b0>

sage: for f in A:
....:     print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...

By the way, if all prescribed values are the same, you can also do:

sage: B = cartesian_product_iterator([xrange(100) for i in range(4)])

sage: for f in B:
....:     print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...

edit flag offensive delete link

Comments

It is better to use product from itertools then the cartesian_product_iterator (speed reason) as in: product(xrange(100), repeat=4).

vdelecroix ( 2013-09-10 14:18:00 +0100 )edit

add a comment

answered 2013-09-09 19:13:41 +0100

ndomes

2429 ●1 ●22 ●55

You may use generators instead of lists. See: https://wiki.python.org/moin/Generators

For example:

def test_generator(n1,n2,n3,n4):
    for a in range(n1):
        for b in range(n2):
            for c in range(n3):
                for d in range(n4):
                    yield (a,b,c,d)

G = test_generator(100,100,100,100)

for k in range(100^4):
    if sum(G.next()) == 10: print k

edit flag offensive delete link

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Out of memory while enumerating vectors

2 Answers

Comments

Your Answer

Question Tools

Stats

Related questions

Out of memory while enumerating vectors edit

2 Answers

Comments

Your Answer

Question Tools

Stats

Related questions

Out of memory while enumerating vectors