Processing math: 100%

First time here? Check out the FAQ!

Ask Your Question
0

Out of memory while enumerating vectors

asked 11 years ago

Alexandru Papiu gravatar image

I am trying to enumerate all vectors of a certain dimension with entries less then some prescribed values and then check some conditions on each of them. This is the code I was thinking about:

A = [(a,b,c,d) for a in range(100) for b in range (100) for c in range (100) for d in range (10)]

for f in A: if Condition(f): print f

The problem is that when the product of my ranges is larger than 107 or so I get a MemoryError.

Is there a more efficient way of enumerating vectors or avoiding the MemoryError?

Thank you.

Preview: (hide)

2 Answers

Sort by » oldest newest most voted
1

answered 11 years ago

tmonteil gravatar image

updated 11 years ago

In your code, by just replacing [] by () and range() by xrange(), you build a generator, that will loop in real time, without storing all values in a list, saving a lot of memory:

sage: A = ((a,b,c,d) for a in xrange(100) for b in xrange (100) for c in xrange (100) for d in xrange (100))

sage: A
<generator object <genexpr> at 0x5d269b0>

sage: for f in A:
....:     print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...

By the way, if all prescribed values are the same, you can also do:

sage: B = cartesian_product_iterator([xrange(100) for i in range(4)])

sage: for f in B:
....:     print f
(0, 0, 0, 0)
(0, 0, 0, 1)
(0, 0, 0, 2)
(0, 0, 0, 3)
(0, 0, 0, 4)
...
Preview: (hide)
link

Comments

It is better to use product from itertools then the cartesian_product_iterator (speed reason) as in: product(xrange(100), repeat=4).

vdelecroix gravatar imagevdelecroix ( 11 years ago )
1

answered 11 years ago

ndomes gravatar image

You may use generators instead of lists. See: https://wiki.python.org/moin/Generators

For example:

def test_generator(n1,n2,n3,n4):
    for a in range(n1):
        for b in range(n2):
            for c in range(n3):
                for d in range(n4):
                    yield (a,b,c,d)

G = test_generator(100,100,100,100)

for k in range(100^4):
    if sum(G.next()) == 10: print k
Preview: (hide)
link

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 11 years ago

Seen: 471 times

Last updated: Sep 10 '13