Ask Your Question
2

Using @parallel in creating a list

asked 2011-06-13 10:04:26 -0500

kcrisman gravatar image

The question is how to create a list one element at a time using a parallel defined function. I do NOT want to use list comprehension like [f(i) for i in [1..100]] with a parallel defined f. I want to create a list that will still exist in Sage even if I interrupt its calculation before it's done.

It doesn't matter what my function is here. The point is what I'm trying to do here.

sage: @parallel(2)
....: def GMP(n): return (n,GetMinPerimAndSequence(n))
....:
sage: L = []
sage: L.append(GMP([1..100]))

What I intended was that I wanted to run this command with parallel execution, but create a list that would exist even if I interrupt the calculation (such as if it takes a looong time).

sage: L
[<generator object __call__ at 0x4964050>]
sage: L[0]
<generator object __call__ at 0x4964050>

which of course is the same as just making a generator in the first place and using it. So this strategy doesn't work.

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
1

answered 2011-06-13 10:56:18 -0500

niles gravatar image

The computations are done as you loop over the generator, so if you use a for loop to append to a list (rather than a list comprehension), I think this will do what you want:

sage: @parallel(2)
def GMP(n): sleep(1); return (n,4*n^3 + 2*n)

sage: gen = GMP(range(100))
sage: for t in gen:
    data += [(t[0][0][0],t[1])]
    print "{0} --> {1}".format(t[0][0][0],t[1])
....:  
1 --> (1, 6)
0 --> (0, 0)
2 --> (2, 36)
3 --> (3, 114)
4 --> (4, 264)
5 --> (5, 510)
6 --> (6, 876)
7 --> (7, 1386)
8 --> (8, 2064)
9 --> (9, 2934)
^CKilling any remaining workers...
---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
...
KeyboardInterrupt: 

sage: data.sort()
sage: data
[(0, (0, 0)), (1, (1, 6)), (2, (2, 36)), (3, (3, 114)), (4, (4, 264)), (5, (5, 510)), (6, (6, 876)), (7, (7, 1386)), (8, (8, 2064)), (9, (9, 2934))]
edit flag offensive delete link more

Comments

Ah, I know how to do that - I use .append instead. But the problem is that I don't know whether this is actually working in parallel! Is it? Would using .append to data also work?

kcrisman gravatar imagekcrisman ( 2011-06-13 12:23:42 -0500 )edit

Yes, for both -- if you actually run this, you'll see the messages printed in pairs, as each of the two processes finishes at about the same time. And if you skip the data.sort() step, you'll see that the items are still in the order they were finished. I'm sure .append will work too; I just haven't tried it :)

niles gravatar imageniles ( 2011-06-14 01:26:21 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

3 followers

Stats

Asked: 2011-06-13 10:04:26 -0500

Seen: 571 times

Last updated: Jun 13 '11