ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Tue, 14 Jun 2011 08:26:21 +0200Using @parallel in creating a listhttps://ask.sagemath.org/question/8163/using-parallel-in-creating-a-list/The question is how to create a list one element at a time using a parallel defined function. I do NOT want to use list comprehension like `[f(i) for i in [1..100]]` with a parallel defined `f`. I want to create a list that will still exist in Sage even if I interrupt its calculation before it's done.
It doesn't matter what my function is here. The point is what I'm trying to do here.
sage: @parallel(2)
....: def GMP(n): return (n,GetMinPerimAndSequence(n))
....:
sage: L = []
sage: L.append(GMP([1..100]))
What I intended was that I wanted to run this command with parallel execution, but create a list that would exist **even if I interrupt the calculation** (such as if it takes a looong time).
sage: L
[<generator object __call__ at 0x4964050>]
sage: L[0]
<generator object __call__ at 0x4964050>
which of course is the same as just making a generator in the first place and using it. So this strategy doesn't work.Mon, 13 Jun 2011 17:04:26 +0200https://ask.sagemath.org/question/8163/using-parallel-in-creating-a-list/Answer by niles for <p>The question is how to create a list one element at a time using a parallel defined function. I do NOT want to use list comprehension like <code>[f(i) for i in [1..100]]</code> with a parallel defined <code>f</code>. I want to create a list that will still exist in Sage even if I interrupt its calculation before it's done.</p>
<p>It doesn't matter what my function is here. The point is what I'm trying to do here.</p>
<pre><code>sage: @parallel(2)
....: def GMP(n): return (n,GetMinPerimAndSequence(n))
....:
sage: L = []
sage: L.append(GMP([1..100]))
</code></pre>
<p>What I intended was that I wanted to run this command with parallel execution, but create a list that would exist <strong>even if I interrupt the calculation</strong> (such as if it takes a looong time).</p>
<pre><code>sage: L
[<generator object __call__ at 0x4964050>]
sage: L[0]
<generator object __call__ at 0x4964050>
</code></pre>
<p>which of course is the same as just making a generator in the first place and using it. So this strategy doesn't work.</p>
https://ask.sagemath.org/question/8163/using-parallel-in-creating-a-list/?answer=12433#post-id-12433The computations are done as you loop over the generator, so if you use a for loop to append to a list (rather than a list comprehension), I think this will do what you want:
sage: @parallel(2)
def GMP(n): sleep(1); return (n,4*n^3 + 2*n)
sage: gen = GMP(range(100))
sage: for t in gen:
data += [(t[0][0][0],t[1])]
print "{0} --> {1}".format(t[0][0][0],t[1])
....:
1 --> (1, 6)
0 --> (0, 0)
2 --> (2, 36)
3 --> (3, 114)
4 --> (4, 264)
5 --> (5, 510)
6 --> (6, 876)
7 --> (7, 1386)
8 --> (8, 2064)
9 --> (9, 2934)
^CKilling any remaining workers...
---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
...
KeyboardInterrupt:
sage: data.sort()
sage: data
[(0, (0, 0)), (1, (1, 6)), (2, (2, 36)), (3, (3, 114)), (4, (4, 264)), (5, (5, 510)), (6, (6, 876)), (7, (7, 1386)), (8, (8, 2064)), (9, (9, 2934))]
Mon, 13 Jun 2011 17:56:18 +0200https://ask.sagemath.org/question/8163/using-parallel-in-creating-a-list/?answer=12433#post-id-12433Comment by kcrisman for <p>The computations are done as you loop over the generator, so if you use a for loop to append to a list (rather than a list comprehension), I think this will do what you want:</p>
<pre><code>sage: @parallel(2)
def GMP(n): sleep(1); return (n,4*n^3 + 2*n)
sage: gen = GMP(range(100))
sage: for t in gen:
data += [(t[0][0][0],t[1])]
print "{0} --> {1}".format(t[0][0][0],t[1])
....:
1 --> (1, 6)
0 --> (0, 0)
2 --> (2, 36)
3 --> (3, 114)
4 --> (4, 264)
5 --> (5, 510)
6 --> (6, 876)
7 --> (7, 1386)
8 --> (8, 2064)
9 --> (9, 2934)
^CKilling any remaining workers...
---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
...
KeyboardInterrupt:
sage: data.sort()
sage: data
[(0, (0, 0)), (1, (1, 6)), (2, (2, 36)), (3, (3, 114)), (4, (4, 264)), (5, (5, 510)), (6, (6, 876)), (7, (7, 1386)), (8, (8, 2064)), (9, (9, 2934))]
</code></pre>
https://ask.sagemath.org/question/8163/using-parallel-in-creating-a-list/?comment=21591#post-id-21591Ah, I know how to do that - I use .append instead. But the problem is that I don't know whether this is actually working in parallel! Is it? Would using .append to data also work?Mon, 13 Jun 2011 19:23:42 +0200https://ask.sagemath.org/question/8163/using-parallel-in-creating-a-list/?comment=21591#post-id-21591Comment by niles for <p>The computations are done as you loop over the generator, so if you use a for loop to append to a list (rather than a list comprehension), I think this will do what you want:</p>
<pre><code>sage: @parallel(2)
def GMP(n): sleep(1); return (n,4*n^3 + 2*n)
sage: gen = GMP(range(100))
sage: for t in gen:
data += [(t[0][0][0],t[1])]
print "{0} --> {1}".format(t[0][0][0],t[1])
....:
1 --> (1, 6)
0 --> (0, 0)
2 --> (2, 36)
3 --> (3, 114)
4 --> (4, 264)
5 --> (5, 510)
6 --> (6, 876)
7 --> (7, 1386)
8 --> (8, 2064)
9 --> (9, 2934)
^CKilling any remaining workers...
---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
...
KeyboardInterrupt:
sage: data.sort()
sage: data
[(0, (0, 0)), (1, (1, 6)), (2, (2, 36)), (3, (3, 114)), (4, (4, 264)), (5, (5, 510)), (6, (6, 876)), (7, (7, 1386)), (8, (8, 2064)), (9, (9, 2934))]
</code></pre>
https://ask.sagemath.org/question/8163/using-parallel-in-creating-a-list/?comment=21585#post-id-21585Yes, for both -- if you actually run this, you'll see the messages printed in pairs, as each of the two processes finishes at about the same time. And if you skip the data.sort() step, you'll see that the items are still in the order they were finished. I'm sure .append will work too; I just haven't tried it :)Tue, 14 Jun 2011 08:26:21 +0200https://ask.sagemath.org/question/8163/using-parallel-in-creating-a-list/?comment=21585#post-id-21585