ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Fri, 15 May 2015 07:19:02 +0200How to make "zip" work faster?https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/ I have these two finite sets $A$ and $B$ where the size of $A$ is typically much larger than the size of $B$. (typically $|A| is 200-500$ and $|B| is 10-50$) I am trying to enumerate all possible maps from $B$ to $A$ using the following idea - but this turns out to be very slow.
- Is there a way to speed this up?
- Without the over all "for i" loop can I access any one of the "k"s?
(for every i each $l$ is a list of tuples)
How can I just pick out any one such "k" list without wanting to wait for the whole code to run.
S = []
from itertools import product
for i in product(A,repeat = len (B)):
k = zip(B,i)
S.append(k)
show(S)Sun, 10 May 2015 23:42:04 +0200https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/Comment by John Palmieri for <p>I have these two finite sets $A$ and $B$ where the size of $A$ is typically much larger than the size of $B$. (typically $|A| is 200-500$ and $|B| is 10-50$) I am trying to enumerate all possible maps from $B$ to $A$ using the following idea - but this turns out to be very slow. </p>
<ul>
<li><p>Is there a way to speed this up?</p>
<ul>
<li>Without the over all "for i" loop can I access any one of the "k"s?
(for every i each $l$ is a list of tuples)</li>
</ul>
<p>How can I just pick out any one such "k" list without wanting to wait for the whole code to run. </p>
<p></p><pre><code>S = []
from itertools import product
for i in product(A,repeat = len (B)):
k = zip(B,i)
S.append(k)
show(S)
</code></pre></li>
</ul><p></p>
https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?comment=26804#post-id-26804The number of such maps will be `|A|**|B|`. You're trying to store all `200**10` (lower bound) such elements in a single list? That's not practical: you should try to construct an iterator instead of a list.Tue, 12 May 2015 01:38:55 +0200https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?comment=26804#post-id-26804Answer by slelievre for <p>I have these two finite sets $A$ and $B$ where the size of $A$ is typically much larger than the size of $B$. (typically $|A| is 200-500$ and $|B| is 10-50$) I am trying to enumerate all possible maps from $B$ to $A$ using the following idea - but this turns out to be very slow. </p>
<ul>
<li><p>Is there a way to speed this up?</p>
<ul>
<li>Without the over all "for i" loop can I access any one of the "k"s?
(for every i each $l$ is a list of tuples)</li>
</ul>
<p>How can I just pick out any one such "k" list without wanting to wait for the whole code to run. </p>
<p></p><pre><code>S = []
from itertools import product
for i in product(A,repeat = len (B)):
k = zip(B,i)
S.append(k)
show(S)
</code></pre></li>
</ul><p></p>
https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?answer=26810#post-id-26810As @John Palmieri suggests, use iterators instead of lists.
You are already importing `product` from
[the itertools module](https://docs.python.org/2/library/itertools.html),
learn more about it, especially
[izip](https://docs.python.org/2/library/itertools.html#itertools.izip)
to replace zip.
Here is how to use iterators instead of lists.
from itertools import product
from itertools import izip
A = range(3)
B = range(2)
S = (izip(B,i) for i in product(A, repeat=len(B)))
To compare this code with the code quoted in the question,
- S is now an iterator instead of a list. This is achieved by doing `S = (... for ... in ...)`.
- each elementof S is now an iterator instead of a list. This is achieved by using `izip` instead of `zip`.
A good resource for learning about iterators is the [SageMath thematic tutorial on comprehensions, iterators ind iterables](http://doc.sagemath.org/html/en/thematic_tutorials/tutorial-comprehensions.html).Tue, 12 May 2015 17:26:12 +0200https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?answer=26810#post-id-26810Comment by Phoenix for <p>As <a href="/users/7/john-palmieri/">@John Palmieri</a> suggests, use iterators instead of lists.</p>
<p>You are already importing <code>product</code> from
<a href="https://docs.python.org/2/library/itertools.html">the itertools module</a>,
learn more about it, especially
<a href="https://docs.python.org/2/library/itertools.html#itertools.izip">izip</a>
to replace zip.</p>
<p>Here is how to use iterators instead of lists.</p>
<pre><code>from itertools import product
from itertools import izip
A = range(3)
B = range(2)
S = (izip(B,i) for i in product(A, repeat=len(B)))
</code></pre>
<p>To compare this code with the code quoted in the question,</p>
<ul>
<li>S is now an iterator instead of a list. This is achieved by doing <code>S = (... for ... in ...)</code>.</li>
<li>each elementof S is now an iterator instead of a list. This is achieved by using <code>izip</code> instead of <code>zip</code>.</li>
</ul>
<p>A good resource for learning about iterators is the <a href="http://doc.sagemath.org/html/en/thematic_tutorials/tutorial-comprehensions.html">SageMath thematic tutorial on comprehensions, iterators ind iterables</a>.</p>
https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?comment=26817#post-id-26817How is this S different from my S ?Wed, 13 May 2015 00:43:20 +0200https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?comment=26817#post-id-26817Comment by Phoenix for <p>As <a href="/users/7/john-palmieri/">@John Palmieri</a> suggests, use iterators instead of lists.</p>
<p>You are already importing <code>product</code> from
<a href="https://docs.python.org/2/library/itertools.html">the itertools module</a>,
learn more about it, especially
<a href="https://docs.python.org/2/library/itertools.html#itertools.izip">izip</a>
to replace zip.</p>
<p>Here is how to use iterators instead of lists.</p>
<pre><code>from itertools import product
from itertools import izip
A = range(3)
B = range(2)
S = (izip(B,i) for i in product(A, repeat=len(B)))
</code></pre>
<p>To compare this code with the code quoted in the question,</p>
<ul>
<li>S is now an iterator instead of a list. This is achieved by doing <code>S = (... for ... in ...)</code>.</li>
<li>each elementof S is now an iterator instead of a list. This is achieved by using <code>izip</code> instead of <code>zip</code>.</li>
</ul>
<p>A good resource for learning about iterators is the <a href="http://doc.sagemath.org/html/en/thematic_tutorials/tutorial-comprehensions.html">SageMath thematic tutorial on comprehensions, iterators ind iterables</a>.</p>
https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?comment=26818#post-id-26818I need to analyze each mapping one by one. So does "k = zip(B,i)" be exactly replaced by "k = izip(B,i)" ?Wed, 13 May 2015 00:46:27 +0200https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?comment=26818#post-id-26818Comment by slelievre for <p>As <a href="/users/7/john-palmieri/">@John Palmieri</a> suggests, use iterators instead of lists.</p>
<p>You are already importing <code>product</code> from
<a href="https://docs.python.org/2/library/itertools.html">the itertools module</a>,
learn more about it, especially
<a href="https://docs.python.org/2/library/itertools.html#itertools.izip">izip</a>
to replace zip.</p>
<p>Here is how to use iterators instead of lists.</p>
<pre><code>from itertools import product
from itertools import izip
A = range(3)
B = range(2)
S = (izip(B,i) for i in product(A, repeat=len(B)))
</code></pre>
<p>To compare this code with the code quoted in the question,</p>
<ul>
<li>S is now an iterator instead of a list. This is achieved by doing <code>S = (... for ... in ...)</code>.</li>
<li>each elementof S is now an iterator instead of a list. This is achieved by using <code>izip</code> instead of <code>zip</code>.</li>
</ul>
<p>A good resource for learning about iterators is the <a href="http://doc.sagemath.org/html/en/thematic_tutorials/tutorial-comprehensions.html">SageMath thematic tutorial on comprehensions, iterators ind iterables</a>.</p>
https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?comment=26834#post-id-26834Maybe you don't want each element of S to be an iterator. In that case, keep using `zip` instead of `izip`. But certainly you want S itself to be an iterator. The good solution for you might be:
S = (zip(B,i) for i in product(A, repeat=len(B)))Fri, 15 May 2015 07:19:02 +0200https://ask.sagemath.org/question/26790/how-to-make-zip-work-faster/?comment=26834#post-id-26834