Ask Your Question
1

How to make "zip" work faster?

asked 2015-05-10 23:42:04 +0100

Phoenix gravatar image

updated 2015-05-10 23:45:40 +0100

I have these two finite sets $A$ and $B$ where the size of $A$ is typically much larger than the size of $B$. (typically $|A| is 200-500$ and $|B| is 10-50$) I am trying to enumerate all possible maps from $B$ to $A$ using the following idea - but this turns out to be very slow.

  • Is there a way to speed this up?

    • Without the over all "for i" loop can I access any one of the "k"s? (for every i each $l$ is a list of tuples)

    How can I just pick out any one such "k" list without wanting to wait for the whole code to run.

    S = []
    from itertools import product
    
    for i in product(A,repeat = len (B)):
    
        k = zip(B,i)
        S.append(k)
    
    
    show(S)
    

edit retag flag offensive close merge delete

Comments

The number of such maps will be |A|**|B|. You're trying to store all 200**10 (lower bound) such elements in a single list? That's not practical: you should try to construct an iterator instead of a list.

John Palmieri gravatar imageJohn Palmieri ( 2015-05-12 01:38:55 +0100 )edit

1 Answer

Sort by ยป oldest newest most voted
3

answered 2015-05-12 17:26:12 +0100

slelievre gravatar image

updated 2015-05-15 07:16:05 +0100

As @John Palmieri suggests, use iterators instead of lists.

You are already importing product from the itertools module, learn more about it, especially izip to replace zip.

Here is how to use iterators instead of lists.

from itertools import product
from itertools import izip

A = range(3)
B = range(2)

S = (izip(B,i) for i in product(A, repeat=len(B)))

To compare this code with the code quoted in the question,

  • S is now an iterator instead of a list. This is achieved by doing S = (... for ... in ...).
  • each elementof S is now an iterator instead of a list. This is achieved by using izip instead of zip.

A good resource for learning about iterators is the SageMath thematic tutorial on comprehensions, iterators ind iterables.

edit flag offensive delete link more

Comments

How is this S different from my S ?

Phoenix gravatar imagePhoenix ( 2015-05-13 00:43:20 +0100 )edit

I need to analyze each mapping one by one. So does "k = zip(B,i)" be exactly replaced by "k = izip(B,i)" ?

Phoenix gravatar imagePhoenix ( 2015-05-13 00:46:27 +0100 )edit

Maybe you don't want each element of S to be an iterator. In that case, keep using zip instead of izip. But certainly you want S itself to be an iterator. The good solution for you might be:

S = (zip(B,i) for i in product(A, repeat=len(B)))
slelievre gravatar imageslelievre ( 2015-05-15 07:19:02 +0100 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2015-05-10 23:42:04 +0100

Seen: 435 times

Last updated: May 15 '15