Revision history [back]
 | 1 | initial version |
The tricky part about filter is that you typically do something like
filter(lambda x: x % 2 == 0, range(10))
But the multiprocessing module does not support using lambda expressions in things like map, etc. since they are not pickleable by default. That's the first problem. If you're using a function that's not defined in the main session, then you can do something along the following lines:
def pool_filter(pool, func, candidates):
return [c for c, keep in zip(candidates, p.map(func, candidates)) if keep]
and
sage: from multiprocessing import Pool
sage: p = Pool(2)
sage: pool_filter(p, is_prime, range(1, 20))
[2, 3, 5, 7, 11, 13, 17, 19]
Finally, there is some code at #9501 which makes an @fork decorator which allows you to do parallel stuff with lambda expressions. It would be fairly easy to make a parallel filter function that works fine with lambda functions and other functions defined in the main session.
