HowTo implement filter for multiprocessing module?

edit

The tricky part about filter is that you typically do something like

filter(lambda x: x % 2 == 0, range(10))

But the multiprocessing module does not support using lambda expressions in things like map, etc. since they are not pickleable by default. That's the first problem. If you're using a function that's not defined in the main session, then you can do something along the following lines:

def pool_filter(pool, func, candidates):
    return [c for c, keep in zip(candidates, p.map(func, candidates)) if keep]

and

sage: from multiprocessing import Pool
sage: p = Pool(2)
sage: pool_filter(p, is_prime, range(1, 20))
[2, 3, 5, 7, 11, 13, 17, 19]

Finally, there is some code at #9501 which makes an @fork decorator which allows you to do parallel stuff with lambda expressions. It would be fairly easy to make a parallel filter function that works fine with lambda functions and other functions defined in the main session.