Revision history [back]

To combine caching with parallel computations, you can use cached_function.precompute (documentation).

@cached_function
def f(a,b,c):
    print("computing f(%s,%s,%s)" % (a,b,c))
    m = g(a+b)
    n = h(c)
    return m+n

@cached_function
def g(d):
    print("computing g(%s)" % d)
    out = d * d  # (expensive calculation involving d)
    return out

@cached_function
def h(c):
    print("computing h(%s)" % c)
    out = c * c * c  # (expensive calculation involving c)
    return out

triples = IntegerListsLex(length=3, max_part=2, element_constructor=tuple).list()
h.precompute(set([c for a,b,c in triples]), num_processes=8)
g.precompute(set([a+b for a,b,c in triples]), num_processes=8)
f.precompute(triples, num_processes=8)
[f(a,b,c) for a,b,c in triples]

The output shows that every function is evaluated only once for each argument:

computing h(0)
computing h(1)
computing h(2)
computing g(0)
computing g(1)
computing g(2)
computing g(3)
computing g(4)
computing f(2,2,2)
computing f(2,2,1)
computing f(2,2,0)
computing f(2,1,2)
computing f(2,1,1)
computing f(2,1,0)
computing f(2,0,2)
computing f(2,0,1)
computing f(2,0,0)
computing f(1,2,2)
computing f(1,2,1)
computing f(1,2,0)
computing f(1,1,2)
computing f(1,1,1)
computing f(1,1,0)
computing f(1,0,2)
computing f(1,0,1)
computing f(1,0,0)
computing f(0,2,2)
computing f(0,2,1)
computing f(0,2,0)
computing f(0,1,2)
computing f(0,1,1)
computing f(0,1,0)
computing f(0,0,2)
computing f(0,0,1)
computing f(0,0,0)
[24, 17, 16, 17, 10, 9, 12, 5, 4, 17, 10, 9, 12, 5, 4, 9, 2, 1, 12, 5, 4, 9, 2, 1, 8, 1, 0]

For this, one needs to know the arguments to h and g in advance, though, which can be difficult. This is not a problem in your example, but I am not sure how to solve this in general.

Regarding your question 1, wrap the arguments in a tuple to use @parallel with multiargs functions.