ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Sat, 22 May 2021 12:30:39 +0200How to use cached_function and parallel togetherhttps://ask.sagemath.org/question/55498/how-to-use-cached_function-and-parallel-together/I have a function `f(a,b,c)` which I'd like to save the output values of, so I'm currently using the `@cached_function` decorator to do so:
@cached_function
def f(a,b,c):
m = g(a+b)
n = h(c)
out = (expensive calculation involving m and m)
return m+n
@cached_function
def g(d):
out = (expensive calculation involving d)
return out
@cached_function
def h(c):
out = (expensive calculation involving c)
return out
I would now like to calculate the values of this function for a number of triples `(a,b,c) = (a1,b1,c1), (a2,b2,c2), ..., (aN,bN,cN)`, and since there are many such triples I'd like to calculate them in parallel. From some Googling it seems that I should use the `@parallel` decorator, but
1. How do I use the parallel decorator to calculate values of a function which has more than one input parameter?
2. Is it possible for the calculations of these individual values of `f` to share memory somehow? The calculation of my function `f` depends on other expensive functions `g` and `h` and thus I have also used the `@cached_function` decorator on them; it would be preferable for `g` and `h` to not be run on the same input if multiple processes are used to perform the calculation of `f`.Fri, 29 Jan 2021 16:49:37 +0100https://ask.sagemath.org/question/55498/how-to-use-cached_function-and-parallel-together/Answer by mwageringel for <p>I have a function <code>f(a,b,c)</code> which I'd like to save the output values of, so I'm currently using the <code>@cached_function</code> decorator to do so:</p>
<pre><code>@cached_function
def f(a,b,c):
m = g(a+b)
n = h(c)
out = (expensive calculation involving m and m)
return m+n
@cached_function
def g(d):
out = (expensive calculation involving d)
return out
@cached_function
def h(c):
out = (expensive calculation involving c)
return out
</code></pre>
<p>I would now like to calculate the values of this function for a number of triples <code>(a,b,c) = (a1,b1,c1), (a2,b2,c2), ..., (aN,bN,cN)</code>, and since there are many such triples I'd like to calculate them in parallel. From some Googling it seems that I should use the <code>@parallel</code> decorator, but</p>
<ol>
<li>How do I use the parallel decorator to calculate values of a function which has more than one input parameter?</li>
<li>Is it possible for the calculations of these individual values of <code>f</code> to share memory somehow? The calculation of my function <code>f</code> depends on other expensive functions <code>g</code> and <code>h</code> and thus I have also used the <code>@cached_function</code> decorator on them; it would be preferable for <code>g</code> and <code>h</code> to not be run on the same input if multiple processes are used to perform the calculation of <code>f</code>.</li>
</ol>
https://ask.sagemath.org/question/55498/how-to-use-cached_function-and-parallel-together/?answer=57218#post-id-57218To combine caching with parallel computations, you can use `cached_function.precompute` ([documentation](https://doc.sagemath.org/html/en/reference/misc/sage/misc/cachefunc.html#sage.misc.cachefunc.CachedFunction.precompute)).
<pre><code>@cached_function
def f(a,b,c):
print("computing f(%s,%s,%s)" % (a,b,c))
m = g(a+b)
n = h(c)
return m+n
@cached_function
def g(d):
print("computing g(%s)" % d)
out = d * d # (expensive calculation involving d)
return out
@cached_function
def h(c):
print("computing h(%s)" % c)
out = c * c * c # (expensive calculation involving c)
return out
triples = IntegerListsLex(length=3, max_part=2, element_constructor=tuple).list()
h.precompute(set([c for a,b,c in triples]), num_processes=8)
g.precompute(set([a+b for a,b,c in triples]), num_processes=8)
f.precompute(triples, num_processes=8)
[f(a,b,c) for a,b,c in triples]</code></pre>
The output shows that every function is evaluated only once for each argument:
<pre><code>computing h(0)
computing h(1)
computing h(2)
computing g(0)
computing g(1)
computing g(2)
computing g(3)
computing g(4)
computing f(2,2,2)
computing f(2,2,1)
computing f(2,2,0)
computing f(2,1,2)
computing f(2,1,1)
computing f(2,1,0)
computing f(2,0,2)
computing f(2,0,1)
computing f(2,0,0)
computing f(1,2,2)
computing f(1,2,1)
computing f(1,2,0)
computing f(1,1,2)
computing f(1,1,1)
computing f(1,1,0)
computing f(1,0,2)
computing f(1,0,1)
computing f(1,0,0)
computing f(0,2,2)
computing f(0,2,1)
computing f(0,2,0)
computing f(0,1,2)
computing f(0,1,1)
computing f(0,1,0)
computing f(0,0,2)
computing f(0,0,1)
computing f(0,0,0)
[24, 17, 16, 17, 10, 9, 12, 5, 4, 17, 10, 9, 12, 5, 4, 9, 2, 1, 12, 5, 4, 9, 2, 1, 8, 1, 0]</code></pre>
For this, one needs to know the arguments to `h` and `g` in advance, though, which can be difficult. This is not a problem in your example, but I am not sure how to solve this in general.
Regarding your question 1, wrap the arguments in a tuple to use `@parallel` with multiargs functions.
Sat, 22 May 2021 12:30:39 +0200https://ask.sagemath.org/question/55498/how-to-use-cached_function-and-parallel-together/?answer=57218#post-id-57218