ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Tue, 10 Mar 2020 14:49:08 +0100getting RESetMapReduce to use multiple coreshttps://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/I am reading the [documentation](https://doc.sagemath.org/html/en/reference/parallel/sage/parallel/map_reduce.html#protocol-description) on how to parallelize recursive-enumeration computations, and I cannot for the life of me get `RESetMapReduce` to use more than one core of my machine.
I am using the example immediately before the bulleted item titled **Generating series** on that page, with a minor modification:
sage: from sage.parallel.map_reduce import RESetMapReduce
sage: S = RESetMapReduce(
....: roots=[[]],
....: children=lambda l: [l + [0], l + [1]] if len(l) < 32 else [],
....: map_function=lambda x: 1,
....: reduce_function=lambda x, y: x + y,
....: reduce_init=0)
sage: S.run()
I changed the length to 32 to make the computation heftier. If I run this and watch the process with `htop` I can see it using only one of my 8 cores to `100%` capacity, ignoring the others.
I have even tried passing the argument `max_proc=8` to the `.run()` method, to no effect.Tue, 10 Mar 2020 04:05:58 +0100https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/Answer by grobber for <p>I am reading the <a href="https://doc.sagemath.org/html/en/reference/parallel/sage/parallel/map_reduce.html#protocol-description">documentation</a> on how to parallelize recursive-enumeration computations, and I cannot for the life of me get <code>RESetMapReduce</code> to use more than one core of my machine.</p>
<p>I am using the example immediately before the bulleted item titled <strong>Generating series</strong> on that page, with a minor modification:</p>
<pre><code>sage: from sage.parallel.map_reduce import RESetMapReduce
sage: S = RESetMapReduce(
....: roots=[[]],
....: children=lambda l: [l + [0], l + [1]] if len(l) < 32 else [],
....: map_function=lambda x: 1,
....: reduce_function=lambda x, y: x + y,
....: reduce_init=0)
sage: S.run()
</code></pre>
<p>I changed the length to 32 to make the computation heftier. If I run this and watch the process with <code>htop</code> I can see it using only one of my 8 cores to <code>100%</code> capacity, ignoring the others.</p>
<p>I have even tried passing the argument <code>max_proc=8</code> to the <code>.run()</code> method, to no effect.</p>
https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?answer=50209#post-id-50209Actually, this worked: I ran the same computation in a terminal after setting the environment variable `SAGE_NUM_THREADS` to whatever was desired (8 in my case).
All cores are used in that case.Tue, 10 Mar 2020 04:16:48 +0100https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?answer=50209#post-id-50209Comment by Sébastien for <p>Actually, this worked: I ran the same computation in a terminal after setting the environment variable <code>SAGE_NUM_THREADS</code> to whatever was desired (8 in my case).</p>
<p>All cores are used in that case.</p>
https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50211#post-id-50211One may also
import os
os.environ["SAGE_NUM_THREADS"] = '16'
as suggested here: https://ask.sagemath.org/question/46711Tue, 10 Mar 2020 08:40:14 +0100https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50211#post-id-50211Comment by grobber for <p>Actually, this worked: I ran the same computation in a terminal after setting the environment variable <code>SAGE_NUM_THREADS</code> to whatever was desired (8 in my case).</p>
<p>All cores are used in that case.</p>
https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50213#post-id-50213Thanks! I should perhaps have searched more thoroughly; this thread parallels that other one..Tue, 10 Mar 2020 12:00:48 +0100https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50213#post-id-50213Comment by grobber for <p>Actually, this worked: I ran the same computation in a terminal after setting the environment variable <code>SAGE_NUM_THREADS</code> to whatever was desired (8 in my case).</p>
<p>All cores are used in that case.</p>
https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50215#post-id-50215Absolutely, that would help! The environment variable is mentioned in passing (twice) on the page I linked, which tipped me off to try to set it, but it's not made clear there that one needs it..Tue, 10 Mar 2020 14:23:56 +0100https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50215#post-id-50215Comment by grobber for <p>Actually, this worked: I ran the same computation in a terminal after setting the environment variable <code>SAGE_NUM_THREADS</code> to whatever was desired (8 in my case).</p>
<p>All cores are used in that case.</p>
https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50216#post-id-50216Oh, and by the way, I tried this just now: if you run this as a `sage` script (I name mine `<blah>.sage`, make them executable, and put the shebang line `#!/usr/bin/env sage` at the top), then you don't actually need to `import os`: it just works. Maybe `sage` imports it automatically? I don't know, but I know the explicit import isn't needed.Tue, 10 Mar 2020 14:49:08 +0100https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50216#post-id-50216Comment by Sébastien for <p>Actually, this worked: I ran the same computation in a terminal after setting the environment variable <code>SAGE_NUM_THREADS</code> to whatever was desired (8 in my case).</p>
<p>All cores are used in that case.</p>
https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50214#post-id-50214It is a good thing to add to the documentation since this question often comes since ncpus() does not return the number of ncpus anymore. I posted a branch at https://trac.sagemath.org/ticket/29308Tue, 10 Mar 2020 12:23:29 +0100https://ask.sagemath.org/question/50208/getting-resetmapreduce-to-use-multiple-cores/?comment=50214#post-id-50214