getting RESetMapReduce to use multiple cores

grobber
65 ●9

I am reading the documentation on how to parallelize recursive-enumeration computations, and I cannot for the life of me get RESetMapReduce to use more than one core of my machine.

I am using the example immediately before the bulleted item titled Generating series on that page, with a minor modification:

sage: from sage.parallel.map_reduce import RESetMapReduce
sage: S = RESetMapReduce(
....:     roots=[[]],
....:     children=lambda l: [l + [0], l + [1]] if len(l) < 32 else [],
....:     map_function=lambda x: 1,
....:     reduce_function=lambda x, y: x + y,
....:     reduce_init=0)
sage: S.run()

I changed the length to 32 to make the computation heftier. If I run this and watch the process with htop I can see it using only one of my 8 cores to 100% capacity, ignoring the others.

I have even tried passing the argument max_proc=8 to the .run() method, to no effect.

answered 5 years ago

grobber
65 ●9

updated 5 years ago

Actually, this worked: I ran the same computation in a terminal after setting the environment variable SAGE_NUM_THREADS to whatever was desired (8 in my case).

All cores are used in that case.

link

Comments

One may also

import os
os.environ["SAGE_NUM_THREADS"] = '16'

as suggested here: https://ask.sagemath.org/question/46711

Sébastien ( 5 years ago )

Thanks! I should perhaps have searched more thoroughly; this thread parallels that other one..

grobber ( 5 years ago )

It is a good thing to add to the documentation since this question often comes since ncpus() does not return the number of ncpus anymore. I posted a branch at https://trac.sagemath.org/ticket/29308

Sébastien ( 5 years ago )

Absolutely, that would help! The environment variable is mentioned in passing (twice) on the page I linked, which tipped me off to try to set it, but it's not made clear there that one needs it..

grobber ( 5 years ago )

Oh, and by the way, I tried this just now: if you run this as a sage script (I name mine <blah>.sage, make them executable, and put the shebang line #!/usr/bin/env sage at the top), then you don't actually need to import os: it just works. Maybe sage imports it automatically? I don't know, but I know the explicit import isn't needed.

grobber ( 5 years ago )

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

getting RESetMapReduce to use multiple cores

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

getting RESetMapReduce to use multiple cores savecancel

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

getting RESetMapReduce to use multiple cores