Ask Your Question
1

Modifying ncpus.py to detect processors on computational nodes

asked 2012-02-27 07:30:13 -0600

Evan F. gravatar image

updated 2013-07-12 02:39:50 -0600

niles gravatar image

Recently we've been attempting to run Sage on a LittleFe cluster (littlefe.net if you're interested to read more), however, the ncpus function reads directly from system settings, thus it only reads the two slots on the head node of the cluster. In order to run other jobs outside of Sage using OpenMPI, we are able to input a machine file generated by a BCCD specific command as an argument for OpenMPI to detect and use the slots on the other nodes. I'm wondering if there is a way to either modify the existing check or if there is another way to detect the slots on the other nodes to fully utilize Sage's parallel capabilities.

edit retag flag offensive close merge delete

1 answer

Sort by ยป oldest newest most voted
0

answered 2012-02-28 04:20:00 -0600

niles gravatar image

Many of Sage's parallel functions allow you to specify the number of CPUs by hand. If you do that, does Sage use your entire cluster? If not, then I would recommend solving that problem first. Most of Sage's parallel funcitonality uses Python's os.fork (see the documentation of sage.parallel.use_fork.p_iter_fork).

If you use os.fork in plain Python, does everything work as expected?

If the problem really is just with the output of ncpus, you can work around it for now by simply giving the number of cpus explicitly and work on patching ncpus.py to correctly identify the number of cpus.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

Stats

Asked: 2012-02-27 07:30:13 -0600

Seen: 137 times

Last updated: Feb 28 '12