ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Tue, 28 Feb 2012 11:20:00 +0100Modifying ncpus.py to detect processors on computational nodeshttps://ask.sagemath.org/question/8746/modifying-ncpuspy-to-detect-processors-on-computational-nodes/Recently we've been attempting to run Sage on a LittleFe cluster (littlefe.net if you're interested to read more), however, the ncpus function reads directly from system settings, thus it only reads the two slots on the head node of the cluster. In order to run other jobs outside of Sage using OpenMPI, we are able to input a machine file generated by a BCCD specific command as an argument for OpenMPI to detect and use the slots on the other nodes. I'm wondering if there is a way to either modify the existing check or if there is another way to detect the slots on the other nodes to fully utilize Sage's parallel capabilities.Mon, 27 Feb 2012 14:30:13 +0100https://ask.sagemath.org/question/8746/modifying-ncpuspy-to-detect-processors-on-computational-nodes/Answer by niles for <p>Recently we've been attempting to run Sage on a LittleFe cluster (<a href="http://littlefe.net">littlefe.net</a> if you're interested to read more), however, the ncpus function reads directly from system settings, thus it only reads the two slots on the head node of the cluster. In order to run other jobs outside of Sage using OpenMPI, we are able to input a machine file generated by a BCCD specific command as an argument for OpenMPI to detect and use the slots on the other nodes. I'm wondering if there is a way to either modify the existing check or if there is another way to detect the slots on the other nodes to fully utilize Sage's parallel capabilities.</p>
https://ask.sagemath.org/question/8746/modifying-ncpuspy-to-detect-processors-on-computational-nodes/?answer=13311#post-id-13311Many of Sage's parallel functions allow you to specify the number of CPUs by hand. If you do that, does Sage use your entire cluster? If not, then I would recommend solving that problem first. Most of Sage's parallel funcitonality uses Python's [`os.fork`](http://docs.python.org/library/os.html#os.fork) (see the documentation of [sage.parallel.use_fork.p_iter_fork](http://hg.sagemath.org/sage-main/file/c239be1054e0/sage/parallel/use_fork.py#l14)).
If you use `os.fork` in plain Python, does everything work as expected?
If the problem really is just with the output of `ncpus`, you can work around it for now by simply giving the number of cpus explicitly and work on patching `ncpus.py` to correctly identify the number of cpus.Tue, 28 Feb 2012 11:20:00 +0100https://ask.sagemath.org/question/8746/modifying-ncpuspy-to-detect-processors-on-computational-nodes/?answer=13311#post-id-13311