ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Thu, 24 Jun 2021 14:40:41 +0200Is parallel computation with mpi4py still supported?https://ask.sagemath.org/question/57732/is-parallel-computation-with-mpi4py-still-supported/I am working with Sage code which calculates many independent instances of a problem and is thus fully parallelizable. Now, I would like to port it to a multi-node cluster environment.
My current idea is to use a main process which manages a problem instance queue from which worker processes fetch instances, solve them and save their results to the file system. In the multi-node environment, I understand that some form of message passing is needed to communicate the problem instances to the workers (ruling out task queue management with the `@parallel` decorator). I have found that [the Python package `mpi4py`](https://mpi4py.readthedocs.io/en/stable/) provides Python bindings for the Message Passing Interface. It also implements a very convenient [MPIPoolExecutor class](https://mpi4py.readthedocs.io/en/stable/mpi4py.futures.html#mpipoolexecutor) which manages such a task queue.
The current Sage (9.3) documentation includes a [thematic tutorial on mpyi4py](https://doc.sagemath.org/html/en/thematic_tutorials/numerical_sage/mpi4py.html) mentioning that `mpi4py` is supported in Sage by means of an optional package. However, I do not find it in the [list of optional packages for Sage 9.3](https://doc.sagemath.org/html/en/reference/spkg/index.html) and on my current v9.2 install, the method `optional_packages()` also does not list it (and neither the other required `openmpi` package).
Is `mpi4py` still supported in a current Sage version? Would it be much effort to try and build it for the current version like it was done for `openmpi` in [Sage Trac ticket 8537](https://trac.sagemath.org/ticket/8537)?
Or are there other recommendations for task distribution with Sage in a multi-node environment?FabianGThu, 24 Jun 2021 14:40:41 +0200https://ask.sagemath.org/question/57732/MPI4Py, MPI in Sagehttps://ask.sagemath.org/question/9813/mpi4py-mpi-in-sage/We've tried following the directions at <http://www.sagemath.org/doc/numerical_sage/mpi4py.html> and checked the forum at <http://ask.sagemath.org/question/14/what-are-my-best-options-for-taking-advantage-of>, but we're still having trouble implementing MPI in Sage.
Ideally, we'd prefer to implement MPI from a notebook instead of the command line, but that doesn't seem to be an option with MPI4Py. When we tried to use the command line, we had trouble importing the packages from the first link.
So, we were wondering:
1. Is there an alternative to MPI which allows us to exploit multiple cores (or a cluster) from notebook form?
2. Does anybody have an easier-to-understand set of instructions than the ones listed above for using MPI4Py?jquigleySat, 16 Feb 2013 15:50:55 +0100https://ask.sagemath.org/question/9813/