maybempi

Use MPI only when the process was launched under MPI, and a serial stand-in for mpi4py otherwise. The same code runs under mpirun/srun and as a plain python script.py, and the serial run never starts MPI.

Documentation: https://max-models.github.io/maybempi/

from maybempi import MPI  # instead of: from mpi4py import MPI

# mpi4py.MPI under mpirun/mpiexec/srun, a serial stand-in otherwise
comm = MPI.COMM_WORLD
total = comm.allreduce(local_total, op=MPI.SUM)  # local_total in a serial run

Why

Importing mpi4py.MPI calls MPI_Init, which costs close to a second and makes every collective cost something, even on one process. maybempi decides from the environment variables that MPI launchers export (Open MPI, MPICH, Intel MPI, PMIx/srun, MVAPICH2, Cray), without importing mpi4py. In a serial run it returns a stand-in with the module constants and communicator methods of mpi4py, which give the results MPI gives on one process: allreduce(x) is x, gather(x) is [x], Allreduce(send, recv) copies. Methods it does not implement raise AttributeError instead of silently doing nothing.

• MAYBEMPI=0/1 overrides the decision.
• maybempi.is_serial(MPI) tells the stand-in from mpi4py.
• maybempi.local_rank() gives the node-local rank before MPI_Init, e.g. to pick a GPU.
• The maybempi command prints what it decides, and why: mpirun -n 2 maybempi --init.

maybempi is pure Python and has no dependencies.

Install

pip install maybempi            # serial runs only
pip install "maybempi[mpi]"     # with mpi4py, for MPI runs

Development

With uv:

make install    # uv sync --extra dev, plus the pre-commit hooks
make lint       # ruff check, ruff format --check, pyright
make test       # pytest with coverage
mpiexec -n 2 .venv/bin/python -m pytest src/maybempi/tests/mpi   # tests under a real launcher

Commit messages follow Conventional Commits; see CONTRIBUTING.md.

The documentation in docs/ is an Astro + Starlight site with the executed notebooks from tutorials/ and an API reference generated from the docstrings. It needs Node 22 or newer:

make docs-install     # npm packages and the Python docs extra
make docs-notebooks   # execute tutorials/*.ipynb and convert them to pages
make docs-dev         # live preview at http://localhost:4321/maybempi/

README.md is rendered from README.qmd with Quarto: make readme.

Releases

Before merging a release to main, update the version in pyproject.toml, src/maybempi/__init__.py and CITATION.cff (including its release date), and add the release notes to CHANGELOG.md. The push to main creates a GitHub release with a vX.Y.Z tag and publishes the package to PyPI with trusted publishing (OIDC). See the publishing guide.