PIPS-IPM++ Python interface

Python interface for the block-structure exploiting interior-point solver PIPS-IPM++.

📖 Documentation

The package takes a standard-form LP plus one integer per variable that says which block the variable belongs to. Everything else is derived from that annotation: which constraints are local to a block, which ones link several blocks, how the blocks are mapped onto MPI ranks, and how the solution is put back into your own ordering. The problem is built and block-structured on rank 0 and then scattered. Every rank calls solve, and the primal and dual vectors are gathered back on rank 0.

import numpy as np
import scipy.sparse as sp
from mpi4py import MPI

import pipsipmpp

problem = pipsipmpp.StructuredProblem(
    n_blocks=2,
    var_block=np.array([0, 1, 2]),                        # root, leaf 1, leaf 2
    c=np.array([3.0, 1.0, 2.0]),
    xlow=np.zeros(3),
    xupp=np.full(3, np.inf),
    A_eq=sp.csr_array(np.array([[0.0, 1.0, 1.0]])),       # y1 + y2 == 4  (linking)
    b_eq=np.array([4.0]),
    A_ineq=sp.csr_array(np.array([[-2.0, 1.0, 0.0],       # y1 <= 2 x     (leaf 1)
                                  [-1.0, 0.0, 1.0]])),    # y2 <=   x     (leaf 2)
    ineq_low=np.full(2, -np.inf),
    ineq_upp=np.zeros(2),
)

comm = MPI.COMM_WORLD
result = pipsipmpp.solve(problem if comm.Get_rank() == 0 else None, comm)

if comm.Get_rank() == 0:
    print(result.status.name, result.objective, result.primal)

mpirun -n 2 python model.py
# SUCCESSFUL_TERMINATION 9.33333333333419 [1.33333333 2.66666667 1.33333333]

What the package offers

Solve in memory	solve(problem, comm=None, ...): rank 0 owns the model, the blocks are scattered, and the solution comes back in your own variable order
Solve from disk	solve_dataset(path, comm=None, ...): each rank reads only the blocks it owns, so the model is never assembled in one process
Write problems	write_problem(problem, path, layout=...) in a monolithic or a per-block distributed layout, optionally carrying variable and constraint names
Read problems	read_monolithic, read_block, read_manifest, read_layout, read_names and is_flat, none of which need the solver library
Solutions as files	write_solution / read_solution, interchangeable with what the solver's own pipsparquet ... writesol writes
Solutions back on the model	linopy, Pyomo, PyOptInterface and GAMSPy each read a solution from file onto the model they wrote, leaving it as an in-memory solve would
Options	an option dictionary, a settings file, or both, to control the behavior of PIPS-IPM++
Communicator	optional on both solve functions, defaulting to MPI.COMM_WORLD, and handed to PIPS-IPM++ itself
Your own solver build	PIPSIPMPP_LIB selects any libpips-ipmpp.so, and libpips_info() reports which one was loaded
Inspect the structure	derive_blocks performs the split without solving, and block_owner says which rank gets which block
Derive a structure	annotate(problem, n_blocks, method=...) finds one in the matrix for a model that carries none, by regular expression over the variable names or by hypergraph partitioning, through the optional pipstools dependency

Reading and writing the files needs neither MPI nor the compiled solver, so the machine that builds a model does not have to be the machine that solves it (see the parquet workflow).

Modelling frameworks

The easiest way to use PIPS-IPM++ is through a modelling framework, which builds the StructuredProblem from a model you already have. All four frameworks below reach PIPS-IPM++ through this package:

modelling framework	the block of a variable comes from
linopy	Model.blocks, or n_blocks over a dimension
Pyomo	a pips_block Suffix
PyOptInterface	block= on the variable
GAMSPy	the GAMS .stage of the variable

If your model does not come from one of these frameworks, pipstools is a command-line tool that finds a block structure in an .lp, .mps or .gdx file by hypergraph partitioning.

The first three carry a PIPS-IPM++ solver of their own. GAMSPy does not, so that example ships a small pipsipmpp_interface.py module that reads the generated instance through the GMO API.

Worked examples for each of them are in the documentation, and as runnable projects under examples/.

Installation

pip install "pipsipmpp[solver]"

The solver extra pulls in mpi4py. Without it the package still imports, and everything except solve/solve_dataset keeps working.

The annotate extra pulls in pipstools, which is what finds a block structure for a model that carries none. Both can be asked for at once:

pip install "pipsipmpp[solver,annotate]"

The wheels bundle libpips-ipmpp.so, built once against Open MPI and once against MPICH, but not the libraries it links against: your system needs an MPI implementation, OpenBLAS and MUMPS. Check what was loaded with:

python -c "import pipsipmpp; print(pipsipmpp.libpips_info())"

Using your own solver build

Point PIPSIPMPP_LIB at any libpips-ipmpp.so: a build of PIPS-IPM++ with HSL or another linear solver, a debug build, or one tuned to the hardware of your cluster:

PIPSIPMPP_LIB=/opt/pips/libpips-ipmpp.so mpirun -n 4 python model.py

The major and the minor version of the library are both checked at import, so a mismatch gives a clear ImportError instead of a crash later on. Full instructions are under custom solver library.

Building from source

Needs at least a C++17 toolchain, CMake, an MPI implementation, MUMPS and a BLAS:

git clone --recurse-submodules https://gitlab.com/pips-ipmpp/pipsipmpppy.git
cd pipsipmpp
uv build

The wheel bundles the libpips-ipmpp.so it built, but not the dependencies of that library, so the installing machine needs the same MPI, MUMPS and BLAS. To build against a solver checkout of your own instead of the submodule:

pip install . -C cmake.define.PIPS_SOURCE_DIR=/path/to/pips-ipmpp

Documentation

The documentation is built from the project environment. uv sync installs the dev dependency group, which is the test and the docs group together:

uv sync
uv run sphinx-build -W --keep-going -b html docs docs/_build/html

The rendered version is available at https://pips-ipmpp.gitlab.io/pipsipmpppy.

Authors

Manuel Wetzel (German Aerospace Center, DLR)

Acknoledgements

The Python interface for PIPS-IPM++ was developed as a deliverable of the PEREGRINE project, which was funded by the German Federal Ministry for Economic Affairs and Energy under grant number 03EI1082A.

License

See LICENSE.

Notice

The shared objects bundled with the Python wheels redistribute unmodified open source libraries from third parties. For the detailed list of third party software see the NOTICE and the ThirdPartyLicences folder.

Disclaimer on the use of coding assistance

This project contains code generated using Claude Code. I have reviewed all generated and modified source code, revised it where necessary, and take the same responsibility for it as for code I write myself.