Skip to main content

Parallel parameter sweeps with joblib/MPI and tqdm progress bars.

Project description

ParameterRun

parameterrun is a lightweight utility for running parameter sweeps in Python with a simple, function-based API.

It is designed for scientific and numerical workflows where you want to evaluate a function over one or more parameter grids, optionally in parallel, while keeping progress bars and output reshaping convenient.

The package supports:

  • single-parameter scans
  • multi-parameter scans
  • grouped parameters that must vary together
  • parallel execution with joblib
  • MPI execution with mpi4py
  • automatic reshaping of outputs into arrays
  • functions returning either one output or multiple outputs

Installation

With pip

pip install parameterrun

With optional MPI support

pip install parameterrun[mpi]

Basic idea

The main entry point is:

from parameterrun import parameterrun

The function parameterrun(...) evaluates a user-defined function over a set of parameter values. The parameters to sweep are identified by their names exactly as they appear in the target function signature. Additional fixed arguments can be passed as keyword arguments.

Quick start

1. Single parameter

from parameterrun import parameterrun


def square(x):
    return x ** 2


result = parameterrun(square, param_names="x", param_values=[0, 1, 2, 3], n_workers=1, )

print(result)  # array([0, 1, 4, 9])

2. Multiple independent parameters

This creates a Cartesian product of the parameter groups.

from parameterrun import parameterrun


def f(x, y, scale=1):
    return scale * (x + y)


result = parameterrun(f, param_names=["x", "y"], param_values=[[1, 2, 3], [10, 20], ], scale=2, )

print(result.shape)  # (3, 2)

3. Grouped parameters

Parameters in the same group vary together and must have the same number of values.

from parameterrun import parameterrun


def f(x, y, z):
    return x + y + z


result = parameterrun(f, param_names=[["x", "y"], ["z"]], param_values=[[[1, 2], [10, 20]],  # (x, y) = (1,10), (2,20)
                                                                        [[100, 200, 300]],  # z varies independently
                                                                        ], n_workers=1, )

print(result.shape)  # (2, 3)

This grouped-input mode is useful when some parameters are logically linked and should not be combined independently.

Multiple outputs

If the target function returns a tuple, parameterrun returns a list of arrays, one per output channel.

from parameterrun import parameterrun


def stats(x):
    return x + 1, x ** 2


mean_like, square_like = parameterrun(stats, param_names="x", param_values=[1, 2, 3], n_workers=1, )

print(mean_like)
# array([2, 3, 4])

print(square_like)  # array([1, 4, 9])

If the function returns a single object, the result is returned as a single array when reshaping is possible.

For multi-dimensional sweeps, reshaping is automatic when reshape=True (default): the first dimensions always match the parameter grid shape.

Validation behavior

parameterrun validates inputs before launching workers and raises clear ValueErrors when:

  • parameter names are duplicated
  • a swept parameter does not exist in the target function signature
  • extra keyword arguments do not match the target function signature
  • a keyword argument conflicts with a swept parameter name

Parallel backend

parameterrun supports two execution backends:

  • joblib: a simple, local parallelization library that works well for most use cases.
  • mpi: a distributed parallelization approach using MPI, suitable for running on clusters or supercomputers.

If backend=None, the package tries to detect whether MPI is available. If MPI is not available, it falls back to joblib; otherwise it uses MPI.

Joblib backend

result = parameterrun(my_function, param_names=["x", "y"], param_values=[[1, 2, 3], [4, 5]], backend="joblib",
                      n_workers=-1, )

If n_workers=1, the function runs serially without spawning joblib workers.

MPI backend

result = parameterrun(my_function, param_names=["x", "y"], param_values=[[1, 2, 3], [4, 5]], backend="mpi", )

Run the script with MPI, for example:

mpirun -n 4 python my_script.py

Under MPI, only rank 0 returns the final result; the other ranks return None.

Progress bars and verbosity

By default, progress bars are enabled:

result = parameterrun(my_function, param_names="x", param_values=[1, 2, 3], pbar_bool=True, )

You can also pass extra keyword arguments to tqdm through pbar_kwargs:

result = parameterrun(my_function, param_names="x", param_values=[1, 2, 3], pbar_kwargs={"leave": False}, )

Verbose logging can be enabled with:

result = parameterrun(my_function, param_names="x", param_values=[1, 2, 3], verbose=True, )

The progress-bar description is generated automatically from the function name and parameter names, unless you provide desc explicitly.

API reference

parameterrun

parameterrun(fun, param_names, param_values, n_workers=-1, pbar_bool=True, verbose=False, pbar_kwargs=None,
             reshape=True, backend=None, desc=None, **kwargs, )
  • fun: function to evaluate
  • param_names: parameter name or grouped parameter names
  • param_values: values corresponding to param_names
  • n_workers: number of workers for joblib
  • pbar_bool: enable or disable progress bars
  • verbose: print backend and timing information
  • pbar_kwargs: additional keyword arguments passed to tqdm
  • reshape: reshape results into arrays when possible
  • backend: "joblib", "mpi", or None
  • desc: custom progress-bar label
  • **kwargs: extra fixed keyword arguments forwarded to fun

Input formats

parameterrun accepts three input styles.

A. Single parameter

param_names = "x"
param_values = [1, 2, 3]

B. Multiple independent parameters

param_names = ["x", "y"]
param_values = [[1, 2, 3], [10, 20], ]

C. Multiple groups of parameters

param_names = [["x", "y"], ["z"]]
param_values = [[[1, 2], [10, 20]], [[100, 200, 300]], ]

Development checks

Before opening a PR or publishing a release, run:

pytest
ruff check .
mypy src/parameterrun
python -m build
python -m twine check dist/*

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

parameterrun-0.2.1.tar.gz (16.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

parameterrun-0.2.1-py3-none-any.whl (10.1 kB view details)

Uploaded Python 3

File details

Details for the file parameterrun-0.2.1.tar.gz.

File metadata

  • Download URL: parameterrun-0.2.1.tar.gz
  • Upload date:
  • Size: 16.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for parameterrun-0.2.1.tar.gz
Algorithm Hash digest
SHA256 532d829d2203a658a0bd2aea7d3cdbfb557274c82144c6cac735038dfb465792
MD5 853d9ed8bdb78c20b18fe2330ae1095b
BLAKE2b-256 4ee87c08467faf1258edc5e76ef1ea878061fc8edca075ad269dc3ccfc82506b

See more details on using hashes here.

Provenance

The following attestation bundles were made for parameterrun-0.2.1.tar.gz:

Publisher: publish.yml on Davtax/ParameterRun

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file parameterrun-0.2.1-py3-none-any.whl.

File metadata

  • Download URL: parameterrun-0.2.1-py3-none-any.whl
  • Upload date:
  • Size: 10.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for parameterrun-0.2.1-py3-none-any.whl
Algorithm Hash digest
SHA256 f3d721fc662a34d35b9e757a5968d7f1ece7d61f7d08aab50d417b76c249c6e5
MD5 711bc454ff7c915c09fd15fd9aa6d10c
BLAKE2b-256 fd65d2a0576a9445020f7c9f9eb744abb72db2f8b724fdb0037b3b5ab3ae2787

See more details on using hashes here.

Provenance

The following attestation bundles were made for parameterrun-0.2.1-py3-none-any.whl:

Publisher: publish.yml on Davtax/ParameterRun

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page