ZoSolvers

Zeroth-order optimisation solvers with Gaussian and sphere random oracles.

ZoSolvers provides gradient-free solvers for minimisation and minimax problems. When the gradient of the objective is unavailable — because the function is non-differentiable, comes from a black-box simulator, or is too expensive to differentiate — ZoSolvers estimates it using random directional perturbations.

📖 Full documentation: ZoSolvers Manual (PDF)

Install

pip install ZoSolvers

Requirements: Python ≥ 3.10, NumPy ≥ 2.2, Matplotlib ≥ 3.7

Features

• Two oracle types — Gaussian (u ~ N(0, B⁻¹)) and sphere (uniform on the unit sphere), both with optional preconditioning via a precision matrix B
• Four solvers — ZOGD, ZOEGm (minimisation), ZOGDA, ZOEGmm (minimax)
• Three finite-difference methods — forward, backward, centered
• Flexible mini-batching — fixed number of oracle samples per step, or a growing schedule (t="iteration")
• Constrained problems — pass any projection function, and choose whether the initial guess is projected too
• Early stopping — based on relative function decrease or oracle norm
• Parallel mini-batches — the t oracle samples per step are evaluated across n_jobs workers (thread or process backend)
• Efficient sampling — Cholesky factorisation cached at construction; diagonal B handled without matrix inversion

Quick Start

Minimisation

import numpy as np
from ZoSolvers.minimisation import ZO_gauss_min

def f(x):
    return x[0]**2 + x[1]**2 + x[0]*x[1]

x0  = np.array([5.0, -5.0])
opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=10)

# Zeroth-order gradient descent
x_traj = opt.ZOGD(method="center")

# Zeroth-order extra-gradient
x_traj = opt.ZOEGm(method="center", gamma=1.0)

Minimax

from ZoSolvers.minimax import ZO_gauss_minmax

def f(x, y):
    return x[0]**2 + x[1]**2 - y[0]**2 - y[1]**2 + x[0]*y[1]

x0 = np.array([5.0, -5.0])
y0 = np.array([3.0, -3.0])

opt = ZO_gauss_minmax(f, x0, y0, h=1e-3, tau=1, mu=1e-8, N=10000, t=5)

# Gradient descent-ascent
x_traj, y_traj = opt.ZOGDA(method="center")

# Extra-gradient
x_traj, y_traj = opt.ZOEGmm(method="center", gamma=0.8)

Sphere Oracle and Precision Matrix

B = np.array([[10.0, 0.5],
              [0.5,  2.0]])

opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=10,
                   B=B, oracle_type="sphere")
x_traj = opt.ZOGD(method="center")

Parallel Function Evaluations

The t oracle samples that make up one step are independent, so they can be evaluated concurrently. Pass n_jobs (-1 = all cores):

opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=32, n_jobs=-1)
x_traj = opt.ZOGD(method="center")
opt.close()          # or use the solver as a context manager

The worker pool is created once and reused across every iteration. Use the solver as a context manager to shut it down automatically:

with ZO_gauss_minmax(f, x0, y0, h=1e-3, mu=1e-8, N=10000, t=16, n_jobs=8) as opt:
    x_traj, y_traj = opt.ZOGDA(method="center")

Two backends are available:

Parallelism pays off when the cost function is expensive relative to the scheduling overhead, the usual case for black-box simulators.

Box-Constrained Problem

proj = lambda x: np.clip(x, -3.0, 3.0)

opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=10, proj=proj)
x_traj = opt.ZOGD(method="center")

Every iterate the solver computes is feasible, but the first row of the returned trajectory is the x0 you passed in, returned untouched. If x0 lies outside the feasible set, that first row is infeasible. Set project_init=True to project it as well, so the whole trajectory is feasible:

opt = ZO_gauss_min(f, x0, h=1e-2, mu=1e-5, N=2000, t=10, proj=proj,
                   project_init=True)
x_traj = opt.ZOGD(method="center")   # x_traj[0] == proj(x0)

The default is False, which keeps x0 visible exactly as given. For minimax, the same flag covers both x0 and y0, each through its own projection.

Solvers

Key Parameters

Package Structure

ZoSolvers/
├── src/ZoSolvers/
│   ├── minimisation.py   # ZO_gauss_min
│   ├── minimax.py        # ZO_gauss_minmax
│   └── utils.py          # shared utilities
├── tests/
│   ├── min_test.py       # pytest suite for minimisation
│   ├── minimax_test.py   # pytest suite for minimax
│   ├── demo_min.py       # minimisation demo with plots
│   └── demo_minimax.py   # minimax demo with plots
├── Docs/
│   └── ZoSolvers_Manual.pdf
└── pyproject.toml

License

MIT