optfuncs 0.0.6

Optimization benchmark functions and pytest-oriented evaluation helpers.

optfunc

optfunc provides differentiable PyTorch optfuncs, CVXPY-backed convex benchmarks, and pytest-oriented helpers for evaluating first-order and second-order optimizers.

The package is published as optfuncs and imported as optfunc.

Install

For CPU-only use:

pip install "optfuncs[torch-cpu]"

For convex-only CVXPY benchmarks:

pip install "optfuncs[convex]"

For CPU-only use plus convex benchmarks:

pip install "optfuncs[torch-cpu,convex]"

With uv in this repository:

uv sync --extra torch-cpu

To enable the CVXPY convex benchmark suite:

uv sync --extra convex

Supported extras:

Extra	Backend	Typical command
convex	CVXPY convex benchmarks, default MOSEK reference solve with SCS fallback	uv sync --extra convex
torch-cpu	CPU, Linux/macOS/Windows	uv sync --extra torch-cpu
torch-cu118	CUDA 11.8, Linux/Windows, PyTorch 2.7.x	uv sync --extra torch-cu118
torch-cu126	CUDA 12.6, Linux/Windows	uv sync --extra torch-cu126
torch-cu128	CUDA 12.8, Linux/Windows	uv sync --extra torch-cu128
torch-cu130	CUDA 13.0, Linux/Windows	uv sync --extra torch-cu130
torch-rocm	ROCm 7.1, Linux	uv sync --extra torch-rocm
torch-xpu	Intel XPU, Linux/Windows	uv sync --extra torch-xpu

Only choose one Torch extra at a time. The Torch extras are configured as mutually exclusive in pyproject.toml. The convex extra can be combined with any Torch extra or used on its own.

Using optfuncs from another uv project

When another uv project wants to expose multiple optional dependency sets, add the convex/base benchmark support and the Torch backend support separately. This keeps the target project's extras explicit and lets users choose the Torch variant independently. optfuncs[convex] does not install or require Torch; add a torch-* extra only when the target project also uses unconstrained PyTorch optfuncs.

Recommended target-project extra layout:

Target extra	Include
test	optfuncs[convex]>=0.0.5 for pytest helpers and CVXPY convex benchmarks
cpu	optfuncs[torch-cpu]>=0.0.5 for CPU Torch optfuncs
cu130	optfuncs[torch-cu130]>=0.0.5 for CUDA 13.0 Torch optfuncs

Example:

uv add --optional test "optfuncs[convex]>=0.0.5"
uv add --optional cu130 "optfuncs[torch-cu130]>=0.0.5"

For a CPU project, use:

uv add --optional test "optfuncs[convex]>=0.0.5"
uv add --optional cpu "optfuncs[torch-cpu]>=0.0.5"

Then install the combination you need from the target project:

uv sync --extra test --extra cu130

Avoid putting more than one optfuncs[torch-*] variant in the same resolved environment. Keep the convex extra separate from the Torch backend extra because convex brings CVXPY/MOSEK support, while torch-* selects the Torch runtime.

Helper scripts:

# bash/zsh on Linux or macOS, and Git Bash/MSYS/Cygwin on Windows
./scripts/sync_torch_variant.sh cpu
./scripts/sync_torch_variant.sh cu130
./scripts/sync_torch_variant.sh rocm
./scripts/sync_torch_variant.sh xpu

# Windows PowerShell or PowerShell 7 on Windows/Linux/macOS
.\scripts\sync_torch_variant.ps1 cpu
.\scripts\sync_torch_variant.ps1 cu130
.\scripts\sync_torch_variant.ps1 rocm
.\scripts\sync_torch_variant.ps1 xpu

If Windows blocks local PowerShell scripts, run:

powershell -ExecutionPolicy Bypass -File .\scripts\sync_torch_variant.ps1 cu130

Platform notes:

• CPU works on Linux, macOS, and Windows.
• CUDA extras work on Linux and Windows.
• ROCm works on Linux.
• XPU works on Linux and Windows.
• macOS should use torch-cpu in this project configuration.

The backend indexes follow the official uv PyTorch guide and PyTorch install selector:

• https://docs.astral.sh/uv/guides/integration/pytorch/
• https://pytorch.org/get-started/locally/

Optfunc Usage

The repository provides these built-in optfuncs:

Registry name	Class	Known minimizer
ackley	Ackley	all zeros
dixonprice	DixonPrice	recursive Dixon-Price optimum
griewank	Griewank	all zeros
levy	Levy	all ones
rastrigin	Rastrigin	all zeros
rosenbrock	Rosenbrock	all ones
rotatedhyperellipsoid	RotatedHyperEllipsoid	all zeros
schwefel	Schwefel	near 420.968746 in every coordinate
sphere	Sphere	all zeros
styblinskitang	StyblinskiTang	near -2.903534 in every coordinate
sumsquares	SumSquares	all zeros
trid	Trid	x_i = i * (d + 1 - i) with 1-based indexing
zakharov	Zakharov	all zeros

Use a class directly when you know which function you want:

import torch
from optfunc import Sphere

opt_func = Sphere(dim=8, dtype=torch.float64)
x = torch.zeros(8, dtype=torch.float64)

value = opt_func(x)
grad = opt_func.grad(x)
hessian = opt_func.hessian(x)
hvp = opt_func.hvp(x, torch.ones_like(x))
x_star = opt_func.global_minimizer()
distance = opt_func.distance_to_optimum(x)

Use OptFuncRegistry when a test should select an optfunc by name:

from optfunc import OptFuncRegistry

opt_func = OptFuncRegistry.create("rosenbrock", dim=4)
print(OptFuncRegistry.available())

Use BenchmarkRegistry when the caller may switch between unconstrained functions and one concrete convex benchmark instance:

from optfunc import BenchmarkRegistry

convex_problem = BenchmarkRegistry.create(
    "gw_maxcut",
    constraints="convex",
    dim=8,
    seed=19,
)

optimal_value = convex_problem.meta.optimal_value
X_star = convex_problem.known_solution("X")
cvxpy_problem = convex_problem.problem

Use CvxRegistry when you want a family of parameterised convex programs:

import numpy as np
from optfunc import CvxRegistry

family = CvxRegistry.create("gw_maxcut", dim=8)
rng = np.random.default_rng(0)

theta = family.sample_parameter(rng)
problem = family.build_instance(theta)

theta_sequence = family.generate_sequence(n=5, magnitude=10.0, rng=rng)
problem_sequence = [family.build_instance(theta) for theta in theta_sequence]

Each optfunc uses the same conventions:

• input x is a 1-D PyTorch tensor with shape (dim,);
• output is a scalar tensor;
• grad, hessian, and hvp use PyTorch autograd unless a subclass provides a better implementation;
• global_minimizer() returns a known theoretical minimizer when available;
• distance_to_optimum(x) defaults to Euclidean distance to global_minimizer();
• project_to_bounds(x) clamps a point into the optfunc's documented search box.

Convex Benchmark Usage

The repository also provides CVXPY convex benchmarks behind constraints="convex":

Registry name	Cone	Shape parameter	Validation path
zero, zero_cone_qp	zero cone / equality constraints	vector dim	known optimum
nonneg, nonnegative_cone_qp	nonnegative cone	vector dim	known optimum
psd, psd_cone_projection	PSD cone	matrix side length dim	known optimum
gw_maxcut, maxcut_sdp	PSD cone	graph vertex count dim	analytic optimum

Each convex benchmark family is generated from CvxRegistry and exposes ProblemFamily methods:

• sample_parameter(rng): sample a Python/NumPy parameter dictionary;
• build_instance(theta): turn that parameter into a concrete CvxpyProblem;
• perturb(theta, magnitude, rng): produce a nearby parameter;
• generate_sequence(n, magnitude, rng): produce a parameter sequence for batch testing.

Each concrete CvxpyProblem exposes:

• problem.problem: the underlying cvxpy.Problem;
• problem.data: deterministic random instance data used to build the problem;
• problem.solve(...): solve with CVXPY, defaulting to MOSEK and falling back to SCS if MOSEK is unavailable in the current environment;
• problem.known_solution() and problem.distance_to_optimum() when the theoretical solution is encoded in the benchmark definition.

The gw_maxcut benchmark is a seeded Goemans-Williamson Max-Cut SDP relaxation on a complete weighted bipartite graph. The seed fixes both the planted cut and the positive edge weights. Its SDP relaxation optimum is known without calling a solver: problem.meta.optimal_value is the total planted cut weight and problem.known_solution("X") is the rank-one planted cut matrix.

If a convex benchmark does not have a closed-form theoretical solution yet, you can leave that solution unspecified and use problem.solve() as the reference value/solution path. The default reference solver is MOSEK.

The sequence API uses numpy.random.Generator for reproducible parameter sampling, so CvxRegistry and CVXPY problem generation work with only the convex extra. Torch is required only for constraints="none" PyTorch optfuncs.

Pytest Optimizer Evaluation

The optimizer evaluation API lives in optfunc.testing. A standard test file has three parts:

1. Define or import an optimizer function.
2. Configure one or more OptimizerCase objects.
3. Assign make_optimizer_tests(...) to a pytest-visible name such as test_my_optimizer.

Run the file with:

uv run pytest tests/test_my_optimizer.py -q --tb=short --optfunc-report

Each OptimizerCase becomes an independent pytest item. If one case fails or raises an exception, pytest continues running the remaining cases. --optfunc-report prints a final serial summary with function gap, distance to the theoretical optimum, gradient norm, Hessian information, step count, and error messages. Do not combine --optfunc-report with pytest-xdist -n in v1.

OptimizerBudget

OptimizerBudget describes the budget passed to the optimizer.

from optfunc.testing import OptimizerBudget

budget = OptimizerBudget(max_steps=500, lr=0.05)

Fields:

• max_steps: positive integer iteration limit.
• lr: positive learning-rate-like scalar. The test harness does not enforce how the optimizer uses it; your optimizer reads it from problem.budget.lr.

For torch.optim.Adam, a typical use is:

optimizer = torch.optim.Adam([x], lr=problem.budget.lr)
for step in range(problem.budget.max_steps):
    ...

ConvergenceTolerances

ConvergenceTolerances decides whether a finished optimizer run passes.

from optfunc.testing import ConvergenceTolerances

tolerances = ConvergenceTolerances(
    value_gap=1e-8,
    x_distance=1e-4,
    grad_norm=1e-4,
    hessian_min_eig=0.0,
)

Fields:

• value_gap: maximum allowed absolute gap between final value and the known theoretical optimum value. Set to None to skip this check.
• x_distance: maximum allowed distance from final x to the known theoretical minimizer. Set to None to skip this check.
• grad_norm: maximum allowed Euclidean norm of the final gradient. Set to None to skip this check.
• hessian_min_eig: optional lower bound on the final Hessian's smallest eigenvalue. Set to None to skip this check.

The report still computes available metrics even when a tolerance is None; None only disables that pass/fail check.

OptimizerCase

OptimizerCase describes one pytest item.

from optfunc.testing import OptimizerCase
from optfunc.testing import OptimizerBudget, ConvergenceTolerances

case = OptimizerCase(
  opt_func="sphere",
  constraints="none",
  dim=8,
  budget=OptimizerBudget(max_steps=350, lr=0.05),
  tolerances=ConvergenceTolerances(value_gap=1e-8, x_distance=1e-4, grad_norm=1e-4),
  start="near_minimizer",
  start_radius=0.5,
  seed=0,
  hessian_max_dim=32,
)

Important fields:

• opt_func: registry name such as "sphere" or an already-created TorchOptFunction instance.
• constraints: "none" for the existing Torch optfunc suite, or "convex" for the CVXPY convex benchmark suite.
• dim: required when opt_func is a string.
• batch_size: for constraints="convex", generate this many same-family problem instances.
• perturb_magnitude: for convex batches, controls successive parameter perturbations in ProblemFamily.generate_sequence(...).
• budget: OptimizerBudget passed to the optimizer through problem.budget.
• tolerances: ConvergenceTolerances used after the optimizer returns.
• case_id: optional pytest id; by default this is like sphere[8].
• x0: optional explicit initial point. If omitted, the harness builds one from start.
• start: "near_minimizer", "random", or "zeros".
• start_radius: offset size used by "near_minimizer".
• seed: random seed used by "random".
• device: optional torch device string, for example "cuda" or "cpu".
• dtype: torch dtype, default torch.float64.
• hessian_max_dim: largest dimension for dense Hessian report metrics. Larger cases skip dense Hessian metrics to avoid slow tests.

For constraints="convex", the optimizer receives a ConvexOptimizationProblem with .instances, .parameters, and .problem for single-instance cases. If the optimizer solves the CVXPY instances in place and returns None, the harness checks objective gaps and known-solution distances when those references are available.

make_optimizer_tests

make_optimizer_tests converts an optimizer plus cases into a pytest test function.

from optfunc.testing import make_optimizer_tests

test_my_optimizer = make_optimizer_tests(
    optimizer=my_optimizer,
    cases=[case1, case2],
    name="test_my_optimizer",
)

Rules:

• Assign the returned function to a module-level variable whose name starts with test_, otherwise pytest will not collect it.
• For constraints="none", optimizer must accept one OptimizationProblem argument.
• For constraints="none", the optimizer may return either a final torch.Tensor or an OptimizerResult.
• Every case becomes an independent parametrized pytest item.

For convex cases, the optimizer accepts ConvexOptimizationProblem instead:

from optfunc.testing import (
  ConvergenceTolerances,
  ConvexOptimizationProblem,
  OptimizerCase,
  make_optimizer_tests,
)


def cvxpy_solver(problem: ConvexOptimizationProblem):
  for instance in problem:
    instance.solve()


test_convex_batch = make_optimizer_tests(
  optimizer=cvxpy_solver,
  cases=[
    OptimizerCase(
      opt_func="gw_maxcut",
      constraints="convex",
      dim=8,
      batch_size=4,
      perturb_magnitude=10.0,
      seed=0,
      tolerances=ConvergenceTolerances(value_gap=1e-5, x_distance=1e-3, grad_norm=None),
    )
  ],
)

Standard Adam Test Example

This example shows the recommended shape for a user-owned optimizer wrapper. The test harness does not hide torch.optim.Adam; the user function decides how to initialize Adam, how to use the budget, and what history to expose.

Create tests/test_torch_adam.py:

import torch

from optfunc.testing import (
  ConvergenceTolerances,
  OptimizerBudget,
  OptimizerCase,
  OptimizerResult,
  make_optimizer_tests,
  optimizer_adapter,
)


def scalar_item(value):
  return float(value.detach().cpu().item())


@optimizer_adapter(name="torch_adam")
def torch_adam(problem):
  x = problem.x0.detach().clone().requires_grad_(True)
  optimizer = torch.optim.Adam([x], lr=problem.budget.lr)
  history = []

  for step in range(1, problem.budget.max_steps + 1):
    optimizer.zero_grad(set_to_none=True)
    loss = problem.value(x)
    loss.backward()
    optimizer.step()

    with torch.no_grad():
      x.copy_(problem.project_to_bounds(x.detach()))

    if step % 25 == 0 or step == problem.budget.max_steps:
      x_now = x.detach()
      grad = problem.grad(x_now)
      history.append(
        {
          "step": step,
          "value": scalar_item(problem.value(x_now)),
          "grad_norm": scalar_item(torch.linalg.vector_norm(grad)),
          "x_norm": scalar_item(torch.linalg.vector_norm(x_now)),
        }
      )

  return OptimizerResult(
    final_x=x.detach().clone(),
    steps=problem.budget.max_steps,
    history=history,
  )


test_torch_adam = make_optimizer_tests(
  optimizer=torch_adam,
  cases=[
    OptimizerCase(
      opt_func="sphere",
      dim=8,
      budget=OptimizerBudget(max_steps=350, lr=0.05),
      tolerances=ConvergenceTolerances(
        value_gap=1e-8,
        x_distance=1e-4,
        grad_norm=1e-4,
      ),
      start="near_minimizer",
      start_radius=0.5,
    ),
    OptimizerCase(
      opt_func="rosenbrock",
      dim=4,
      budget=OptimizerBudget(max_steps=1200, lr=0.02),
      tolerances=ConvergenceTolerances(
        value_gap=5e-4,
        x_distance=5e-2,
        grad_norm=1e-2,
      ),
      start="near_minimizer",
      start_radius=0.25,
    ),
  ],
)

Run:

uv run pytest tests/test_torch_adam.py -q --tb=short --optfunc-report

The OptimizationProblem object passed into torch_adam exposes:

• problem.opt_func: the selected TorchOptFunction;
• problem.x0: the initial point for this case;
• problem.budget: the OptimizerBudget;
• problem.value(x): scalar objective value;
• problem.grad(x): gradient;
• problem.value_and_grad(x): objective and gradient;
• problem.hessian(x): dense Hessian;
• problem.hvp(x, v): Hessian-vector product;
• problem.project_to_bounds(x): clamp into the optfunc's bounds.

For second-order methods, call problem.hessian(x) or problem.hvp(x, v) inside the same optimizer wrapper and return the same OptimizerResult shape.

Built-In Adam Helper

For smoke tests or examples, optfunc.testing.make_torch_adam() provides the same Adam loop as a convenience:

from optfunc.testing import make_torch_adam, make_optimizer_tests

test_adam = make_optimizer_tests(
    optimizer=make_torch_adam(),
    cases=[...],
)

For production optimizer tests, prefer writing the small wrapper yourself so the learning rate, projection, stopping rule, and history format are explicit in your test file.

Development

Developer workflows, release steps, PyPI verification, project structure, and API design notes live in README.dev.md.

Optfunc definitions are adapted from SFU's optimization benchmark collection:

https://www.sfu.ca/~ssurjano/optimization.html