optfuncs 0.0.9

Optimization benchmark functions and pytest-oriented evaluation helpers.

optfunc

optfunc provides differentiable PyTorch optfuncs, CVXPY-backed convex benchmarks, and pytest-oriented helpers for evaluating first-order and second-order optimizers.

The package is published as optfuncs and imported as optfunc.

Detailed user guides live under docs/.

Install

For CPU-only use:

pip install optfuncs

For convex-only CVXPY benchmarks:

pip install "optfuncs[convex]"

For CPU-only use plus convex benchmarks:

pip install "optfuncs[torch-cpu,convex]"

With uv in this repository:

uv sync --group torch-cpu

To enable the CVXPY convex benchmark suite:

uv sync --extra convex

Supported extras:

Extra	Backend	Typical command
convex	CVXPY convex benchmarks, default MOSEK reference solve with SCS fallback	uv sync --extra convex
torch-cpu	CPU Torch runtime plus CPU native cone backend	pip install "optfuncs[torch-cpu]"
torch-cu130	Torch CUDA 13.x plus optfuncs-cuda130 CUDA cone addon	pip install "optfuncs[torch-cu130]"
cupy-cu13	CuPy CUDA 13.x plus optfuncs-cuda130 CUDA cone addon	pip install "optfuncs[cupy-cu13]"

Torch hardware variants are also available as local uv dependency groups. Only choose one Torch group at a time:

Local group	Backend	Typical command
torch-cpu	CPU, Linux/macOS/Windows	uv sync --group torch-cpu
torch-cu118	CUDA 11.8, Linux/Windows, PyTorch 2.7.x	uv sync --group torch-cu118
torch-cu126	CUDA 12.6, Linux/Windows	uv sync --group torch-cu126
torch-cu128	CUDA 12.8, Linux/Windows	uv sync --group torch-cu128
torch-cu130	CUDA 13.0, Linux/Windows	uv sync --group torch-cu130
torch-rocm	ROCm 7.1, Linux	uv sync --group torch-rocm
torch-xpu	Intel XPU, Linux/Windows	uv sync --group torch-xpu
cupy-cu13	CuPy for CUDA 13.x cone interoperability tests	uv sync --group cupy-cu13

The CUDA 13 convex development environment is:

uv sync --group torch-cu130 --group cupy-cu13 --extra convex --dev

Detailed CPU/CUDA wheel behavior is documented in docs/native-wheel-variants.md.

Using optfuncs from another uv project

When another uv project wants to expose multiple optional dependency sets, add the convex/base benchmark support and the Torch backend support separately. This keeps the target project's extras explicit and lets users choose the Torch variant independently. optfuncs[convex] does not install or require Torch; add a torch-* extra only when the target project also uses unconstrained PyTorch optfuncs.

Recommended target-project extra layout:

Target extra	Include
test	optfuncs[convex]>=0.0.7 for pytest helpers and CVXPY convex benchmarks
torch	optfuncs[torch-cu130]>=0.0.7 or another explicit hardware extra

Example:

uv add --optional test "optfuncs[convex]>=0.0.7"
uv add --optional torch "optfuncs[torch-cu130]>=0.0.7"

For a CPU project, use:

uv add --optional test "optfuncs[convex]>=0.0.7"
uv add --optional torch "optfuncs[torch-cpu]>=0.0.7"

Then install the combination you need from the target project:

uv sync --extra test --extra torch

Keep the target project's Torch source/index selection aligned with the selected optfuncs[...] hardware extra.

Helper scripts:

# bash/zsh on Linux or macOS, and Git Bash/MSYS/Cygwin on Windows
./scripts/sync_torch_variant.sh cpu
./scripts/sync_torch_variant.sh cu130
./scripts/sync_torch_variant.sh rocm
./scripts/sync_torch_variant.sh xpu

# Windows PowerShell or PowerShell 7 on Windows/Linux/macOS
.\scripts\sync_torch_variant.ps1 cpu
.\scripts\sync_torch_variant.ps1 cu130
.\scripts\sync_torch_variant.ps1 rocm
.\scripts\sync_torch_variant.ps1 xpu

If Windows blocks local PowerShell scripts, run:

powershell -ExecutionPolicy Bypass -File .\scripts\sync_torch_variant.ps1 cu130

Platform notes:

• CPU works on Linux, macOS, and Windows.
• CUDA extras work on Linux and Windows.
• ROCm works on Linux.
• XPU works on Linux and Windows.
• macOS should use torch-cpu in this project configuration.

The backend indexes follow the official uv PyTorch guide and PyTorch install selector:

• https://docs.astral.sh/uv/guides/integration/pytorch/
• https://pytorch.org/get-started/locally/

Optfunc Usage

The repository provides these built-in optfuncs:

Registry name	Class	Known minimizer
ackley	Ackley	all zeros
dixonprice	DixonPrice	recursive Dixon-Price optimum
griewank	Griewank	all zeros
levy	Levy	all ones
rastrigin	Rastrigin	all zeros
rosenbrock	Rosenbrock	all ones
rotatedhyperellipsoid	RotatedHyperEllipsoid	all zeros
schwefel	Schwefel	near 420.968746 in every coordinate
sphere	Sphere	all zeros
styblinskitang	StyblinskiTang	near -2.903534 in every coordinate
sumsquares	SumSquares	all zeros
trid	Trid	x_i = i * (d + 1 - i) with 1-based indexing
zakharov	Zakharov	all zeros

Use a class directly when you know which function you want:

import torch
from optfunc import Sphere

opt_func = Sphere(dim=8, dtype=torch.float64)
x = torch.zeros(8, dtype=torch.float64)

value = opt_func(x)
grad = opt_func.grad(x)
hessian = opt_func.hessian(x)
hvp = opt_func.hvp(x, torch.ones_like(x))
x_star = opt_func.global_minimizer()
distance = opt_func.distance_to_optimum(x)

Use OptFuncRegistry when a test should select an optfunc by name:

from optfunc import OptFuncRegistry

opt_func = OptFuncRegistry.create("rosenbrock", dim=4)
print(OptFuncRegistry.available())

Use BenchmarkRegistry when the caller may switch between unconstrained functions and one concrete convex benchmark instance:

from optfunc import BenchmarkRegistry

convex_problem = BenchmarkRegistry.create(
    "gw_maxcut",
    constraints="convex",
    dim=8,
    seed=19,
)

optimal_value = convex_problem.meta.optimal_value
X_star = convex_problem.known_solution("X")
cpp_data = convex_problem.problem_data
cvxpy_problem = convex_problem.problem  # lazily constructed when requested

Use ConvexFamily when you want a family of parameterised convex programs:

from optfunc import ConvexFamily

family = ConvexFamily.create("gw_maxcut", dim=8)

theta = family.sample_parameters(seed=0)
problem = family.build_instance(theta)

theta_sequence = family.sample_parameters(5, seed=0)
problem_sequence = family.build_instance(theta_sequence)
same_problem_sequence = family.generate_sequence(5, seed=0)

Each optfunc uses the same conventions:

• input x is a 1-D PyTorch tensor with shape (dim,);
• output is a scalar tensor;
• grad, hessian, and hvp use PyTorch autograd unless a subclass provides a better implementation;
• global_minimizer() returns a known theoretical minimizer when available;
• distance_to_optimum(x) defaults to Euclidean distance to global_minimizer();
• project_to_bounds(x) clamps a point into the optfunc's documented search box.

Convex Benchmark Usage

The repository also provides CVXPY convex benchmarks behind constraints="convex":

Registry name	Cone	Shape parameter	Validation path
zero, zero_cone_qp	zero cone / equality constraints	vector dim	known optimum
nonneg, nonnegative_cone_qp	nonnegative cone	vector dim	known optimum
psd, psd_cone_projection	PSD cone	matrix side length dim	known optimum
gw_maxcut, maxcut_sdp	PSD cone	graph vertex count dim	analytic optimum

Each convex benchmark family is generated from ConvexFamily and exposes ProblemFamily methods:

• sample_parameters(seed=0): sample one theta dictionary;
• sample_parameters(n, seed=0): sample a uniform theta sequence;
• sample_theta_*: family-specific sampling methods with the same single-or-many convention;
• build_instance(theta_or_theta_sequence): turn theta into one cpp-native problem instance or a list of instances;
• perturb(theta, n=None, magnitude=..., seed=0): produce one or more nearby theta dictionaries;
• perturb_*: family-specific perturbation methods that match the corresponding sample_theta_* naming;
• generate_sequence(n, seed=0): build a cpp-native problem sequence from the default seed theta sampler.

Each concrete CppNativeProblem exposes:

• problem.to_cvxpy_problem() / problem.problem: the CVXPY problem converted from the native description;
• problem.data: deterministic random instance data used to build the problem;
• problem.problem_data: Clarabel-style canonical data P, q, A, b, and ordered cone blocks;
• problem.cpp_native / problem.to_problem_data(): the native conic data used by cpp solver integrations;
• problem.solve(...): solve with CVXPY, defaulting to MOSEK and falling back to SCS if MOSEK is unavailable in the current environment;
• problem.known_solution() and problem.distance_to_optimum() when the theoretical solution is encoded in the benchmark definition.

The internal conic data follows Clarabel's A x + s = b, s in K convention. Cone blocks are stored in the order zero, nonnegative, second_order, psd, then exponential. Use optfunc.cvxs.sort_cones, add_cone_block, delete_cone_block, and to_ptf_text when building or exporting solver-specific formats.

The gw_maxcut benchmark is a seeded Goemans-Williamson Max-Cut SDP relaxation on a complete weighted bipartite graph. The seed fixes both the planted cut and the positive edge weights. Its SDP relaxation optimum is known without calling a solver: problem.meta.optimal_value is the total planted cut weight and problem.known_solution("X") is the rank-one planted cut matrix.

If a convex benchmark does not have a closed-form theoretical solution yet, you can leave that solution unspecified and use problem.solve() as the reference value/solution path. The default reference solver is MOSEK.

The sequence API uses numpy.random.Generator for reproducible parameter sampling, so ConvexFamily native problem generation and CVXPY conversion work with the convex extra. Torch is required only for constraints="none" PyTorch optfuncs.

Detailed Clarabel and gw_maxcut benchmark examples are available in docs/convex-benchmark-usage.md.

Using C++ Cone Operators In External Conic Solvers

External conic solvers can use the cone descriptor and operator packages without using CVXPY benchmark families. The minimum runtime path is NumPy plus the compiled optfunc.cvxs.cones.cpp extension; convex, SciPy, CVXPY, Torch, and CuPy are only needed when the downstream project explicitly uses those features.

Recommended imports:

import numpy as np

from optfunc.cvxs.cones import NonnegativeCone, PsdCone, make_cone_operator
from optfunc.cvxs.cones.cpp import NonnegativeOperatorCpp

cone = NonnegativeCone(3, name="ineq")
operator = make_cone_operator(cone, backend="cpp")

x = np.asarray([-1.0, 2.0, -3.0], dtype=np.float64)
projected = np.empty_like(x)
operator.project_into(x, projected)

assert projected.tolist() == [0.0, 2.0, 0.0]
assert operator.to_descriptor(name=cone.name) == cone
assert NonnegativeOperatorCpp(3).rows == cone.rows

The descriptor classes ZeroCone, NonnegativeCone, SecondOrderCone, PsdCone, and ExponentialCone describe Clarabel-style cone metadata only: kind, dim, rows, and optional name. The C++ operator classes implement operations on cone vectors: project, project_into, contains, violation, basis, unit_vector_into, project_batch_into, and workspace/fused helpers.

Downstream solvers may also pass their own descriptor objects to make_cone_operator when they expose compatible attributes:

from dataclasses import dataclass


@dataclass(frozen=True)
class SolverCone:
    kind: str
    dim: int
    rows: int
    name: str | None = None


operator = make_cone_operator(
    SolverCone(kind="psd", dim=2, rows=3, name="sdp_block"),
    backend="cpp",
)

Use optfunc.cvxs.native.ConicProblemData and the P, q, A, b builder helpers only when you want optfuncs to assemble Clarabel-style problem data. Those utilities require SciPy sparse matrices. CVXPY-backed benchmark families and translators require the convex extra.

Detailed installed-package guidance for external solver integration is available in docs/cpp-cone-operators.md.

Pytest Optimizer Evaluation

The optimizer evaluation API lives in optfunc.testing. A standard test file has three parts:

1. Define or import an optimizer function.
2. Configure one or more OptimizerCase objects.
3. Assign make_optimizer_tests(...) to a pytest-visible name such as test_my_optimizer.

Run the file with:

uv run pytest tests/test_my_optimizer.py -q --tb=short --optfunc-report

Each OptimizerCase becomes an independent pytest item. If one case fails or raises an exception, pytest continues running the remaining cases. --optfunc-report prints a final serial summary with function gap, distance to the theoretical optimum, gradient norm, Hessian information, step count, and error messages. Do not combine --optfunc-report with pytest-xdist -n in v1.

OptimizerBudget

OptimizerBudget describes the budget passed to the optimizer.

from optfunc.testing import OptimizerBudget

budget = OptimizerBudget(max_steps=500, lr=0.05)

Fields:

• max_steps: positive integer iteration limit.
• lr: positive learning-rate-like scalar. The test harness does not enforce how the optimizer uses it; your optimizer reads it from problem.budget.lr.

For torch.optim.Adam, a typical use is:

optimizer = torch.optim.Adam([x], lr=problem.budget.lr)
for step in range(problem.budget.max_steps):
    ...

ConvergenceTolerances

ConvergenceTolerances decides whether a finished optimizer run passes.

from optfunc.testing import ConvergenceTolerances

tolerances = ConvergenceTolerances(
    value_gap=1e-8,
    x_distance=1e-4,
    grad_norm=1e-4,
    hessian_min_eig=0.0,
)

Fields:

• value_gap: maximum allowed absolute gap between final value and the known theoretical optimum value. Set to None to skip this check.
• x_distance: maximum allowed distance from final x to the known theoretical minimizer. Set to None to skip this check.
• grad_norm: maximum allowed Euclidean norm of the final gradient. Set to None to skip this check.
• hessian_min_eig: optional lower bound on the final Hessian's smallest eigenvalue. Set to None to skip this check.

The report still computes available metrics even when a tolerance is None; None only disables that pass/fail check.

OptimizerCase

OptimizerCase describes one pytest item.

from optfunc.testing import OptimizerCase
from optfunc.testing import OptimizerBudget, ConvergenceTolerances

case = OptimizerCase(
  opt_func="sphere",
  constraints="none",
  dim=8,
  budget=OptimizerBudget(max_steps=350, lr=0.05),
  tolerances=ConvergenceTolerances(value_gap=1e-8, x_distance=1e-4, grad_norm=1e-4),
  start="near_minimizer",
  start_radius=0.5,
  seed=0,
  hessian_max_dim=32,
)

Important fields:

• opt_func: registry name such as "sphere" or an already-created TorchOptFunction instance.
• constraints: "none" for the existing Torch optfunc suite, or "convex" for the CVXPY convex benchmark suite.
• dim: required when opt_func is a string.
• batch_size: for constraints="convex", generate this many same-family problem instances.
• perturb_magnitude: for convex batches, controls successive parameter perturbations through ProblemFamily.perturb(...); when it is zero, the harness uses ProblemFamily.sample_parameters(n, seed=...).
• budget: OptimizerBudget passed to the optimizer through problem.budget.
• tolerances: ConvergenceTolerances used after the optimizer returns.
• case_id: optional pytest id; by default this is like sphere[8].
• x0: optional explicit initial point. If omitted, the harness builds one from start.
• start: "near_minimizer", "random", or "zeros".
• start_radius: offset size used by "near_minimizer".
• seed: random seed used by "random".
• device: optional torch device string, for example "cuda" or "cpu".
• dtype: torch dtype, default torch.float64.
• hessian_max_dim: largest dimension for dense Hessian report metrics. Larger cases skip dense Hessian metrics to avoid slow tests.

For constraints="convex", the optimizer receives a ConvexOptimizationProblem with .instances, .parameters, and .problem for single-instance cases. If the optimizer solves the CVXPY instances in place and returns None, the harness checks objective gaps and known-solution distances when those references are available.

make_optimizer_tests

make_optimizer_tests converts an optimizer plus cases into a pytest test function.

from optfunc.testing import make_optimizer_tests

test_my_optimizer = make_optimizer_tests(
    optimizer=my_optimizer,
    cases=[case1, case2],
    name="test_my_optimizer",
)

Rules:

• Assign the returned function to a module-level variable whose name starts with test_, otherwise pytest will not collect it.
• For constraints="none", optimizer must accept one OptimizationProblem argument.
• For constraints="none", the optimizer may return either a final torch.Tensor or an OptimizerResult.
• Every case becomes an independent parametrized pytest item.

For convex cases, the optimizer accepts ConvexOptimizationProblem instead:

from optfunc.testing import (
  ConvergenceTolerances,
  ConvexOptimizationProblem,
  OptimizerCase,
  make_optimizer_tests,
)


def cvxpy_solver(problem: ConvexOptimizationProblem):
  for instance in problem:
    instance.solve()


test_convex_batch = make_optimizer_tests(
  optimizer=cvxpy_solver,
  cases=[
    OptimizerCase(
      opt_func="gw_maxcut",
      constraints="convex",
      dim=8,
      batch_size=4,
      perturb_magnitude=10.0,
      seed=0,
      tolerances=ConvergenceTolerances(value_gap=1e-5, x_distance=1e-3, grad_norm=None),
    )
  ],
)

Standard Adam Test Example

This example shows the recommended shape for a user-owned optimizer wrapper. The test harness does not hide torch.optim.Adam; the user function decides how to initialize Adam, how to use the budget, and what history to expose.

Create tests/test_torch_adam.py:

import torch

from optfunc.testing import (
  ConvergenceTolerances,
  OptimizerBudget,
  OptimizerCase,
  OptimizerResult,
  make_optimizer_tests,
  optimizer_adapter,
)


def scalar_item(value):
  return float(value.detach().cpu().item())


@optimizer_adapter(name="torch_adam")
def torch_adam(problem):
  x = problem.x0.detach().clone().requires_grad_(True)
  optimizer = torch.optim.Adam([x], lr=problem.budget.lr)
  history = []

  for step in range(1, problem.budget.max_steps + 1):
    optimizer.zero_grad(set_to_none=True)
    loss = problem.value(x)
    loss.backward()
    optimizer.step()

    with torch.no_grad():
      x.copy_(problem.project_to_bounds(x.detach()))

    if step % 25 == 0 or step == problem.budget.max_steps:
      x_now = x.detach()
      grad = problem.grad(x_now)
      history.append(
        {
          "step": step,
          "value": scalar_item(problem.value(x_now)),
          "grad_norm": scalar_item(torch.linalg.vector_norm(grad)),
          "x_norm": scalar_item(torch.linalg.vector_norm(x_now)),
        }
      )

  return OptimizerResult(
    final_x=x.detach().clone(),
    steps=problem.budget.max_steps,
    history=history,
  )


test_torch_adam = make_optimizer_tests(
  optimizer=torch_adam,
  cases=[
    OptimizerCase(
      opt_func="sphere",
      dim=8,
      budget=OptimizerBudget(max_steps=350, lr=0.05),
      tolerances=ConvergenceTolerances(
        value_gap=1e-8,
        x_distance=1e-4,
        grad_norm=1e-4,
      ),
      start="near_minimizer",
      start_radius=0.5,
    ),
    OptimizerCase(
      opt_func="rosenbrock",
      dim=4,
      budget=OptimizerBudget(max_steps=1200, lr=0.02),
      tolerances=ConvergenceTolerances(
        value_gap=5e-4,
        x_distance=5e-2,
        grad_norm=1e-2,
      ),
      start="near_minimizer",
      start_radius=0.25,
    ),
  ],
)

Run:

uv run pytest tests/test_torch_adam.py -q --tb=short --optfunc-report

The OptimizationProblem object passed into torch_adam exposes:

• problem.opt_func: the selected TorchOptFunction;
• problem.x0: the initial point for this case;
• problem.budget: the OptimizerBudget;
• problem.value(x): scalar objective value;
• problem.grad(x): gradient;
• problem.value_and_grad(x): objective and gradient;
• problem.hessian(x): dense Hessian;
• problem.hvp(x, v): Hessian-vector product;
• problem.project_to_bounds(x): clamp into the optfunc's bounds.

For second-order methods, call problem.hessian(x) or problem.hvp(x, v) inside the same optimizer wrapper and return the same OptimizerResult shape.

Built-In Adam Helper

For smoke tests or examples, optfunc.testing.make_torch_adam() provides the same Adam loop as a convenience:

from optfunc.testing import make_torch_adam, make_optimizer_tests

test_adam = make_optimizer_tests(
    optimizer=make_torch_adam(),
    cases=[...],
)

For production optimizer tests, prefer writing the small wrapper yourself so the learning rate, projection, stopping rule, and history format are explicit in your test file.

Development

Developer workflows, release steps, PyPI verification, project structure, and API design notes live in README.dev.md.

Optfunc definitions are adapted from SFU's optimization benchmark collection:

https://www.sfu.ca/~ssurjano/optimization.html