invexapi

A minimal, PyTorch-only toolkit for invex optimization: penalties with proven optimality certificates, and the solvers that use them.

Why invexity?

Invex functions are a strict generalization of convexity: every stationary point is still a global minimum, but the function itself need not be convex. That gives non-convex penalties (sparsity-promoting, non-smooth, highly structured) the same global-optimality guarantee convex optimization enjoys - without paying for it with local minima.

invexapi packages this idea as code: penalties that carry a machine-checkable certificate of which mathematical class they belong to (convex, invex, quasi-convex, quasi-invex), and generic solvers that read those certificates to warn you when no optimality guarantee applies.

Install

pip install -e .              # editable install
pip install -e ".[dev]"       # + pytest, numpy for the test suite
pip install -e ".[examples]"  # + deps used only by examples/

Quick start

import torch
from invexapi import QuasinormInvexPenalty
from invexapi.optim import FISTA

class DataFidelity:
    def __init__(self, y): self.y = y
    def value(self, x): return 0.5 * torch.sum((x - self.y) ** 2)
    def grad(self, x): return x - self.y

y = torch.randn(100)
smooth = DataFidelity(y)
penalty = QuasinormInvexPenalty(lamb=0.1, q=0.5)
solver = FISTA(smooth, penalty, step=1.0)

x_hat, history = solver.run(y.clone())

Any object exposing the right methods (value, grad, prox) works as a smooth/penalty/objective - the built-in penalties are just one plug-in choice.

What's inside

Penalties (invexapi.penalties) - loss/penalty terms plus a certificate of what's provably known about each one:

Certificates are attached explicitly and never inferred - convexity composes additively, but invexity does not, so a combined objective only carries a certificate someone has actually proven for it.

Solvers (invexapi.optim) - generic, decoupled from the penalties above:

• GradientDescent - with optional Armijo backtracking line search
• FISTA - accelerated proximal gradient for smooth(x) + penalty(x)
• NonlinearCG - Polak-Ribière+ with automatic restart
• LinearizedADMM - for smooth(x) + penalty(D@x) (e.g. total variation)

All four warn (never error) when run on an objective without a convex/invex certificate, since no global-optimum guarantee applies in that case.

Linear operators (invexapi.penalties.operators) - Identity and FiniteDifference2D (2D total variation), each with a verified adjoint.

Structured documentation (invexapi.metadata) - design provenance, rejected alternatives, and invariants as introspectable dataclasses rather than prose, exportable as JSON for downstream tooling:

import invexapi
print(invexapi.metadata.dump_all_json(indent=2))

GPU support

Nothing in invexapi is device-specific - every operation derives its device from its input tensors. Move your data to CUDA before calling a solver and everything downstream follows:

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
x_hat, history = solver.run(y.to(device))

Testing

pytest

Penalty math is validated against independent NumPy reference implementations on fixed-seed random inputs, with no CUDA dependency anywhere in this repo.

Examples

See examples/ for denoising and deblurring scripts covering FISTA and Linearized ADMM with total variation, using both the quasinorm and log invex penalties. The ADMM/TV examples need a test image, fetched separately:

python examples/data/download.py

License

BSD 3-Clause, see LICENSE.

References

This library reproduces and generalizes results from the following papers:

1. Pinilla, S., Sanabria, A., Bi, J., & Egiazarian, K. (2025). What makes neural networks trainable? Invexity as a structural design principle in AI.
2. Pinilla, S., & Thiyagalingam, J. (2024). Global optimality for non-linear constrained restoration problems via invexity. International Conference on Learning Representations (ICLR), 2024, pp. 11990–12027.
3. Pinilla, S., Mu, T., Bourne, N., & Thiyagalingam, J. (2022). Improved imaging by invex regularizers with global optima guarantees. Advances in Neural Information Processing Systems (NeurIPS), 35, pp. 10780–10794.
4. Pinilla, S., Yeung, S.-L., & Thiyagalingam, J. (2024). Global convergence of alternating direction method of multipliers for invex objective losses. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2024, pp. 9361–9365.