anchorax
Fixed-point solvers and differentiation utilities for implicit neural networks in JAX/Equinox.
- Solvers: Picard, Anderson acceleration, and limited-memory Broyden.
- Backpropagation: JFB (Jacobian-Free Backprop) and GDEQ-style adjoint preconditioning.
Install
pip install anchorax
Requirements
- einops>=0.8.1
- equinox>=0.12.1
- jax>=0.6.0
Quickstart
Define your implicit update f(u, Qd) as an Equinox module. The solvers will call it as f(u, Qd, inference=..., key=...).
import equinox as eqx
import jax
import jax.numpy as jnp
class ImplicitBlock(eqx.Module):
w: jnp.ndarray
b: jnp.ndarray
def __call__(self, u, Qd, *, inference: bool = False, key=None):
# example: contractive residual layer
return jnp.tanh(u @ self.w + Qd + self.b)
# Instantiate and partition into trainable params and static parts
model = ImplicitBlock(
w=jax.random.normal(jax.random.PRNGKey(0), (128, 128)) * 0.1,
b=jnp.zeros((128,)),
)
params, static = eqx.partition(model, eqx.is_array)
Qd = jnp.zeros((128,))
Solve with any solver (same API and outputs):
from anchorax.solvers import broyden, picard, anderson
key = jax.random.PRNGKey(42)
u_star, u_prev, tnstep = broyden(params, static, Qd, inference=True, key=key)
# Or:
u_star, u_prev, tnstep = picard(params, static, Qd, inference=True, key=key)
u_star, u_prev, tnstep = anderson(params, static, Qd, inference=True, key=key)
Train end-to-end with GDEQ or JFB:
import optax
from anchorax.backprop import gdeq, jfb
opt = optax.adam(1e-3)
opt_state = opt.init(eqx.filter(params, eqx.is_array))
def loss_fn(train_params, key):
u_star, _, _ = gdeq((train_params, Qd), static, solver=broyden, inference=False, key=key)
return jnp.mean(u_star**2)
loss_and_grad = eqx.filter_value_and_grad(loss_fn)
for step in range(1000):
key, key_step = jax.random.split(key)
loss, grads = loss_and_grad(params, key_step)
updates, opt_state = opt.update(grads, opt_state, params)
params = eqx.apply_updates(params, updates)
API summary
All solvers share the same signature and semantics:
u_star, u_prev, tnstep = solver(
params,
static,
Qd,
*,
u0=None,
eps=1e-3,
max_depth=100,
stop_mode="abs", # "abs" or "rel", on residual g(u)=f(u)-u
LBFGS_thres=3, # used by broyden; accepted by others for API uniformity
ls=False, # Armijo line search (broyden only)
return_final=False, # if False, return best-so-far; if True, last iterate
return_B=False, # if True, also return (Us, VTs, valid_mask)
inference=False,
key=...,
**kwargs,
)
- Returns:
u_star: final implicit state (best-so-far unlessreturn_final=True)u_prev: the iterate immediately precedingu_star(used for accurate autodiff)tnstep: total forward evaluations offperformed by the solver- Optionally
(Us, VTs, valid_mask)whenreturn_B=True(Broyden’s limited-memory inverse-Jacobian factors)
Backprop wrappers:
# JFB: last-step VJP only (baseline)
u_star, u_prev, tnstep = jfb((params, Qd), static, solver=picard, inference=False, key=key)
# GDEQ: last-step VJP + adjoint preconditioning with inverse-Jacobian factors
u_star, u_prev, tnstep = gdeq((params, Qd), static, solver=broyden, inference=False, key=key)
- With Picard/Anderson plus
gdeq, the stored factors act as a no-op preconditioner (degrades to JFB), ensuring compatibility.
Design notes
- Residual and stopping:
- Residual is
g(u) = f(u, Qd) - u. stop_mode="abs"uses||g(u)|| < eps.stop_mode="rel"uses||g(u)|| / (||g(u)+u|| + 1e-9) < eps.
- Residual is
- Determinism and randomness:
- Each solver accepts
keyand folds in the iteration index for every call tof. This ensures deterministic behavior of stochastic layers per-iteration.
- Each solver accepts
- JIT and static args:
- The following are treated as static for compilation:
static,eps,max_depth,stop_mode,LBFGS_thres,ls,return_final,return_B. Changing them triggers recompilation.
- The following are treated as static for compilation:
- Autodiff accuracy:
- Both
jfbandgdeqrecompute the final explicit step asu_star = f(stop_gradient(u_prev), Qd)under inference mode and use its VJP for gradients. This aligns training and inference and avoids off-by-one iteration effects.
- Both
References
- Broyden, C. G. (1965). "A Class of Methods for Solving Nonlinear Simultaneous Equations". Mathematics of Computation. 19 (92). American Mathematical Society: 577–593. doi:10.1090/S0025-5718-1965-0198670-6. JSTOR 2003941.
- Anderson, Donald G. (October 1965). "Iterative Procedures for Nonlinear Integral Equations". Journal of the ACM. 12 (4): 547–560. doi:10.1145/321296.321305.
- Fung et al. (2021). "JFB: Jacobian-Free Backpropagation for Implicit Networks". arXiv:2103.12803.
- Nguyen, B. et al. (2023). "Efficient Training of Deep Equilibrium Models". arXiv:2304.11663.
Metadata
Release files for anchorax 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| anchorax-0.1.0.tar.gz | 14.1 kB | Without provenance | Details for anchorax-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| anchorax-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for anchorax-0.1.0-py3-none-any.whl |
Total release size: 27.2 kB
Release files / anchorax-0.1.0.tar.gz
| Download URL | anchorax-0.1.0.tar.gz |
|---|---|
| Size | 14.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0872ede947b8de99b7f7821e390a738b1f9f6f8208b2afe433899dc33f16129d
|
|
BLAKE2b-256 checksum How to use checksums |
f098e840d487613464ea54c5c2c33f5f440637a3bcf88612d6d02be1d88cdb78
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.14
|
Release files / anchorax-0.1.0-py3-none-any.whl
| Download URL | anchorax-0.1.0-py3-none-any.whl |
|---|---|
| Size | 13.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
28b6a4b294566f1faf8119af4b379129c3f7650958ee32b4637816a26aee4a0c
|
|
BLAKE2b-256 checksum How to use checksums |
2b4df42696bbbe7d52528cbe635979ee63026249e585be3b9f688923629c6d59
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.14
|
Security for anchorax 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.