smartgrad

Smart Gradient computes finite-difference gradients in a rotated coordinate system. The rotation is built from the directions the optimizer has recently moved in, so the differences are taken along the directions that matter — which in practice reaches the optimum in fewer function evaluations than plain coordinate-wise finite differences.

This is the Python implementation. R and C++ implementations live in the same repository.

Install

pip install smartgrad

Use

Wrap your objective, then hand fun_and_grad to any optimizer that takes jac=True:

import numpy as np
from scipy.optimize import minimize
from smartgrad import SmartGradientOptimizer

def rosenbrock(x):
    return sum((1 - x[i])**2 + 100 * (x[i+1] - x[i]**2)**2
               for i in range(0, len(x), 2))

n = 4
opt = SmartGradientOptimizer(objective_fun=rosenbrock, n=n)

res = minimize(opt.fun_and_grad, np.zeros(n), method="L-BFGS-B", jac=True)
print(res.x, res.fun)

SmartGradientOptimizer takes two optional tuning arguments:

• fd_step — finite-difference step size (default 1e-5)
• noise_stddev — jitter added to the step directions, which keeps the rotation matrix from going singular when successive steps align (default 7e-8)

If you only want the gradient itself, call opt.calculate_smart_grad(x).

Example

examples/rosenbrock.py compares Smart Gradient against SciPy's built-in finite differences on the same problem:

python examples/rosenbrock.py