Classical gradient based optimization in PyTorch
Project description
GradOpTorch
Classical gradient based optimization in PyTorch.
What is GradOpTorch?
GradOpTorch is a suite of classical gradient-based optimization tools for PyTorch. The toolkit includes conjugate gradients, BFGS, and some methods for line-search.
Why not torch.optim?
Not every problem is high-dimensional, nonlinear, with noisy gradients.
For such problems, classical optimization techniques
can be more efficient.
Installation
GradOpTorch can be installed from PyPI:
pip install gradoptorch
Usage
There are two primary interfaces for making use of the library.
- The standard PyTorch object oriented interface:
from gradoptorch import optimize_module
from torch import nn
class MyModel(nn.Module):
...
model = MyModule()
def loss_fn(model):
...
hist = optimize_module(model, loss_fn, opt_method="bfgs", ls_method="back_tracking")
- The functional interface:
from gradoptorch import optimizer
def f(x):
...
x_guess = ...
x_opt, hist = optimizer(f, x_guess, opt_method="conj_grad_pr", ls_method="quad_search")
Newton's method is only available in the functional interface
Included optimizers:
'grad_exact' : exact gradient optimization
'conj_grad_fr' : conjugate gradient descent using Fletcher-Reeves search direction
'conj_grad_pr' : conjugate gradient descent using Polak-Ribiere search direction
'newton_exact' : exact newton optimization
'bfgs' : approximate newton optimization using bfgs
Included line-search methods:
'back_tracking' : backing tracking based line-search
'quad_search' : quadratic line-search
'constant' : no line search, constant step size used
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
gradoptorch-0.1.0.tar.gz
(7.7 kB
view hashes)
Built Distribution
Close
Hashes for gradoptorch-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 93522aaef643a66994ea386f52d158435272110bb992096d5a44d9a3e10b9c79 |
|
MD5 | 00c0b02e4b5dfa96db5904f599058e7a |
|
BLAKE2b-256 | 86ca9e5473bc2443ff72e75ab45dfd6e6ed25e01e42ca4267d9efb07eadf8ef1 |