Skip to main content

Access scipy optimizers from your favorite deep learning framework.

Project description

Rosenbrock

The Dict Minimize Package

Access scipy optimizers from your favorite deep learning framework.

Installation

Only Python>=3.6 is officially supported, but older versions of Python likely work as well.

The core package itself can be installed with:

pip install dict_minimize

To also get the dependencies for all the supported frameworks (torch, JAX, tensorflow) in the README install with

pip install dict_minimize[framework]

See the GitHub, PyPI, and Read the Docs.

Example Usage

In these examples we optimize a modified Rosenbrock function. However, the arguments have been split into two chunks and stored as two entries in a dictionary. This is to illustrate how this package optimizes dictionaries of (tensor) parameters rather then vectors. We also pass in an extra shift argument to demonstrate how minimize allows extra constant arguments to be passed into the objective.

PyTorch

import torch
from dict_minimize.torch_api import minimize

def rosen_obj(params, shift):
    """Based on augmented Rosenbrock from botorch."""
    X, Y = params["x_half_a"], params["x_half_b"]
    X = X - shift
    Y = Y - shift
    obj = 100 * (X[1] - X[0] ** 2) ** 2 + 100 * (Y[1] - Y[0] ** 2) ** 2
    return obj

def d_rosen_obj(params, shift):
    obj = rosen_obj(params, shift=shift)
    da, db = torch.autograd.grad(obj, [params["x_half_a"], params["x_half_b"]])
    d_obj = OrderedDict([("x_half_a", da), ("x_half_b", db)])
    return obj, d_obj

torch.manual_seed(123)

n_a = 2
n_b = 2
shift = -1.0

params = OrderedDict([("x_half_a", torch.randn((n_a,))), ("x_half_b", torch.randn((n_b,)))])

params = minimize(d_rosen_obj, params, args=(shift,), method="L-BFGS-B", options={"disp": True})

TensorFlow

import tensorflow as tf
from dict_minimize.tensorflow_api import minimize

def rosen_obj(params, shift):
    """Based on augmented Rosenbrock from botorch."""
    X, Y = params["x_half_a"], params["x_half_b"]
    X = X - shift
    Y = Y - shift
    obj = 100 * (X[1] - X[0] ** 2) ** 2 + 100 * (Y[1] - Y[0] ** 2) ** 2
    return obj

def d_rosen_obj(params, shift):
    with tf.GradientTape(persistent=True) as t:
        t.watch(params["x_half_a"])
        t.watch(params["x_half_b"])

        obj = rosen_obj(params, shift=shift)

    da = t.gradient(obj, params["x_half_a"])
    db = t.gradient(obj, params["x_half_b"])
    d_obj = OrderedDict([("x_half_a", da), ("x_half_b", db)])
    del t  # Explicitly drop the reference to the tape
    return obj, d_obj

tf.random.set_seed(123)

n_a = 2
n_b = 2
shift = -1.0

params = OrderedDict([("x_half_a", tf.random.normal((n_a,))), ("x_half_b", tf.random.normal((n_b,)))])

params = minimize(d_rosen_obj, params, args=(shift,), method="L-BFGS-B", options={"disp": True})

NumPy

import numpy as np
from scipy.optimize import rosen, rosen_der
from dict_minimize.numpy_api import minimize

def rosen_obj(params, shift):
    val = rosen(params["x_half_a"] - shift) + rosen(params["x_half_b"] - shift)

    dval = OrderedDict(
        [
            ("x_half_a", rosen_der(params["x_half_a"] - shift)),
            ("x_half_b", rosen_der(params["x_half_b"] - shift)),
        ]
    )
    return val, dval

np.random.seed(0)

n_a = 3
n_b = 5
shift = -1.0

params = OrderedDict([("x_half_a", np.random.randn(n_a)), ("x_half_b", np.random.randn(n_b))])

params = minimize(rosen_obj, params, args=(shift,), method="L-BFGS-B", options={"disp": True})

JAX

from jax import random, value_and_grad
import jax.numpy as np
from dict_minimize.jax_api import minimize

def rosen(x):
    r = np.sum(100.0 * (x[1:] - x[:-1] ** 2.0) ** 2.0 + (1 - x[:-1]) ** 2.0, axis=0)
    return r

def rosen_obj(params, shift):
    val = rosen(params["x_half_a"] - shift) + rosen(params["x_half_b"] - shift)
    return val

n_a = 3
n_b = 5
shift = -1.0

# Jax makes it this simple:
d_rosen_obj = value_and_grad(rosen_obj, argnums=0)

# Setup randomness in JAX
key = random.PRNGKey(0)
key, subkey_a = random.split(key)
key, subkey_b = random.split(key)

params = OrderedDict(
    [("x_half_a", random.normal(subkey_a, shape=(n_a,))), ("x_half_b", random.normal(subkey_b, shape=(n_b,)))]
)

params = minimize(d_rosen_obj, params, args=(shift,), method="L-BFGS-B", options={"disp": True})

Contributing

The following instructions have been tested with Python 3.7.4 on Mac OS (10.14.6).

Install in editable mode

First, define the variables for the paths we will use:

GIT=/path/to/where/you/put/repos
ENVS=/path/to/where/you/put/virtualenvs

Then clone the repo in your git directory $GIT:

cd $GIT
git clone https://github.com/twitter/dict_minimize.git

Inside your virtual environments folder $ENVS, make the environment:

cd $ENVS
virtualenv dict_minimize --python=python3.7
source $ENVS/dict_minimize/bin/activate

Now we can install the pip dependencies. Move back into your git directory and run

cd $GIT/dict_minimize
pip install -r requirements/base.txt
pip install -e .  # Install the package itself

Contributor tools

First, we need to setup some needed tools:

cd $ENVS
virtualenv dict_minimize_tools --python=python3.7
source $ENVS/dict_minimize_tools/bin/activate
pip install -r $GIT/dict_minimize/requirements/tools.txt

To install the pre-commit hooks for contributing run (in the dict_minimize_tools environment):

cd $GIT/dict_minimize
pre-commit install

To rebuild the requirements, we can run:

cd $GIT/dict_minimize

# Check if there any discrepancies in the .in files
pipreqs dict_minimize/core/ --diff requirements/base.in
pipreqs dict_minimize/ --diff requirements/frameworks.in
pipreqs tests/ --diff requirements/tests.in
pipreqs docs/ --diff requirements/docs.in

# Regenerate the .txt files from .in files
pip-compile-multi --no-upgrade

Generating the documentation

First setup the environment for building with Sphinx:

cd $ENVS
virtualenv dict_minimize_docs --python=python3.7
source $ENVS/dict_minimize_docs/bin/activate
pip install -r $GIT/dict_minimize/requirements/docs.txt

Then we can do the build:

cd $GIT/dict_minimize/docs
make all
open _build/html/index.html

Documentation will be available in all formats in Makefile. Use make html to only generate the HTML documentation.

Running the tests

The tests for this package can be run with:

cd $GIT/dict_minimize
./local_test.sh

The script creates an environment using the requirements found in requirements/test.txt. A code coverage report will also be produced in $GIT/dict_minimize/htmlcov/index.html.

Deployment

The wheel (tar ball) for deployment as a pip installable package can be built using the script:

cd $GIT/dict_minimize/
./build_wheel.sh

This script will only run if the git repo is clean, i.e., first run git clean -x -ff -d.

License

This project is licensed under the Apache 2 License - see the LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dict_minimize-0.0.4.tar.gz (15.2 kB view details)

Uploaded Source

File details

Details for the file dict_minimize-0.0.4.tar.gz.

File metadata

  • Download URL: dict_minimize-0.0.4.tar.gz
  • Upload date:
  • Size: 15.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.24.0 setuptools/47.3.1 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.7.4

File hashes

Hashes for dict_minimize-0.0.4.tar.gz
Algorithm Hash digest
SHA256 3c279c82c5a25839507ec66835b803ce6560ae7c3f36bf3f19538b8811e65372
MD5 cb9d399819b31c12975d9cbca60a2768
BLAKE2b-256 44429e26e9e17d18ed4eb623d312fc1dfcfb920a286b4d63d811f194b10b7f60

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page