Skip to main content

The EVer Evolving Optimizer

Project description

EVE: The EVer Evolving Deep Learning Optimizer

Upload Python Package

EVE is a new optimizer library built on top of PyTorch that combines the best of multiple state-of-the-art optimizer algorithms into one flexible, infinitely customizable super-optimizer.

The goal of EVE is not to provide one final, static optimizer, but rather an interface to a PyTorch optimizer that will continue to implement the latest, well-tested methods from modern research.

In preliminary testing, the current implementation of EVE was able to beat Adam and other near state-of-the-art optimizers without a significant increase in compute time. Here are some inital results from training a ResNet18 on the ImageNette (subset of ImageNet that encompasses a few hard to classify classes) 5 epoch challenge.

Adam (Final Accuracy = 40.00%)

epoch train_loss valid_loss accuracy time
0 2.479557 9.522848 0.129936 00:33
1 2.223202 2.041943 0.433121 00:33
2 2.529300 2.300190 0.212994 00:34
3 2.018234 1.866597 0.347261 00:35
4 1.780924 1.732265 0.400000 00:35

EVE (Final Accuracy = 70.62%)

epoch train_loss valid_loss accuracy time
0 2.396812 2.617368 0.335287 00:39
1 2.170482 1.626544 0.478726 00:39
2 1.526003 1.672156 0.501146 00:39
3 0.956125 0.949652 0.696306 00:39
4 0.567583 0.949395 0.706242 00:39

Here are a few animations demonstrating EVE's convergence properties on simple functions:

2D Convex Surface 2D Non-Convex Surface 3D Surface with Saddle Point

Installation and Getting Started

The simplest way to use EVE in your PyTorch models is to install it using pip:

pip install eve-optimizer

Then, the main EVE optimizer can be imported as follows:

from eve.optimizers import eveo3

This will import a function that returns a torch.optim.Optimizer object, which can be used in the usual way.

The EVE library also provides a direct interface to other optimizers (like Ranger, RAdam, etc.) that were used in part or were built upon to create the main EVE optimizer. These can also be accessed from eve.optimizers in the same way.

What Exactly is EVE?

At present, EVE implements (and combines) the following algorithms:

We are currently working on adding in the following variants as well:

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

eve_optimizer-0.0.7.tar.gz (8.0 kB view details)

Uploaded Source

Built Distribution

eve_optimizer-0.0.7-py3-none-any.whl (9.8 kB view details)

Uploaded Python 3

File details

Details for the file eve_optimizer-0.0.7.tar.gz.

File metadata

  • Download URL: eve_optimizer-0.0.7.tar.gz
  • Upload date:
  • Size: 8.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.8.2

File hashes

Hashes for eve_optimizer-0.0.7.tar.gz
Algorithm Hash digest
SHA256 0c9b550fb9a6c8f0881cae59e06315b2be2f28a89e5cef5992136e60a0e14ca0
MD5 d74b2e2425447f1813699a9af37f8096
BLAKE2b-256 5137bed84d1dab16728335f5e2c64d02d3f8986d46ac707b34824f6eee323a5b

See more details on using hashes here.

File details

Details for the file eve_optimizer-0.0.7-py3-none-any.whl.

File metadata

  • Download URL: eve_optimizer-0.0.7-py3-none-any.whl
  • Upload date:
  • Size: 9.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.8.2

File hashes

Hashes for eve_optimizer-0.0.7-py3-none-any.whl
Algorithm Hash digest
SHA256 45349de2c3f88ad94d5b27acd402b0f231f0da99383ec5ae7a951e3e2e79401c
MD5 b93bc4f0672c939df1367b93e79b35f7
BLAKE2b-256 afc8abb7a0c5c5056824b4cb84493aaa1012641f86dc358adc5c7bef1be507e8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page