Skip to main content

Optimizers for/and sklearn compatible Machine Learning models

Project description

OptiML

Build Status Coverage Status Python Version PyPI Version Binder

OptiML is a sklearn compatible implementation of Support Vector Machines and Deep Neural Networks, both with some of the most successful features according to the state of art.

This work was motivated by the possibility of being able to solve the optimization problem deriving from the mathematical formulation of these models through a wide range of optimization algorithms object of study and developed for the Numerical Methods and Optimization course @ Department of Computer Science @ University of Pisa under the supervision of Antonio Frangioni.

Contents

  • Numerical Optimization

    • Unconstrained Optimization
      • Line Search Methods
        • 0th Order Methods
          • [x] Subgradient
        • 1st Order Methods
          • [x] Steepest Gradient Descent
          • [ ] Conjugate Gradient
            • [ ] Fletcher–Reeves formula
            • [ ] Polak–Ribière formula
            • [ ] Hestenes-Stiefel formula
            • [ ] Dai-Yuan formula
          • [x] Nonlinear Conjugate Gradient
            • [x] Fletcher–Reeves formula
            • [x] Polak–Ribière formula
            • [x] Hestenes-Stiefel formula
            • [x] Dai-Yuan formula
          • [x] Heavy Ball Gradient
        • 2nd Order Methods
          • [x] Newton
          • Quasi-Newton
            • [x] BFGS
            • [ ] L-BFGS
      • Stochastic Methods
        • [x] Momentum
          • [x] standard
          • [x] Nesterov
        • [x] Stochastic Gradient Descent
        • [x] Adam
        • [x] AMSGrad
        • [x] AdaMax
        • [x] AdaGrad
        • [x] AdaDelta
        • [x] RProp
        • [x] RMSProp
      • [x] Proximal Bundle with cvxpy interface to cvxopt, osqp, ecos, etc.
    • Box-Constrained Quadratic Optimization
      • [x] Projected Gradient
      • [x] Frank-Wolfe or Conditional Gradient
      • [x] Active Set
      • [x] Interior Point
      • [x] Lagrangian Dual
  • Machine Learning

    • [x] Support Vector Machines
      • Formulations
        • Primal
        • Wolfe Dual
        • Lagrangian Dual
      • [x] Support Vector Classifier
        • Losses
          • [x] Hinge (L1 Loss)
          • [x] Squared Hinge (L2 Loss)
      • [x] Support Vector Regression
        • Losses
          • [x] Epsilon-Insensitive (L1 Loss)
          • [x] Squared Epsilon-Insensitive (L2 Loss)
      • Kernels
        • [x] Linear
        • [x] Polynomial
        • [x] Gaussian
        • [x] Sigmoid
      • Optimizers (ad hoc)
    • [x] Neural Networks
      • [x] Neural Network Classifier
      • [x] Neural Network Regressor
      • Losses
        • [x] Mean Absolute Error (L1 Loss)
        • [x] Mean Squared Error (L2 Loss)
        • [x] Binary Cross Entropy
        • [x] Categorical Cross Entropy
        • [x] Sparse Categorical Cross Entropy
      • Regularizers
        • [x] L1 or Lasso
        • [x] L2 or Ridge or Tikhonov
      • Activations
        • [x] Linear
        • [x] Sigmoid
        • [x] Tanh
        • [x] ReLU
        • [x] SoftMax
      • Layers
        • [x] Fully Connected
      • Initializers
        • [x] Xavier or Glorot normal and uniform
        • [x] He normal and uniform

Install

pip install optiml

Contribution

Any bug report, bug fix or new pull request will be appreciated, especially from my classmates who will have the opportunity to deal with some of these algorithms and could be improving their performance or adding new features.

License License: MIT

This software is released under the MIT License. See the LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for optiml, version 0.0.7
Filename, size File type Python version Upload date Hashes
Filename, size optiml-0.0.7.tar.gz (3.3 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page