Package for applying gradient descent optimization algorithms
Project description
gradient-descent
gradient-descent is a package that contains different gradient-based algorithms, usually used to optimize Neural Networks and other machine learning models. The package contains the following algorithms:
- Gradients Descent
- Momentum
- RMSprop
- Nasterov accelerated gradient
- Adam
The package purpose is to facilitate the user experience when using optimization algorithms and to allow the users to have a better intuition about how this black-boxes algorithms works.
This is an open-source project, any feedback, improvement ideas, and contributors are welcome.
Installation
Dependencies
- Python (>= 3.6)
- NumPy (>= 1.13.3)
- Matplotlib (>=3.2.1)
User installation
pip install gradient-descent
Development
All contributors of all levels are welcome to help in any possible away.
Souce Code
git clone https://github.com/DanielDaCosta/gradient-descent.git
Tests
pytest tests
TO DO
The package is still on its early days and there are a lot of improvements to make:
- Build new optimization algorithms
- Extend its use for multivariable functions
- New ideas of functions for better usability
- Improve Documentation
Acknowledgements
First of all I would like to thank Hammad Shaikh by his well documented and very well explained GitHub repository Math of Machine Learning Course by Siraj
I would like to appreciate the help of the following contents and articles in the package development:
- Optimizing Gradient Descent by Sebastian Ruder
- Optimization Techniques for Gradient Descent by www.geeksforgeeks.org website
- optimization_algos GitHub repository by Iain Carmichael
- [Deep Learning] (http://www.deeplearningbook.org) by Begnio, Goodfellow and Courtville
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.