keras-one-cycle-lr 0.0.1

Keras implementation of One Cycle Policy and LR Finder

Keras-training-tools

Implementation of some of the very effective tools for training Deep Learning (DL) models that I came across while doing the fastai course on Practical Deep Learning for Coders.

The tools were first presented in the following papers by Leslie N. Smith:

• LR Finder: A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
• One Cycle Scheduler: Cyclical Learning Rates for Training Neural Networks

My implementations are a port of the code in fastai library (originally, based on Pytorch) to Keras and are heavily inspired by some of earlier efforts in this direction:

• https://github.com/surmenok/keras_lr_finder
• https://github.com/titu1994/keras-one-cycle

Here's another article I referred to: How Do You Find A Good Learning Rate by Sylvain Gugger of fastai which provides an intuitive understanding of how fastai's LR finder works.

I'll keep updating this repository with the new tools I come across that could be practically useful for training a DL model.