Skip to main content

Learning rate multiplier

Project description

Keras LR Multiplier

Travis Coverage Version Downloads

[中文|English]

Learning rate multiplier wrapper for optimizers.

Install

pip install keras-lr-multiplier

Usage

Basic

LRMultiplier is a wrapper for optimizers to assign different learning rates to specific layers (or weights). The first argument is the original optimizer which could be either an identifier (e.g. 'Adam') or an initialized object (e.g. Adam(lr=1e-2)). The second argument is a dict that maps prefixes to learning rate multipliers. The multiplier for a weight is the value mapped from the longest matched prefix in the given dict, and the default multiplier 1.0 will be used if there is no prefix matched.

from keras.models import Sequential
from keras.layers import Dense
from keras_lr_multiplier import LRMultiplier

model = Sequential()
model.add(Dense(
    units=5,
    input_shape=(5,),
    activation='tanh',
    name='Dense',
))
model.add(Dense(
    units=2,
    activation='softmax',
    name='Output',
))
model.compile(
    optimizer=LRMultiplier('adam', {'Dense': 0.5, 'Output': 1.5}),
    loss='sparse_categorical_crossentropy',
)

Lambda

The multiplier can be a callable object. The input argument is the number of steps starting from 0.

from keras import backend as K
from keras_lr_multiplier import LRMultiplier

LRMultiplier('adam', {'Dense': lambda t: 2.0 - K.minimum(1.9, t * 1e-4)})

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for keras-lr-multiplier, version 0.8.0
Filename, size File type Python version Upload date Hashes
Filename, size keras-lr-multiplier-0.8.0.tar.gz (5.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page