Learning rate multiplier
Project description
Keras LR Multiplier
Learning rate multiplier wrapper for optimizers.
Install
pip install keras-lr-multiplier
Usage
Basic
LRMultiplier
is a wrapper for optimizers to assign different learning rates to specific layers (or weights). The first argument is the original optimizer which could be either an identifier (e.g. 'Adam'
) or an initialized object (e.g. Adam(lr=1e-2)
). The second argument is a dict that maps prefixes to learning rate multipliers. The multiplier for a weight is the value mapped from the longest matched prefix in the given dict, and the default multiplier 1.0
will be used if there is no prefix matched.
from keras.models import Sequential from keras.layers import Dense from keras_lr_multiplier import LRMultiplier model = Sequential() model.add(Dense( units=5, input_shape=(5,), activation='tanh', name='Dense', )) model.add(Dense( units=2, activation='softmax', name='Output', )) model.compile( optimizer=LRMultiplier('adam', {'Dense': 0.5, 'Output': 1.5}), loss='sparse_categorical_crossentropy', )
Lambda
The multiplier can be a callable object. The input argument is the number of steps starting from 0.
from keras import backend as K from keras_lr_multiplier import LRMultiplier LRMultiplier('adam', {'Dense': lambda t: 2.0 - K.minimum(1.9, t * 1e-4)})
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size | File type | Python version | Upload date | Hashes |
---|---|---|---|---|
Filename, size keras-lr-multiplier-0.8.0.tar.gz (5.7 kB) | File type Source | Python version None | Upload date | Hashes View |
Hashes for keras-lr-multiplier-0.8.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d64df32ff92855ef1cf92c29f87ccb01ea8a8b915e7a733c601454b417b0e063 |
|
MD5 | 789302c62f4c518a5b51fc0418017474 |
|
BLAKE2-256 | 7d780eed4862a7274fb491b50881dd2f0dac996ff5774dc4a30c4b628fb78b25 |