RAdam implemented in Keras
Project description
Keras RAdam
Unofficial implementation of RAdam.
Install
pip install keras-rectified-adam
Usage
import keras
import numpy as np
from keras_radam import RAdam
# Build toy model with RAdam optimizer
model = keras.models.Sequential()
model.add(keras.layers.Dense(input_shape=(17,), units=3))
model.compile(RAdam(), loss='mse')
# Generate toy data
x = np.random.standard_normal((4096 * 30, 17))
w = np.random.standard_normal((17, 3))
y = np.dot(x, w)
# Fit
model.fit(x, y, epochs=5)
Use Warmup
from keras_radam import RAdam
RAdam(total_step=10000, warmup_proportion=0.1, min_lr=1e-5)
Q & A
About Correctness
The optimizer produces similar losses and weights to the official optimizer after 500 steps.
Use tf.keras
or tf-2.0
Add TF_KERAS=1
to environment variables to use tensorflow.python.keras
.
Use theano
Backend
Add KERAS_BACKEND=theano
to environment variables to enable theano
backend.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for keras-rectified-adam-0.3.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 37e8ad66c6c5a7e33f781c0c2f3c6df85c8c89a5ea54c2af813b481422b37fc5 |
|
MD5 | 4a4f2b08b7bcf3d3b522120037be5fc4 |
|
BLAKE2b-256 | dc052bc41116bc49f5e0cc07f2c8eadc5b9aeb5c0b7e5982b9b76d8ac151130a |