Keras RAdam
Unofficial implementation of RAdam in Keras and TensorFlow.
Install
pip install keras-rectified-adam
External Link
Usage
import keras
import numpy as np
from keras_radam import RAdam
# Build toy model with RAdam optimizer
model = keras.models.Sequential()
model.add(keras.layers.Dense(input_shape=(17,), units=3))
model.compile(RAdam(), loss='mse')
# Generate toy data
x = np.random.standard_normal((4096 * 30, 17))
w = np.random.standard_normal((17, 3))
y = np.dot(x, w)
# Fit
model.fit(x, y, epochs=5)
TensorFlow without Keras
from keras_radam.training import RAdamOptimizer
RAdamOptimizer(learning_rate=1e-3)
Use Warmup
from keras_radam import RAdam
RAdam(total_steps=10000, warmup_proportion=0.1, min_lr=1e-5)
Q & A
About Correctness
The optimizer produces similar losses and weights to the official optimizer after 500 steps.
Use tf.keras or tf-2.0
Add TF_KERAS=1 to environment variables to use tensorflow.python.keras.
Use theano Backend
Add KERAS_BACKEND=theano to environment variables to enable theano backend.
1 maintainer
Release files for keras-radam 0.15.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| keras-radam-0.15.0.tar.gz | 11.7 kB | Details |
Release files / keras-radam-0.15.0.tar.gz
| Download URL | keras-radam-0.15.0.tar.gz |
|---|---|
| Size | 11.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4a0bbcaa3bcd20e0a7a05192a15fcf490fe26f4cfc677b65a056155b866d58e1
|
|
BLAKE2b-256 checksum How to use checksums |
468db83ccaa94253fbc920b21981f038393041d92236bb541751b98a66a2ac1d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3
|
