RAdam implemented in Keras & TensorFlow
Project description
Keras RAdam
Unofficial implementation of RAdam in Keras.
Install
pip install keras-rectified-adam
External Link
Usage
from tensorflow import keras
import numpy as np
from keras_radam import RAdam
# Build toy model with RAdam optimizer
model = keras.models.Sequential()
model.add(keras.layers.Dense(input_shape=(17,), units=3))
model.compile(RAdam(), loss='mse')
# Generate toy data
x = np.random.standard_normal((4096 * 30, 17))
w = np.random.standard_normal((17, 3))
y = np.dot(x, w)
# Fit
model.fit(x, y, epochs=5)
Use Warmup
from keras_radam import RAdam
RAdam(total_steps=10000, warmup_proportion=0.1, min_lr=1e-5)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
File details
Details for the file keras-rectified-adam-0.20.0.tar.gz
.
File metadata
- Download URL: keras-rectified-adam-0.20.0.tar.gz
- Upload date:
- Size: 7.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.6.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
649102f602a79d5f971d227e7defb432302e0b19b3acf48002a6304a5f376a3a
|
|
MD5 |
16a3573e7965c2930069f00fc94eece7
|
|
BLAKE2b-256 |
64ee5a1eaf6e6fc907ef871bc6241d97ecb55936b112c0cbb93e5df1268bca64
|