Rectified Nadam implemented in Keras
Project description
Keras RNAdam
use of nesterov accelerated geadient instead of momentum in rectified-adam
Install
pip install keras_rnadam
Usage
import keras
import numpy as np
from keras_rnadam import RNAdam
# Build toy model with RNAdam optimizer
model = keras.models.Sequential()
model.add(keras.layers.Dense(input_shape=(17,), units=3))
model.compile(RNAdam(), loss='mse')
# Generate toy data
x = np.random.standard_normal((4096 * 30, 17))
w = np.random.standard_normal((17, 3))
y = np.dot(x, w)
# Fit
model.fit(x, y, epochs=5)
Use Warmup
from keras_nradam import RNAdam
RNAdam(total_step=10000, warmup_proportion=0.1, min_lr=1e-5)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
keras_rnadam-0.2.0.tar.gz
(3.5 kB
view details)
File details
Details for the file keras_rnadam-0.2.0.tar.gz
.
File metadata
- Download URL: keras_rnadam-0.2.0.tar.gz
- Upload date:
- Size: 3.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/2.7.15+
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 |
7436f9d1a7b9bc36251993f53f04db91f7bc1b6576bfa7cddd0dc2f05c433cd3
|
|
MD5 |
b509fb34eafefbe57cce3c40a2c0137a
|
|
BLAKE2b-256 |
0ad91f0fccf4fd7252a5cdf913f02273c4cbebe8b61f4251d438e7040011b8fd
|