Skip to main content

Rectified Nadam implemented in Keras

Project description

Keras RNAdam

use of nesterov accelerated geadient instead of momentum in rectified-adam

Install

pip install keras_rnadam

Usage

import keras
import numpy as np
from keras_rnadam import RNAdam

# Build toy model with RNAdam optimizer
model = keras.models.Sequential()
model.add(keras.layers.Dense(input_shape=(17,), units=3))
model.compile(RNAdam(), loss='mse')

# Generate toy data
x = np.random.standard_normal((4096 * 30, 17))
w = np.random.standard_normal((17, 3))
y = np.dot(x, w)

# Fit
model.fit(x, y, epochs=5)

Use Warmup

from keras_nradam import RNAdam
RNAdam(total_step=10000, warmup_proportion=0.1, min_lr=1e-5)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

keras_rnadam-0.2.0.tar.gz (3.5 kB view details)

Uploaded Source

File details

Details for the file keras_rnadam-0.2.0.tar.gz.

File metadata

  • Download URL: keras_rnadam-0.2.0.tar.gz
  • Upload date:
  • Size: 3.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.18.4 setuptools/39.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/2.7.15+

File hashes

Hashes for keras_rnadam-0.2.0.tar.gz
Algorithm Hash digest
SHA256 7436f9d1a7b9bc36251993f53f04db91f7bc1b6576bfa7cddd0dc2f05c433cd3
MD5 b509fb34eafefbe57cce3c40a2c0137a
BLAKE2b-256 0ad91f0fccf4fd7252a5cdf913f02273c4cbebe8b61f4251d438e7040011b8fd

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page