Treat the initial state(s) of TensorFlow Keras recurrent neural network (RNN) layers as a parameter or parameters to be learned during training, as recommended in, e.g., [1].
Ordinary RNNs use an all-zero initial state by default. Why not let the neural network learn a smarter initial state?
The trainable-initial-state-rnn package provides a class TrainableInitialStateRNN that can wrap any tf.keras RNN (or bidirectional RNN) and manage new initial state variables in addition to the RNN’s weights.
Typical usage looks as follows.
import tensorflow as tf
from trainable_initial_state_rnn import TrainableInitialStateRNN
base_rnn = tf.keras.layers.LSTM(256)
rnn = TrainableInitialStateRNN(base_rnn) # Treats initial state as a variable!
model = tf.keras.Model(...) # Use rnn like any other tf.keras layer in your model
model.compile(...)
history = model.fit(...)Documentation is available at Read the Docs.
Installation
The trainable_initial_state_rnn package can be installed using the pip utility directly from the package’s GitHub page:
pip install git+https://github.com/artemmavrin/trainable-initial-state-rnn.gitAlternatively, install a recent release from the Python Package Index (PyPI):
pip install trainable-initial-state-rnnNote. To install the project for development (e.g., to make changes to the
source code), clone the project repository from GitHub and run make dev:
git clone https://github.com/artemmavrin/trainable-initial-state-rnn.git
cd trainable-initial-state-rnn
# Optional but recommended: create and activate a new Python virtual environment
make devThis will additionally install the requirements needed to run tests, check code coverage, and produce documentation.
References
1 maintainer
Metadata
1 maintainer
Release files for trainable-initial-state-rnn 0.0.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| trainable-initial-state-rnn-0.0.3.tar.gz | 6.9 kB | Without provenance | Details for trainable-initial-state-rnn-0.0.3.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| trainable_initial_state_rnn-0.0.3-py3-none-any.whl | Python 3 | none | any | Details for trainable_initial_state_rnn-0.0.3-py3-none-any.whl |
Total release size: 16.9 kB
Release files / trainable-initial-state-rnn-0.0.3.tar.gz
| Download URL | trainable-initial-state-rnn-0.0.3.tar.gz |
|---|---|
| Size | 6.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
eed8dd6c1ee98739910580ef1285d298e1f92109cd8688d3fb99e6fbcef0c735
|
|
BLAKE2b-256 checksum How to use checksums |
acd5041e82e263e87273d00b56ada6d83d4c3bf005420294f966e80c83865e18
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/47.1.1 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.8.3
|
Release files / trainable_initial_state_rnn-0.0.3-py3-none-any.whl
| Download URL | trainable_initial_state_rnn-0.0.3-py3-none-any.whl |
|---|---|
| Size | 10.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
be3d92098f7cf5b05ee2088d0897fe35f591217407393a842d5b84a07ef841b2
|
|
BLAKE2b-256 checksum How to use checksums |
b6b79dcbab790394844366ce572d3d74fa2eaf37085a107e9890b76b3b232f81
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/47.1.1 requests-toolbelt/0.9.1 tqdm/4.46.1 CPython/3.8.3
|
Security for trainable-initial-state-rnn 0.0.3
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
