Transformer-XL implemented in Keras
Project description
Keras Transformer-XL
Unofficial implementation of Transformer-XL.
Install
pip install keras-transformer-xl
Usage
Load Pretrained Weights
Several configuration files can be found at the info directory.
import os
from keras_transformer_xl import load_trained_model_from_checkpoint
checkpoint_path = 'foo/bar/sota/enwiki8'
model = load_trained_model_from_checkpoint(
config_path=os.path.join(checkpoint_path, 'config.json'),
checkpoint_path=os.path.join(checkpoint_path, 'model.ckpt')
)
model.summary()
About IO
Suppose the number of transformer blocks is n
. The last n
inputs are used for inputs of memorization, and the last n
outputs represents new data to be memorized.
Use tensorflow.python.keras
Add TF_KERAS=1
to environment variables to use tensorflow.python.keras
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
keras-transformer-xl-0.3.1.tar.gz
(12.8 kB
view hashes)
Close
Hashes for keras-transformer-xl-0.3.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 124e7dc5a6ce8293b5df33855fdaf4d09b379d6ee8e84b84dd3ba15d15243da2 |
|
MD5 | bb01c7a4ac1ae8c6d5e44fbc1c4f6cbd |
|
BLAKE2b-256 | 37c814179685449c5e87074535bdf6489e7c9d1c493b7aa1ed4188e95e044caa |