Transformer-XL implemented in Keras
Project description
Keras Transformer-XL
Unofficial implementation of Transformer-XL.
Install
pip install keras-transformer-xl
Usage
Load Pretrained Weights
Several configuration files can be found at the info directory.
import os
from keras_transformer_xl import load_trained_model_from_checkpoint
checkpoint_path = 'foo/bar/sota/enwiki8'
model = load_trained_model_from_checkpoint(
config_path=os.path.join(checkpoint_path, 'config.json'),
checkpoint_path=os.path.join(checkpoint_path, 'model.ckpt')
)
model.summary()
About IO
Suppose the number of transformer blocks is n
. The last n
inputs are used for inputs of memorization, and the last n
outputs represents new data to be memorized.
Use tensorflow.python.keras
Add TF_KERAS=1
to environment variables to use tensorflow.python.keras
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
keras-transformer-xl-0.3.0.tar.gz
(12.8 kB
view hashes)
Close
Hashes for keras-transformer-xl-0.3.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | f19340b140416213beef30858ea654e8488db4f6791d4401c6645cac4b51898a |
|
MD5 | adcd41e86ce5b609a33a6a0646b3d7e4 |
|
BLAKE2b-256 | b41831781fc78c8ed05ab588b6a6e8d477ebc54ba271d14dbf037f90d21f35aa |