keras-self-attention 0.0.9

Attention mechanism for processing sequence data that considers the context for each timestamp

Attention mechanism for processing sequence data that considers the context for each timestamp.

• 
• 
• 
• 

Install

pip install keras-self-attention

Usage

import keras
from keras_self_attention import Attention


model = keras.models.Sequential()
model.add(keras.layers.Embedding(input_dim=10000,
                                 output_dim=300,
                                 mask_zero=True))
model.add(keras.layers.Bidirectional(keras.layers.LSTM(units=128,
                                                       return_sequences=True)))
model.add(Attention())
model.add(keras.layers.Dense(units=5))
model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['categorical_accuracy'],
)
model.summary()