[中文|English]

## Install

pip install keras-adaptive-softmax


## Usage

Generally, AdaptiveEmbedding and AdaptiveSoftmax should be used together. AdaptiveEmbedding provides variable length embeddings, while AdaptiveSoftmax calculates the similarities between the outputs and the generated embeddings.

import keras

input_layer = keras.layers.Input(shape=(None,))
input_dim=30,
output_dim=32,
cutoffs=[5, 15, 25],
div_val=2,
return_embeddings=True,
return_projections=True,
)(input_layer)
dense_layer = keras.layers.Dense(
units=32,
activation='tanh',
)(embed_layer[0])
input_dim=32,
output_dim=30,
cutoffs=[5, 15, 25],
div_val=2,
bind_embeddings=True,
bind_projections=True,
)([dense_layer] + embed_layer[1:])
model = keras.models.Model(inputs=input_layer, outputs=softmax_layer)
model.summary()


cutoffs and div_val controls the length of embeddings for each token. Suppose we have 30 distinct tokens, in the above example:

• The lengths of the embeddings of the first 5 tokens are 32
• The lengths of the embeddings of the next 10 tokens are 16
• The lengths of the embeddings of the next 10 tokens are 8
• The lengths of the embeddings of the last 5 tokens are 4

## Project details

Uploaded source