embedding-as-service: one-stop solution to encode sentence to vectors using various embedding methods
Project description
embedding-as-service
One-Stop Solution to encode sentence to fixed length vectors from various embedding techniques
• Inspired from bert-as-service
What is it • Installation • Getting Started • Supported Embeddings • API •
What is it :question:
Encoding/Embedding is a upstream task of encoding any inputs in the form of text, image, audio, video, transactional data to fixed length vector. Embeddings are quite popular in the field of NLP, there has been various Embeddings models being proposed in recent years by researchers, some of the famous one are bert, xlnet, word2vec etc. The goal of this repo is to build one stop solution for all embeddings techniques available, here we are starting with popular text embeddings for now and later on we aim to add as much technique for image, audio, video input also.
Finally, embedding-as-service
help you to encode any given text to fixed length vector from supported embeddings and models.
:floppy_disk: Installation
Install the embedding-as-servive via pip
.
pip install embedding-as-service
Note that the code MUST be running on Python >= 3.6 with Tensorflow >= 1.10 (one-point-ten). Again, this module does not support Python 2!
:zap: Getting Started
1. Intialise encoder using supported embedding and models from here
>>> from embedding_as_service.text.encode import Encoder
>>> en = Encoder(embedding='xlnet', model='xlnet_base_cased', download=True)
2. Get sentences tokens embedding
>>> vecs = en.encode(texts=['hello aman', 'how are you?'])
>>> vecs
array([[[ 1.7049843 , 0. , 1.3486509 , ..., -1.3647075 ,
0.6958289 , 1.8013777 ], ... [ 0.4913215 , 0.60877025, 0.73050433, ..., -0.64490885, 0.8525057 , 0.3080206 ]]], dtype=float32)
>>> vecs.shape
(2, 128, 768) # batch x max_sequence_length x embedding_size
3. Using pooling strategy, click here for more.
>>> vecs = en.encode(texts=['hello aman', 'how are you?'], pooling='REDUCE_MEAN')
>>> vecs
array([[-0.33547154, 0.34566957, 1.1954105 , ..., 0.33702594,
1.0317835 , -0.785943 ], [-0.3439088 , 0.36881036, 1.0612687 , ..., 0.28851607, 1.1107115 , -0.6253736 ]], dtype=float32)
>>> vecs.shape
(2, 768) # batch x embedding_size
4. Use custom max_seq_length
, default is 128
>>> vecs = en.encode(texts=['hello aman', 'how are you?'], max_seq_length=256)
>>> vecs
array([[ 0.48388457, -0.01327741, -0.76577514, ..., -0.54265064,
-0.5564591 , 0.6454179 ], [ 0.53209245, 0.00526248, -0.71091074, ..., -0.5171917 , -0.40458363, 0.6779779 ]], dtype=float32)
>>> vecs.shape
(2, 256, 768) # batch x max_sequence_length x embedding_size
5. Show embedding Tokens
>>> en.tokenize(texts=['hello aman', 'how are you?'])
[['_hello', '_aman'], ['_how', '_are', '_you', '?']]
6. Using your own tokenizer
>>> texts = ['hello aman!', 'how are you']
# a naive whitespace tokenizer
>>> tokens = [s.split() for s in texts]
>>> vecs = en.encode(tokens, is_tokenized=True)
:clipboard: API
1. class embedding_as_service.text.encoder.Encoder
Argument | Type | Default | Description |
---|---|---|---|
embedding |
str | Required | embedding method to be used, check Embedding column here |
model |
str | Required | Model to be used for mentioned embedding, check Model column here |
download |
bool | False |
Download model if model does not exists |
2. def embedding_as_service.text.encoder.Encoder.encode
Argument | Type | Default | Description |
---|---|---|---|
Texts |
List[str] or List[List[str]] | Required | List of sentences or list of list of sentence tokens in case of is_tokenized=True |
pooling_strategy |
str | (Optional) | Pooling methods to apply, here is available methods |
max_seq_length |
int | 128 |
Maximum Sequence Length, default is 128 |
is_tokenized |
bool | False |
set as True in case of tokens are passed for encoding |
max_batch_size |
int | 128 |
maximum number of sequences handled by encoder, larger batch will be partitioned into small batches. |
2. def embedding_as_service.text.encoder.Encoder.tokenize
Argument | Type | Default | Description |
---|---|---|---|
Texts |
List[str] | Required | List of sentences |
:white_check_mark: Supported Embeddings and Models
Here are the list of supported embeddings and their respective models.
Embedding | Model | Embedding dimensions | Paper | |
---|---|---|---|---|
:one: | xlnet |
xlnet_large_cased |
1024 | Read Paper :bookmark: |
xlnet_base_cased |
768 | |||
:two: | bert |
bert_base_uncased |
768 | Read Paper :bookmark: |
bert_base_cased |
768 | |||
bert_multi_cased |
768 | |||
bert_large_uncased |
1024 | |||
bert_large_cased |
1024 | |||
:three: | elmo |
elmo_bi_lm |
512 | Read Paper :bookmark: |
:four: | ulmfit |
ulmfit_forward |
300 | Read Paper :bookmark: |
ulmfit_backward |
300 | |||
:five: | use |
use_dan |
512 | Read Paper :bookmark: |
use_transformer_large |
512 | |||
use_transformer_lite |
512 | |||
:six: | word2vec |
google_news_300 |
300 | Read Paper :bookmark: |
:seven: | fasttext |
wiki_news_300 |
300 | Read Paper :bookmark: |
wiki_news_300_sub |
300 | |||
common_crawl_300 |
300 | |||
common_crawl_300_sub |
300 | |||
:eight: | glove |
twitter_200 |
200 | Read Paper :bookmark: |
twitter_100 |
100 | |||
twitter_50 |
50 | |||
twitter_25 |
25 | |||
wiki_300 |
300 | |||
wiki_200 |
200 | |||
wiki_100 |
100 | |||
wiki_50 |
50 | |||
crawl_42B_300 |
300 | |||
crawl_840B_300 |
300 |
:heavy_plus_sign: Pooling Strategies
Here is a table summarizes all supported pooling strategies
Strategy | Description |
---|---|
None |
no pooling at all, useful when you want to use word embedding instead of sentence embedding. This will results in a [max_seq_len, embedding_size] encode matrix for a sequence. |
REDUCE_MEAN |
take the average of all token embeddings |
REDUCE_MIN |
take the minumun of all token embeddings |
REDUCE_MAX |
take the maximum of all token embeddings |
REDUCE_MEAN_MAX |
do REDUCE_MEAN and REDUCE_MAX separately and then concat them together |
FIRST_TOKEN |
get the token embedding of first token of a sentence |
LAST_TOKEN |
get the token embedding of last token of a sentence |
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for embedding_as_service-0.7.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3aab9402c1aa8a678154d8abd0ac4c60f0d23c0878111f57af5635dadbaf9980 |
|
MD5 | dfd8d632bb16b8c7b321d16cb4d91079 |
|
BLAKE2b-256 | 938e1071c82b4258c18603a180d85b2b4ab4b989a70deae68dd3efbba78ee9eb |
Hashes for embedding_as_service-0.7.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 26a4a17dde4c9bbc57f329aed088aa0a24749f814b83b0a2f913a5c9498bb63c |
|
MD5 | 424cea8632d345e36829ada4529d0f42 |
|
BLAKE2b-256 | d6ba959f4168e98e1640df40d28bf0cdfe9c72fc585cbb189d3091a06ea88c9a |