Skip to main content

How to encode sentences in a high-dimensional vector space, a.k.a., sentence embedding.

Project description

license doc

Sent2Vec - How to Compute Sentence Embedding Fast and Flexible

In the past, we mostly encode text data using, for example, one-hot, term frequency, or TF-IDF (normalized term frequency). There are many challenges to these techniques. In recent years, the latest advancements give us the opportunity to encode sentences or words in more meaningful formats. The word2vec technique and BERT language model are two important ones.

The sentence embedding is an important step of various NLP tasks such as sentiment analysis and summarization. A flexible sentence embedding library is needed to prototype fast and contextualized. The open-source sent2vec Python package gives you the opportunity to do so. You currently have access to the standard encoders. More advanced techniques will be added in the later releases. Hope you can use this library in your exciting NLP projects.

🔓 Install

The sent2vec is developed to help you prototype faster. That is why it has many dependencies on other libraries. The module requires the following libraries:

  • gensim
  • numpy
  • spacy
  • transformers
  • torch

Then, it can be installed using pip:

pip3 install sent2vec

📚 Documentation

class sent2vec.vectorizer.Vectorizer(pretrained_weights='distilbert-base-uncased', ensemble_method='average')

Parameters

  • pretrained_weights: str, default='distilbert-base-uncased' - How word embeddings are computed. => You can pass other BERT models into this parameter such as base multilingual model, i.e., distilbert-base-multilingual-cased. Basically, the vectorizer uses the BERT vectorizer with specified weights unless you pass a file path with extensions .txt, .gz or .bin to this parameter. In that case, the Gensim library will load the provided word2ved model (pretrained weights). For example, you can pass glove-wiki-gigaword-300.gz to load the Wiki vectors (when saved in the same folder you are running the code).
  • ensemble_method: str, default='average' - How word vectors are aggregated into sentece vectors.

Methods

run(sentences, remove_stop_words = ['not'], add_stop_words = [])
  • sentences: list, - List of sentences.
  • remove_stop_words: list, default=['not'] - When using sent2vec, list of words to remove from stop words when splitting sentences.
  • add_stop_words: list, default=[] - When using sent2vec, list of words to add to stop words when splitting sentences.

🧰 Usage

1. How to use BERT model?

If you want to use the BERT language model (more specifically, distilbert-base-uncased) to encode sentences for downstream applications, you must use the code below.

from sent2vec.vectorizer import Vectorizer

sentences = [
    "This is an awesome book to learn NLP.",
    "DistilBERT is an amazing NLP model.",
    "We can interchangeably use embedding, encoding, or vectorizing.",
]
vectorizer = Vectorizer()
vectorizer.run(sentences)
vectors = vectorizer.vectors

Now it's possible to compute distance among sentences by using their vectors. In the example, as expected, the distance between vectors[0] and vectors[1] is less than the distance between vectors[0] and vectors[2].

from scipy import spatial

dist_1 = spatial.distance.cosine(vectors[0], vectors[1])
dist_2 = spatial.distance.cosine(vectors[0], vectors[2])
print('dist_1: {0}, dist_2: {1}'.format(dist_1, dist_2))
assert dist_1 < dist_2
# dist_1: 0.043, dist_2: 0.192

Note: The default vectorizer for the BERT model is distilbert-base-uncased but it's possible to pass the argument pretrained_weights to chose another BERT model. For example, you can use the code below to load the base multilingual model.

vectorizer = Vectorizer(pretrained_weights='distilbert-base-multilingual-cased')

2. How to use Word2Vec model?

If you want to use a Word2Vec approach instead, you must pass a valid path to the model weights. Under the hood, the sentences will be split into lists of words using the sent2words method from the Splitter class. It is possible to customize the list of stop-words by adding or removing to/from the default list. Two additional arguments (both lists) must be passed when the vectorizer's method .run is called: remove_stop_words and add_stop_words.

Note: The default method to computes the sentence embeddings after extracting list of vectors is average of vectors corresponding to the remaining words.

from sent2vec.vectorizer import Vectorizer

sentences = [
    "Alice is in the Wonderland.",
    "Alice is not in the Wonderland.",
]

vectorizer = Vectorizer(pretrained_weights= PRETRAINED_VECTORS_PATH)
vectorizer.run(sentences, remove_stop_words=['not'], add_stop_words=[])
vectors = vectorizer.vectors

And, that's pretty much it!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sent2vec-0.3.0.tar.gz (8.6 kB view details)

Uploaded Source

Built Distribution

sent2vec-0.3.0-py3-none-any.whl (8.1 kB view details)

Uploaded Python 3

File details

Details for the file sent2vec-0.3.0.tar.gz.

File metadata

  • Download URL: sent2vec-0.3.0.tar.gz
  • Upload date:
  • Size: 8.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.0

File hashes

Hashes for sent2vec-0.3.0.tar.gz
Algorithm Hash digest
SHA256 4403620f89edd6fa12e935b965aeaf4edd2ad1116f1d00101d9b10974b916257
MD5 eabf087ef8922bedbbb9d67ddc2f255b
BLAKE2b-256 bb511e9512ef96b5a5a489d83fed2f07dec8a5d3ee09f2e6ef51a9ecf30f4bd0

See more details on using hashes here.

File details

Details for the file sent2vec-0.3.0-py3-none-any.whl.

File metadata

  • Download URL: sent2vec-0.3.0-py3-none-any.whl
  • Upload date:
  • Size: 8.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/4.0.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.0

File hashes

Hashes for sent2vec-0.3.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ddae860c95d2f514a88ffd30a899caae6be7cb7d3ae45dd30ea83301109a3297
MD5 7ec4e4d4a2b116dc1d196b8803a69d80
BLAKE2b-256 c0e3b7874aa9a3b5b92fd35ccb08269fd661e9d2a240b86e333d869c0f4b53d9

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page