Skip to main content

Keyword extraction using transformer-based language models

Project description

WordWise

| |     / /___  _________/ / |     / (_)_______ 
| | /| / / __ \/ ___/ __  /| | /| / / / ___/ _ \
| |/ |/ / /_/ / /  / /_/ / | |/ |/ / (__  )  __/
|__/|__/\____/_/   \__,_/  |__/|__/_/____/\___/ 

WordWise is a minimal keyword extraction library that uses quality contextual embeddings generated by Sentence-BERT to extract keywords from blocks of text.

Installation

WordWise is available on PyPI.

pip install wordwise

To clone this repository, run

git clone https://github.com/jaketae/wordwise.git

Quickstart

At the core of WordWise is the Extractor class, which can be configured to generate keywords from some given text. The Extractor can be initialized and used out-of-the-box with minimal configuration as follows.

from wordwise import Extractor

extractor = Extractor()
keywords = extractor.generate(text)

For advanced users, the Extractor class, as well as the .generate() method, can be configured in a more granular fashion to induce specific behaviors. For instance, the underlying spaCy backend can be specified.

extractor = Extractor(spacy_model="en_core_web_lg")

By default, the Extractor will only generate the top 5 most relevant keywords. This behavior can be changed as follows.

# generate 10 keywords
keywords = extractor.generate(text, top_k=10)

Demo

Below is an example text adapted from the Wikipedia article on supervised learning.

text = """
         Supervised learning is the machine learning task of
         learning a function that maps an input to an output based
         on example input-output pairs.[1] It infers a function
         from labeled training data consisting of a set of
         training examples.[2] In supervised learning, each
         example is a pair consisting of an input object
         (typically a vector) and a desired output value (also
         called the supervisory signal). A supervised learning
         algorithm analyzes the training data and produces an
         inferred function, which can be used for mapping new
         examples. An optimal scenario will allow for the algorithm
         to correctly determine the class labels for unseen
         instances. This requires the learning algorithm to
         generalize from the training data to unseen situations
         in a 'reasonable' way (see inductive bias).
      """

The extractor selects the three most relevant keywords from the block of text.

>>> from wordwise import Extractor
>>> extractor = Extractor()
>>> keywords = extractor.generate(text, 3)
>>> print(keywords)
['algorithm', 'learning', 'supervised learning']

How it Works

Using spaCy, the Extractor object generates n-gram candidate noun phrases from the provided block of text. By default, it only considers uni-grams or bi-grams, since only rarely are keywords go beyond three words. Then, using a BERT model, it generates contextual embeddings for both the provided text and the n-gram keywords. Using cosine similarity as a distance function, it extracts the top_k candidate keywords that are most similar to the embedding of the inputted text.

For a more detailed write-up, please refer to my blog post here.

Credit

WordWise was largely inspired by KeyBERT, a library that similarly uses sentence embeddings for keyword extraction. WordWise also relies on NLP libraries, such as spaCy and HuggingFace transformers, without which its development would not have been possible.

License

Released under the MIT License.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

wordwise-0.0.5.tar.gz (5.0 kB view details)

Uploaded Source

Built Distribution

wordwise-0.0.5-py3-none-any.whl (6.2 kB view details)

Uploaded Python 3

File details

Details for the file wordwise-0.0.5.tar.gz.

File metadata

  • Download URL: wordwise-0.0.5.tar.gz
  • Upload date:
  • Size: 5.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.7.9

File hashes

Hashes for wordwise-0.0.5.tar.gz
Algorithm Hash digest
SHA256 d24970ba2aa7d70a490b91e1da083eea3700c6902473e5c23654313980f77f25
MD5 ba1e60ff180afcbb3f2f899921c8756e
BLAKE2b-256 ab84f2ba805f3f4505435327f9a632ec35a4c342bfb50b7a0a84311113e2cde1

See more details on using hashes here.

File details

Details for the file wordwise-0.0.5-py3-none-any.whl.

File metadata

  • Download URL: wordwise-0.0.5-py3-none-any.whl
  • Upload date:
  • Size: 6.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.0 CPython/3.7.9

File hashes

Hashes for wordwise-0.0.5-py3-none-any.whl
Algorithm Hash digest
SHA256 0d424b4b4e140c284186427494da21f79af533281ce0d2f513828821580e7ba8
MD5 8269f5256fa2644d114f067695459fde
BLAKE2b-256 72d3f0adcc85c012940ddc06427389563735c6fec69ef4b8cacc087d4486b9c8

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page