Skip to main content

State of the art Lexical Substitution in Context

Project description

Setup

pip3 install lexsub

Release: https://pypi.org/project/lexsub/   
Version: 1.0.5 

Background

The Lexical Substitution task involves selecting and ranking lexical paraphrases for a target word in a given sentential context. In the task, annotators and systems find an alternative substitute word or phrase for a target word in context. The task involves both finding the synonyms and disambiguating the context. To give an example:

Context: The wine was too strong to drink.
Target Word: strong

Predicted Candidates (Ranked): 
powerful 
potent 
warm
hot 
solid 
hard 

Powerful and potent are much better replacements as indicated by the score next to them as well. Whereas, all are viable replacement candidates given the context from a language modeling perspective.
Now notice two important things that is interesting:

  • Not all synonyms fit in the context.
    (direct lexical substitutions aka synonyms won’t always work)
  • Not all words that fit in the context preserve the meaning of the sentence. (LM score doesn’t always correlate)

Datasets:

A. Lexical Substitution:

B. Word Sense Disambiguation:

References:

  1. SOTA-BERT
  2. Pre-BERT-SOTA,Melamud
  3. PIC- Katrin
  4. SemBERT, AAAI 2020
  5. LIBERT
  6. Morgifier LSTM, ICLR 2020

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lexsub-1.0.6.tar.gz (6.8 kB view details)

Uploaded Source

Built Distribution

lexsub-1.0.6-py3-none-any.whl (10.2 kB view details)

Uploaded Python 3

File details

Details for the file lexsub-1.0.6.tar.gz.

File metadata

  • Download URL: lexsub-1.0.6.tar.gz
  • Upload date:
  • Size: 6.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.37.0 CPython/3.7.4

File hashes

Hashes for lexsub-1.0.6.tar.gz
Algorithm Hash digest
SHA256 58c185b4182e2e1c583ccb55a1c1a3bbed19b90076c598c482cd67d10c42278d
MD5 45daa561ce0d139605a5faba584722d1
BLAKE2b-256 603fbbb40768c2d1565429e02559833c97fe8b8f62680d4701f9b309d4747019

See more details on using hashes here.

File details

Details for the file lexsub-1.0.6-py3-none-any.whl.

File metadata

  • Download URL: lexsub-1.0.6-py3-none-any.whl
  • Upload date:
  • Size: 10.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.37.0 CPython/3.7.4

File hashes

Hashes for lexsub-1.0.6-py3-none-any.whl
Algorithm Hash digest
SHA256 321dd7cf3719d29e24dffab876c56237902bfb64129812e0c9770848c830c7e1
MD5 18ce767e54f0d2544bb439f1aa5dc371
BLAKE2b-256 c312bc1532c6e9265d4721f924ecc48a4cdea08a6947a24c8cfacac7908b4248

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page