Skip to main content

A spaCy pipeline object for negation.

Project description

negspacy: negation for spaCy

Build Status Build Status Built with spaCy pypi Version DOI Code style: black

spaCy pipeline object for negating concepts in text. Based on the NegEx algorithm.

NegEx - A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries Chapman, Bridewell, Hanbury, Cooper, Buchanan https://doi.org/10.1006/jbin.2001.1029

Installation and usage

Install the library.

pip install negspacy

Import library and spaCy.

import spacy
from negspacy.negation import Negex

Load spacy language model. Add negspacy pipeline object. Filtering on entity types is optional.

nlp = spacy.load("en_core_web_sm")
negex = Negex(nlp, ent_types=["PERSON","ORG"])
nlp.add_pipe(negex, last=True)

View negations.

doc = nlp("She does not like Steve Jobs but likes Apple products.")

for e in doc.ents:
	print(e.text, e._.negex)
Steve Jobs True
Apple False

Consider pairing with scispacy to find UMLS concepts in text and process negations.

NegEx Patterns

  • psuedo_negations - phrases that are false triggers, ambiguous negations, or double negatives
  • preceding_negations - negation phrases that precede an entity
  • following_negations - negation phrases that follow an entity
  • termination - phrases that cut a sentence in parts, for purposes of negation detection (.e.g., "but")

Termsets

Designate termset to use, en_clinical is used by default.

negex = Negex(nlp, language = "en_clinical")

  • en = phrases for general english language text
  • en_clinical DEFAULT = adds phrases specific to clinical domain to general english
  • en_clinical_sensitive = adds additional phrases to help rule out historical and possibly irrelevant entities

Additional Functionality

Use own patterns or view patterns in use

Use own patterns

nlp = spacy.load("en_core_web_sm")
negex = Negex(nlp, termination=["but", "however", "nevertheless", "except"])

View patterns in use

patterns_dict = negex.get_patterns

Negations in noun chunks

Depending on the Named Entity Recognition model you are using, you may have negations "chunked together" with nouns. For example when using scispacy:

nlp = spacy.load("en_core_sci_sm")
doc = nlp("There is no headache.")
for e in doc.ents:
    print(e.text)

# no headache

This would cause the Negex algorithm to miss the preceding negation. To account for this, you can add a chunk_prefix:

nlp = spacy.load("en_core_sci_sm")
negex = Negex(nlp, language = "en_clinical", chunk_prefix = ["no"])
nlp.add_pipe(negex)
doc = nlp("There is no headache.")
for e in doc.ents:
    print(e.text, e._.negex)

# no headache True

Contributing

contributing

Authors

  • Jeno Pizarro

License

license

API Documentation

Docs

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for negspacy, version 0.1.7
Filename, size File type Python version Upload date Hashes
Filename, size negspacy-0.1.7.tar.gz (7.8 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page