unv-scispacy-abbr 0.1.1

Standalone abbreviation detection component extracted from scispacy.

unv_scispacy_abbr

A standalone abbreviation detection component extracted from the scispacy project. This package provides robust abbreviation detection for scientific and biomedical text using the Schwartz & Hearst algorithm.

Features

• Standalone: No scispacy dependencies - works with any spaCy pipeline
• Compatible API: Drop-in replacement for scispacy.abbreviation
• Robust Detection: Handles various abbreviation patterns including reverse definitions
• Scientific Focus: Optimized for scientific and biomedical text
• Lightweight: Minimal dependencies (only requires spaCy)

Installation

pip install unv-scispacy-abbr

Quick Start

As a spaCy Pipeline Component

import spacy
from unv_scispacy_abbr import AbbreviationDetector

# Create a spaCy pipeline (any language model works)
nlp = spacy.load("en_core_web_sm")  # or spacy.blank("en")

# Add the abbreviation detector to the pipeline
nlp.add_pipe("abbreviation_detector")

# Process text
doc = nlp("Spinal and bulbar muscular atrophy (SBMA) is a motor neuron disease.")

# Access detected abbreviations
for abbrev in doc._.abbreviations:
    print(f"Abbreviation: {abbrev}")
    print(f"Long form: {abbrev._.long_form}")

Standalone Functions

from unv_scispacy_abbr import find_abbreviation, filter_matches
import spacy

nlp = spacy.blank("en")
doc = nlp("machine learning (ML) and artificial intelligence (AI)")

# Manual abbreviation detection
long_form = doc[0:2]  # "machine learning"
short_form = doc[3:4]  # "ML" 
short, long = find_abbreviation(long_form, short_form)
if long:
    print(f"Found: {short} -> {long}")

Supported Patterns

The detector recognizes various abbreviation patterns:

• Standard: Long Form (SF) → "Long Form" abbreviated as "SF"
• Reverse: SF (Long Form) → "SF" is the abbreviation of "Long Form"
• Multiple words: Multiple Word Form (MWF)
• With numbers/hyphens: Long-Form 2.0 (LF-2)

Examples

import spacy
from unv_scispacy_abbr import AbbreviationDetector

nlp = spacy.blank("en")
nlp.add_pipe("abbreviation_detector")

# Example 1: Biomedical text
text1 = """
The androgen receptor (AR) plays a crucial role in spinal and bulbar 
muscular atrophy (SBMA). AR mutations cause SBMA in patients.
"""
doc1 = nlp(text1)
for abbrev in doc1._.abbreviations:
    print(f"{abbrev} = {abbrev._.long_form}")
# Output: 
# AR = androgen receptor
# SBMA = spinal and bulbar muscular atrophy  
# AR = androgen receptor
# SBMA = spinal and bulbar muscular atrophy

# Example 2: Reverse pattern
text2 = "We used BERT (Bidirectional Encoder Representations from Transformers) for NLP."
doc2 = nlp(text2)
for abbrev in doc2._.abbreviations:
    print(f"{abbrev} = {abbrev._.long_form}")
# Output: BERT = Bidirectional Encoder Representations from Transformers

Migration from scispacy

If you're using scispacy.abbreviation, you can easily migrate:

# Before (scispacy)
from scispacy.abbreviation import AbbreviationDetector

# After (this package)  
from unv_scispacy_abbr import AbbreviationDetector

The API is fully compatible, so no other code changes are needed.

Algorithm

This package implements the abbreviation detection algorithm described in:

Schwartz, A. S., & Hearst, M. A. (2003). A simple algorithm for identifying abbreviation definitions in biomedical text. Pacific symposium on biocomputing, 451-462.

Requirements

• Python ≥ 3.8
• spaCy ≥ 3.3, < 4.0

Development

# Clone the repository
git clone <repository-url>
cd unv_scispacy_abbr

# Install in development mode
pip install -e .

# Run tests
pytest tests/

License

Apache License 2.0

Credits

This package is extracted and adapted from the scispacy project by AllenAI. The original implementation was created by the scispacy team and contributors.