NLPMed-Engine

The NLP backend for NLPMed-Portal.

NLPMed-Engine is a robust and extensible natural language processing engine tailored for medical text. It supports a range of NLP tasks commonly used in clinical and biomedical applications.

⚠️ Important: This software is intended for research use only. It must not be used in real-world medical or clinical decision-making settings.

Installation

NLPMed-Engine requires Python 3.11 or 3.12.

Install from PyPI

For the core Python library:

pip install nlpmed-engine

To use the REST API:

pip install "nlpmed-engine[api]"

Optional GPU support can be installed with one of the following extras:

# Apple GPU (MPS)
pip install "nlpmed-engine[gpu-apple]"

# NVIDIA CUDA 11
pip install "nlpmed-engine[gpu-cuda11]"

# NVIDIA CUDA 12
pip install "nlpmed-engine[gpu-cuda12]"

The default sentence segmenter uses en_core_sci_lg_spacy38, a spaCy 3.8-compatible package of the SciSpaCy en_core_sci_lg 0.5.4 pipeline. It is distributed separately from PyPI and hosted on Hugging Face. Install it after installing NLPMed-Engine:

pip install "https://huggingface.co/omid-jf/en_core_sci_lg_spacy38/resolve/main/en_core_sci_lg_spacy38-0.5.4-py3-none-any.whl"

Install from the GitHub repository

git clone https://github.com/omid-jf/NLPMed-Engine.git
cd NLPMed-Engine

Install the project based on your environment:

# CPU
pip install -r requirements/cpu.txt

# Apple GPU (MPS)
pip install -r requirements/gpu_apple.txt

# NVIDIA CUDA 11
pip install -r requirements/gpu_cuda11.txt

# NVIDIA CUDA 12
pip install -r requirements/gpu_cuda12.txt

These source-install requirement files also install the REST API dependencies and the default SciSpaCy model. Package dependencies and optional extras are defined in pyproject.toml.

Usage

Run REST API

1. Copy the provided environment template and update the model paths and other settings for your environment:

   cp .env.example .env
   

   The template uses the following configuration structure:

   API_ML_MODEL_NAMES=VTE_MULTICLASS,BLEED_BINARY
   
   API_ML_VTE_MULTICLASS_DEVICE=cpu
   API_ML_VTE_MULTICLASS_MODEL_PATH=omid-jf/VTE-ModernBERT-DeID
   API_ML_VTE_MULTICLASS_TOKENIZER_PATH=omid-jf/VTE-ModernBERT-DeID
   API_ML_VTE_MULTICLASS_MAX_LENGTH=1024
   
   API_ML_BLEED_BINARY_DEVICE=cuda:0
   API_ML_BLEED_BINARY_MODEL_PATH=/path/to/model/model
   API_ML_BLEED_BINARY_TOKENIZER_PATH=/path/to/model/tokenizer
   API_ML_BLEED_BINARY_MAX_LENGTH=512
   
   API_HOST=127.0.0.1
   API_PORT=8000
   API_WORKERS=1
   

2. From a source checkout, run:

   python scripts/run_api.py
   

   When installed from PyPI, run:

   uvicorn nlpmed_engine.api.main:app --env-file .env --host 127.0.0.1 --port 8000
   

Run Single or Batch Pipelines

Instead of using the API, you can directly use the SinglePipeline or BatchPipeline classes in your Python code.

• Sample Jupyter notebooks are provided under the notebooks/ directory.
• Note: Since BatchPipeline uses parallel processing, the output order may differ from the input. Always use patient_id, note_id when merging results back with the input data.

Resources

• Demo: Visit our demo site
• VTE-ModernBERT-DeID: Hugging Face model fine-tuned from BioClinical ModernBERT on de-identified training text, with support for longer clinical-document context.
• VTE-BERT-DeID: Hugging Face model fine-tuned from Bio_ClinicalBERT on de-identified training text.
• Original VTE-BERT Model: The original model remains available under gated access on the Ang Li Lab Hugging Face account.
• Sentence-segmentation model: en_core_sci_lg_spacy38, a spaCy 3.8 compatibility package of SciSpaCy en_core_sci_lg 0.5.4.
• Publication: Development and Validation of VTE-BERT Natural Language Processing Model for Venous Thromboembolism (Open Access)

Model weights are hosted separately on Hugging Face and are not included in the NLPMed-Engine package. Review the model card and license for the specific model you use.

Documentation

See documentation for full API and module reference (generated with Sphinx).

Use and Safety

See the Use and Safety Notice for information about intended use, model licensing, data handling, and clinical safety. The source code is licensed under the AGPLv3 license included in this repository.

Citation

If you use NLPMed-Engine in your research or applications, please cite our paper:

@article{jafaridevelopment,
  title={Development and Validation of VTE-BERT Natural Language Processing Model for Venous Thromboembolism},
  author={Jafari, Omid and Ma, Shengling and Lam, Barbara D and Jiang, Jun Y and Zhou, Emily and Ranjan, Mrinal and Ryu, Justine and Bandyo, Raka and Maghsoudi, Arash and Peng, Bo and others},
  journal={Journal of Thrombosis and Haemostasis},
  publisher={Elsevier},
  year={2025},
  doi={10.1016/j.jtha.2025.07.021}
}