Metadata-Version: 2.4
Name: nlpmed-engine
Version: 2.2.1
Summary: An extensible natural language processing engine for medical text.
Author: Omid Jafari
Maintainer: Omid Jafari
License-Expression: AGPL-3.0-or-later
Project-URL: Homepage, https://github.com/omid-jf/NLPMed-Engine
Project-URL: Documentation, https://omid-jf.github.io/NLPMed-Engine/
Project-URL: Repository, https://github.com/omid-jf/NLPMed-Engine.git
Project-URL: Issues, https://github.com/omid-jf/NLPMed-Engine/issues
Keywords: clinical-nlp,medical-nlp,natural-language-processing
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <3.13,>=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click==8.2.1
Requires-Dist: datasketch==1.6.5
Requires-Dist: ftfy==6.2.3
Requires-Dist: spacy==3.8.14
Requires-Dist: torch==2.12.0
Requires-Dist: transformers[accelerate]==5.10.1
Requires-Dist: typer==0.25.1
Provides-Extra: api
Requires-Dist: fastapi==0.115.0; extra == "api"
Requires-Dist: python-dotenv==1.0.1; extra == "api"
Requires-Dist: uvicorn==0.37.0; extra == "api"
Requires-Dist: pydantic<3,>=2; extra == "api"
Provides-Extra: gpu-apple
Requires-Dist: spacy[apple]==3.8.14; extra == "gpu-apple"
Provides-Extra: gpu-cuda11
Requires-Dist: spacy[cuda11x]==3.8.14; extra == "gpu-cuda11"
Provides-Extra: gpu-cuda12
Requires-Dist: spacy[cuda12x]==3.8.14; extra == "gpu-cuda12"
Provides-Extra: test
Requires-Dist: httpx==0.27.2; extra == "test"
Requires-Dist: mypy==1.11.2; extra == "test"
Requires-Dist: pytest==8.3.3; extra == "test"
Requires-Dist: pytest-benchmark==4.0.0; extra == "test"
Requires-Dist: pytest-cov==5.0.0; extra == "test"
Requires-Dist: pytest-profiling==1.7.0; extra == "test"
Requires-Dist: pytest-sugar==1.0.0; extra == "test"
Provides-Extra: docs
Requires-Dist: sphinx==8.0.2; extra == "docs"
Requires-Dist: sphinx-autobuild==2024.9.19; extra == "docs"
Dynamic: license-file

<!--
SPDX-FileCopyrightText: Copyright (C) 2025 Omid Jafari <omidjafari.com>
SPDX-License-Identifier: AGPL-3.0-or-later

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
-->

# NLPMed-Engine

**The NLP backend for NLPMed-Portal.**

NLPMed-Engine is a robust and extensible natural language processing engine tailored for medical text. It supports a range of NLP tasks commonly used in clinical and biomedical applications.

> ⚠️ **Important:** This software is intended for research use only. It must not be used in real-world medical or clinical decision-making settings.

![Static Badge](https://img.shields.io/badge/license-AGPLv3-blue)
[![CI](https://github.com/omid-jf/NLPMed-Engine/actions/workflows/ci.yml/badge.svg)](https://github.com/omid-jf/NLPMed-Engine/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/omid-jf/NLPMed-Engine/branch/main/graph/badge.svg)](https://codecov.io/gh/omid-jf/NLPMed-Engine)

---

## Installation

NLPMed-Engine requires Python 3.11 or 3.12.

### Install from PyPI

For the core Python library:

```bash
pip install nlpmed-engine
```

To use the REST API:

```bash
pip install "nlpmed-engine[api]"
```

Optional GPU support can be installed with one of the following extras:

```bash
# Apple GPU (MPS)
pip install "nlpmed-engine[gpu-apple]"

# NVIDIA CUDA 11
pip install "nlpmed-engine[gpu-cuda11]"

# NVIDIA CUDA 12
pip install "nlpmed-engine[gpu-cuda12]"
```

The default sentence segmenter uses `en_core_sci_lg_spacy38`, a spaCy 3.8-compatible package of the SciSpaCy `en_core_sci_lg` 0.5.4 pipeline. It is distributed separately from PyPI and hosted on Hugging Face. Install it after installing NLPMed-Engine:

```bash
pip install "https://huggingface.co/omid-jf/en_core_sci_lg_spacy38/resolve/main/en_core_sci_lg_spacy38-0.5.4-py3-none-any.whl"
```

### Install from the GitHub repository

```bash
git clone https://github.com/omid-jf/NLPMed-Engine.git
cd NLPMed-Engine
```

Install the project based on your environment:

```bash
# CPU
pip install -r requirements/cpu.txt

# Apple GPU (MPS)
pip install -r requirements/gpu_apple.txt

# NVIDIA CUDA 11
pip install -r requirements/gpu_cuda11.txt

# NVIDIA CUDA 12
pip install -r requirements/gpu_cuda12.txt
```

These source-install requirement files also install the REST API dependencies and the default SciSpaCy model. Package dependencies and optional extras are defined in `pyproject.toml`.

## Usage

### Run REST API

1. Copy the provided environment template and update the model paths and other settings for your environment:

   ```bash
   cp .env.example .env
   ```

   The template uses the following configuration structure:

   ```ini
   API_ML_MODEL_NAMES=VTE_MULTICLASS,BLEED_BINARY

   API_ML_VTE_MULTICLASS_DEVICE=cpu
   API_ML_VTE_MULTICLASS_MODEL_PATH=omid-jf/VTE-ModernBERT-DeID
   API_ML_VTE_MULTICLASS_TOKENIZER_PATH=omid-jf/VTE-ModernBERT-DeID
   API_ML_VTE_MULTICLASS_MAX_LENGTH=1024

   API_ML_BLEED_BINARY_DEVICE=cuda:0
   API_ML_BLEED_BINARY_MODEL_PATH=/path/to/model/model
   API_ML_BLEED_BINARY_TOKENIZER_PATH=/path/to/model/tokenizer
   API_ML_BLEED_BINARY_MAX_LENGTH=512

   API_HOST=127.0.0.1
   API_PORT=8000
   API_WORKERS=1
   ```

2. From a source checkout, run:

   ```bash
   python scripts/run_api.py
   ```

   When installed from PyPI, run:

   ```bash
   uvicorn nlpmed_engine.api.main:app --env-file .env --host 127.0.0.1 --port 8000
   ```

### Run Single or Batch Pipelines

Instead of using the API, you can directly use the `SinglePipeline` or `BatchPipeline` classes in your Python code.

- Sample Jupyter notebooks are provided under the `notebooks/` directory.
- **Note:** Since `BatchPipeline` uses parallel processing, the output order may differ from the input. Always use `patient_id, note_id` when merging results back with the input data.

## Resources

- **Demo**: [Visit our demo site](https://nlpmed.demo.angli-lab.com/)
- **VTE-ModernBERT-DeID**: [Hugging Face model](https://huggingface.co/omid-jf/VTE-ModernBERT-DeID) fine-tuned from BioClinical ModernBERT on de-identified training text, with support for longer clinical-document context.
- **VTE-BERT-DeID**: [Hugging Face model](https://huggingface.co/omid-jf/VTE-BERT-DeID) fine-tuned from Bio_ClinicalBERT on de-identified training text.
- **Original VTE-BERT Model**: The original model remains available under gated access on the [Ang Li Lab Hugging Face account](https://huggingface.co/ang-li-lab/VTE-BERT).
- **Sentence-segmentation model**: [en_core_sci_lg_spacy38](https://huggingface.co/omid-jf/en_core_sci_lg_spacy38), a spaCy 3.8 compatibility package of SciSpaCy `en_core_sci_lg` 0.5.4.
- **Publication**: Development and Validation of VTE-BERT Natural Language Processing Model for Venous Thromboembolism ([Open Access](https://www.jthjournal.org/article/S1538-7836(25)00484-2/fulltext))

Model weights are hosted separately on Hugging Face and are not included in the NLPMed-Engine package. Review the model card and license for the specific model you use.

## Documentation

See [documentation](https://omid-jf.github.io/NLPMed-Engine/) for full API and module reference (generated with Sphinx).

## Use and Safety

See the [Use and Safety Notice](./TERMS.md) for information about intended use, model licensing, data handling, and clinical safety. The source code is licensed under the AGPLv3 license included in this repository.

## Citation

If you use NLPMed-Engine in your research or applications, please cite our paper:

```bibtex
@article{jafaridevelopment,
  title={Development and Validation of VTE-BERT Natural Language Processing Model for Venous Thromboembolism},
  author={Jafari, Omid and Ma, Shengling and Lam, Barbara D and Jiang, Jun Y and Zhou, Emily and Ranjan, Mrinal and Ryu, Justine and Bandyo, Raka and Maghsoudi, Arash and Peng, Bo and others},
  journal={Journal of Thrombosis and Haemostasis},
  publisher={Elsevier},
  year={2025},
  doi={10.1016/j.jtha.2025.07.021}
}
```

---
