Metadata-Version: 2.4
Name: clinparser
Version: 0.1.0
Summary: Structure-aware PDF parsing for clinical documents. No LLM, fully offline.
Author: Nandhakumar Saravanan
License: MIT
Project-URL: Homepage, https://github.com/Nandha-kumar-S/clinparser
Project-URL: Repository, https://github.com/Nandha-kumar-S/clinparser
Project-URL: Issues, https://github.com/Nandha-kumar-S/clinparser/issues
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Healthcare Industry
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Text Processing
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: docling
Requires-Dist: tqdm
Requires-Dist: pandas
Requires-Dist: markdown
Requires-Dist: pymupdf
Requires-Dist: pillow
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

# ClinParser

**Structure-aware PDF parsing for clinical documents. No LLM, fully offline.**

[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![Python 3.9+](https://img.shields.io/badge/python-3.9%2B-blue.svg)](pyproject.toml)

Clinical protocol PDFs are dense with information that's essential to almost every downstream process built around a trial — study design, eligibility criteria, visit schedules, statistical plans, safety definitions. As AI reshapes the clinical trials domain, more of that downstream work is becoming automatable, and nearly all of it starts by reading the protocol:

- **Generating other required documents that must stay consistent with it** — the SAP (Statistical Analysis Plan), IB (Investigator's Brochure), informed consent forms, and CRF (Case Report Form) design.
- **Operational and compliance checks during the trial** — protocol deviation detection, data quality checks against the protocol's own specifications, eligibility-criteria extraction for patient screening, and risk-based monitoring.
- **Standardizing the protocol into standard formats for reuse** — CDISC's USDM is the industry's emerging standard here, enabling cross-trial analytics and portfolio-level review.

Every one of those tasks starts from the same bottleneck: turning an information-rich protocol PDF into a machine-readable format without losing anything that matters — section numbering and nesting, which tables and images belong to which section, cross-references between sections, and the overall organization the authors intended.

ClinParser is that starting point. It parses a clinical protocol PDF into a structure-aware JSON tree — sections nested exactly as the protocol numbers them, each with its own text, tables, and images attached — ready to hand to an LLM for any of the tasks above, or to query, chunk, and feed into a RAG pipeline directly.

```json
{
  "number": "1.1",
  "title": "Background",
  "subsections": [],
  "text": "Alisporivir (also known as DEB025...) is a cyclophilin (Cyp) inhibitor...",
  "tables": [],
  "images": []
}
```

## Why not just use docling / Unstructured / LlamaParse directly?

Those tools infer heading *level* from layout — font size, boldness, indentation. That's brittle across the wildly inconsistent templates different sponsors and CROs use for protocols, and it gives you a flat stream of typed elements (paragraph, table, image + bounding box), leaving "which section does this belong to" as a join you have to do yourself.

ClinParser does two things differently:

- **Numbering-anchored hierarchy, not layout heuristics.** Protocol numbering is maintained rigorously by authors for regulatory reasons — it's a more reliable, fully deterministic signal than re-inferring structure from visual styling.
- **Per-section bundling.** Every node in the output already has its own `text`, `tables`, and `images` attached — no post-hoc join required.

And because there's no LLM anywhere in the pipeline, it's fully reproducible and runs entirely offline — relevant when the documents in question contain PHI (Protected Health Information) or sponsor IP (intellectual property) that can't leave a secure environment.

## Install

```bash
pip install clinparser
# or, for local development:
git clone https://github.com/Nandha-kumar-S/clinparser && cd clinparser
pip install -e ".[dev]"
```

## Usage

```bash
clinparser path/to/protocol.pdf
```

```python
from clinparser import PDFParser

parser = PDFParser()
result = parser.parse("path/to/protocol.pdf")
# {'toc': [...], 'non_toc': [...]}
```

By default, `parse()` writes everything to `output/<timestamp>/`: `markdown.md` and `toc.json`/`document.json` (debug artifacts), plus each image and table as its own file (`images/<section>/`, `tables/<section>/`). All three are independent and default to on:

```python
result = parser.parse(
    "path/to/protocol.pdf",
    save_intermediate_files=True,  # markdown.md, toc.json, document.json
    save_images=True,              # images/<section>/image_NNN.png
    save_table_csv=True,           # tables/<section>/table_NNN.csv
)
```

`examples/input/protocol_001.pdf` plus its output in `examples/output/` is a full worked example.

## Output shape

Every node — in `toc` (numbered sections) or `non_toc` (front/back matter that doesn't belong to a numbered section) — has the same shape:

```json
{
  "number": "3.1",
  "title": "Study design",
  "subsections": [ ... ],
  "text": "...",
  "tables": [{"columns": [...], "data": [[...]], "path": "tables/Study_design/table_001.csv"}],
  "images": [{"page": 12, "bbox": [...], "path": "images/Study_design/image_001.png"}]
}
```

## How it works

1. **PDF → Markdown** via `docling`.
2. **TOC extraction** — parses each heading's numeric prefix and builds the nesting tree from it directly.
3. **Content merge** — slices each section's body text out of the Markdown.
4. **Table extraction** — converts Markdown pipe-tables to `{columns, data}` JSON (and optionally CSV).
5. **Image extraction** — pulls embedded images via PyMuPDF and assigns each to its containing section by page/position.

## Contributing

Issues and PRs welcome. No LLM/API keys required to run or test anything in this repo.

## Dependencies & licensing

ClinParser is MIT licensed, and its dependencies are all permissively licensed (MIT/BSD/HPND) — **except [PyMuPDF](https://pypi.org/project/pymupdf/)**, used for image extraction and coordinate lookups. PyMuPDF is dual-licensed under **AGPL-3.0** (free to use, including commercially, but distributing software built on it — or running it as a network service — requires releasing the complete source of that combined application under AGPL-compatible terms) or a **commercial license from [Artifex](https://artifex.com/licensing)**. If you're building a proprietary or SaaS product on top of ClinParser, this applies to you. See [NOTICE](NOTICE) for the full dependency list.

## License

[MIT](LICENSE)
