Metadata-Version: 2.4
Name: saraldocling
Version: 1.0.1
Summary: Fast PDF text and image/table extraction using PyMuPDF + YOLOv8 DocLayNet ONNX
Author: SARAL
License-Expression: AGPL-3.0-only
Project-URL: Homepage, https://github.com/DemocratiseResearch/SARALDocling
Project-URL: Issues, https://github.com/DemocratiseResearch/SARALDocling/issues
Keywords: pdf,parsing,yolo,document-layout,onnx,table-extraction
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: General
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: pymupdf>=1.24.0
Requires-Dist: numpy<3.0,>=1.24.0
Requires-Dist: Pillow>=10.0.0
Requires-Dist: onnxruntime>=1.17.0
Provides-Extra: gpu
Requires-Dist: onnxruntime-gpu>=1.17.0; extra == "gpu"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: mypy; extra == "dev"

# SARAL Docling

Fast PDF text and image/table extraction.  
Powered by **PyMuPDF** + **YOLOv8 DocLayNet** (bundled) — no model file needed.

```bash
pip install saraldocling
saraldocling paper.pdf
```

> **License:** This package bundles `yolov8n-doclaynet.onnx` which is released
> under **AGPL-3.0**. The entire `saraldocling` package is therefore also
> distributed under **AGPL-3.0**.  
> Full text: https://www.gnu.org/licenses/agpl-3.0.html

> **No OCR.** Only PDFs with a native text layer are supported.  
> For scanned PDFs, pre-process with `ocrmypdf` first (see [Limitations](#limitations)).

---

## Table of contents

- [How it works](#how-it-works)
- [Installation](#installation)
- [CLI usage](#cli-usage)
- [Python API](#python-api)
- [Output structure](#output-structure)
- [FastAPI / server deployment](#fastapi--server-deployment)
- [Configuration reference](#configuration-reference)
- [Limitations](#limitations)

---

## How it works

| Stage                 | What happens                                                                                                                   |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| **1 — Text**          | PyMuPDF extracts the native text layer from every page                                                                         |
| **2 — Images/Tables** | YOLOv8 DocLayNet (bundled ONNX) detects Picture and Table regions, re-renders those regions at 300 DPI, and saves them as PNGs |

The ONNX model (`yolov8n-doclaynet.onnx`) is included inside the wheel — there
is no separate download step.

---

## Installation

> `onnxruntime` is included by default (CPU). For GPU, use `[gpu]` instead.
> Do **not** install both — the two `onnxruntime` builds conflict.

### Default (CPU) — works everywhere, no GPU required

```bash
pip install saraldocling
```

### GPU — NVIDIA CUDA

Requires CUDA 11.8+ and cuDNN on the host.  
Compatibility: https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html

```bash
pip install "saraldocling[gpu]"
```

### Apple Silicon — CoreML acceleration (M1/M2/M3/M4)

The default install works fine on ARM. For ANE-accelerated inference:

```bash
pip install saraldocling
pip install onnxruntime-silicon      # replaces the cpu wheel
```

### Linux / Debian server — one-time system packages

```bash
sudo apt-get install -y libgl1 libglib2.0-0
pip install saraldocling
```

### From source (development)

```bash
git clone https://github.com/yourname/saraldocling
cd saraldocling
pip install -e ".[dev]"
```

---

## CLI usage

The bundled model runs automatically. No `--model` flag needed.

```bash
# Full pipeline: text + image/table crops (default)
saraldocling paper.pdf

# Text only — skip YOLO, much faster
saraldocling paper.pdf --no-images

# Custom output directory
saraldocling paper.pdf -o ./results

# Preview extracted text in the terminal
saraldocling paper.pdf --preview 800

# One .txt file per page
saraldocling paper.pdf --pages

# GPU (CUDA) — needs saraldocling[gpu]
saraldocling paper.pdf --gpu

# Apple Silicon CoreML
saraldocling paper.pdf --coreml

# Debug: see every step logged
saraldocling paper.pdf -v

# All flags
saraldocling --help
```

### What gets written

```
paper_out/
├── extracted_text.txt        ← all pages joined
├── manifest.json             ← JSON summary of the run
├── pages/                    ← only with --pages
│   ├── page_001.txt
│   └── page_002.txt
└── extracted_images/         ← Picture + Table crops at 300 DPI
    ├── p0_Picture_231.png
    └── p2_Table_445.png
```

Output directory is auto-named `<pdf_stem>_out/` next to the PDF unless you
pass `-o`.

---

## Python API

### One-call pipeline (recommended)

```python
from saraldocling import parse_pdf, ParseConfig

# Full pipeline — bundled model used automatically
result = parse_pdf(ParseConfig(
    pdf_path="paper.pdf",
    output_dir="./out",
))

print(result.num_pages)       # int
print(result.text[:500])      # full extracted text
print(result.image_paths)     # list[str] of saved PNG paths
```

```python
# Text only — skip YOLO
result = parse_pdf(ParseConfig(
    pdf_path="paper.pdf",
    output_dir="./out",
    extract_images=False,
))
```

### Lower-level: text extraction only

```python
from saraldocling import PDFProcessor

with PDFProcessor("paper.pdf") as proc:
    full_text  = proc.extract_text()
    page_text  = proc.extract_text_by_page(1)      # 0-indexed
    page_image = proc.extract_page_image(0, dpi=150)  # PIL.Image
```

### Lower-level: YOLO extraction only

```python
from saraldocling import ImageExtractor

# CPU (default)
with ImageExtractor() as ext:
    paths = ext.extract_images_from_pdf("paper.pdf", "./out")

# GPU
with ImageExtractor(providers=["CUDAExecutionProvider", "CPUExecutionProvider"]) as ext:
    paths = ext.extract_images_from_pdf("paper.pdf", "./out")

# Apple Silicon
with ImageExtractor(providers=["CoreMLExecutionProvider", "CPUExecutionProvider"]) as ext:
    paths = ext.extract_images_from_pdf("paper.pdf", "./out")
```

### Using a custom model (advanced)

```python
result = parse_pdf(ParseConfig(
    pdf_path="paper.pdf",
    output_dir="./out",
    model_path="/path/to/custom.onnx",   # overrides the bundled model
))
```

---

## Output structure

`manifest.json` example:

```json
{
  "pdf": "/abs/path/to/paper.pdf",
  "pages": 12,
  "text_chars": 38201,
  "text_file": "/abs/path/to/paper_out/extracted_text.txt",
  "per_page_files": [],
  "image_paths": [
    "/abs/path/to/paper_out/extracted_images/p0_Picture_231.png",
    "/abs/path/to/paper_out/extracted_images/p3_Table_112.png"
  ]
}
```

Crop filenames follow the pattern `p<page>_<Label>_<x>.png` where `<Label>`
is either `Picture` or `Table`.

---

## FastAPI / server deployment

`saraldocling` is a plain pip package — drop it into any FastAPI app on Debian
with no special system configuration beyond the apt packages above.

### Minimal endpoint

```python
# main.py
import shutil, uuid
from pathlib import Path

from fastapi import FastAPI, File, UploadFile
from fastapi.responses import JSONResponse
from saraldocling import parse_pdf, ParseConfig

OUTPUT_ROOT = Path("/tmp/saraldocling")

app = FastAPI()

@app.post("/parse")
async def parse(file: UploadFile = File(...)):
    job_dir = OUTPUT_ROOT / str(uuid.uuid4())
    job_dir.mkdir(parents=True, exist_ok=True)
    pdf_path = job_dir / file.filename

    with open(pdf_path, "wb") as f:
        shutil.copyfileobj(file.file, f)

    result = parse_pdf(ParseConfig(
        pdf_path=str(pdf_path),
        output_dir=str(job_dir),
    ))

    return JSONResponse({
        "pages":       result.num_pages,
        "text":        result.text,
        "image_paths": result.image_paths,
    })
```

```bash
pip install saraldocling fastapi uvicorn python-multipart
uvicorn main:app --host 0.0.0.0 --port 8000
```

### Pre-load the ONNX session at startup

Creating an `ImageExtractor` loads and optimises the ONNX graph (~300 ms).
Do this once at startup and reuse it — `session.run()` is thread-safe:

```python
from contextlib import asynccontextmanager
from fastapi import FastAPI
from saraldocling import ImageExtractor

extractor: ImageExtractor | None = None

@asynccontextmanager
async def lifespan(app: FastAPI):
    global extractor
    extractor = ImageExtractor()      # loads bundled model
    yield
    extractor.close()

app = FastAPI(lifespan=lifespan)

# then in your endpoint, call extractor.extract_images_from_pdf() directly
```

---

## Configuration reference (`ParseConfig`)

| Field            | Default                    | Description                                     |
| ---------------- | -------------------------- | ----------------------------------------------- |
| `pdf_path`       | required                   | Path to the source PDF                          |
| `output_dir`     | required                   | Root directory for all output files             |
| `extract_images` | `True`                     | Run YOLO stage. `False` = text only             |
| `model_path`     | `None`                     | Custom `.onnx` path. `None` = use bundled model |
| `conf_threshold` | `0.30`                     | YOLO confidence cutoff                          |
| `nms_threshold`  | `0.45`                     | NMS IoU cutoff                                  |
| `min_box_size`   | `30`                       | Minimum crop side-length in pixels              |
| `num_workers`    | `cpu_count`                | Thread-pool size for page processing            |
| `onnx_providers` | `["CPUExecutionProvider"]` | ONNX Runtime execution providers                |

---

## DocLayNet labels

The model detects 11 document element types.  
Only **`Picture`** and **`Table`** regions are saved as crops.

```
Caption   Footnote   Formula   List-item   Page-footer
Page-header   Picture   Section-header   Table   Text   Title
```

---

## Limitations

**Not OCR** — requires a native text layer in the PDF. For scanned documents:

```bash
pip install ocrmypdf
ocrmypdf scanned.pdf scanned_ocr.pdf
saraldocling scanned_ocr.pdf
```

**Thread safety** — both `PDFProcessor` and `ImageExtractor` use internal
mutex locks (mirroring the Go `SafeDocument`). Safe for concurrent use.

---

## License

This package is distributed under **AGPL-3.0** because it bundles
`yolov8n-doclaynet.onnx`, which carries that licence.

What this means in practice:

- Free to use for any purpose, including commercial.
- If you modify the source and run it as a network service, you must make
  your modified source available under AGPL-3.0.
- The full licence text is at https://www.gnu.org/licenses/agpl-3.0.html
