Metadata-Version: 2.4
Name: prettypipeline-ocr
Version: 0.1.0
Summary: Turn PDFs into structured JSON with local OCR and a cheap cloud LLM.
Author: greatshoor
License-Expression: MIT
Project-URL: Homepage, https://github.com/virajshoor/PrettyPipeline
Keywords: ocr,pdf,llm,extraction,json
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Requires-Python: <3.14,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pymupdf>=1.24
Requires-Dist: transformers<5,>=4.57.1
Requires-Dist: torch
Requires-Dist: torchvision
Requires-Dist: pillow
Requires-Dist: openai>=1.0
Requires-Dist: einops
Requires-Dist: addict
Requires-Dist: easydict
Requires-Dist: safetensors
Requires-Dist: tqdm
Requires-Dist: matplotlib
Requires-Dist: requests
Dynamic: license-file

# PrettyPipeline

Turn PDFs into structured JSON. OCR runs **locally**; field extraction uses a **cheap cloud LLM**.

## Why this is cheap

Cloud OCR APIs charge per page. PrettyPipeline does the expensive vision work on your GPU with [baidu/Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR) (MIT, 3B params, 32K context — a whole PDF in one `infer_multi()` pass). The only API call is [GPT-5.4 nano](https://developers.openai.com/api/docs/models/gpt-5.4-nano) on the resulting text (~$0.20 / 1M input tokens), prompted with **your** JSON schema.

You pay for a small text completion, not for pixels.

```
PDF  →  PyMuPDF pages  →  Unlimited-OCR (local GPU)  →  raw text
                                                      ↓
                              GPT-5.4 nano (OpenAI)  →  JSON
                                                      ↓
                              validation             →  needs_review flags
```

Nulls, model-marked uncertain fields, and OCR that looks garbled are flagged for a human instead of being silently accepted.

## Install

Python **3.10–3.13** (3.12 recommended). NVIDIA GPU (CUDA) or Apple Silicon (MPS). CPU works but is slow.

```bash
python3.12 -m venv .venv
source .venv/bin/activate
pip install prettypipeline-ocr
# or from this repo:
pip install -e .
```

On NVIDIA Linux, install a CUDA build of PyTorch first if the default wheel is CPU-only:

```bash
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124
pip install -e .
```

The first run downloads `baidu/Unlimited-OCR` from Hugging Face (~6GB).

## Usage

```bash
export OPENAI_API_KEY=sk-...
prettypipeline run invoice.pdf --schema examples/invoice.schema.json
prettypipeline run invoice.pdf --schema examples/invoice.schema.json -o out.json
prettypipeline run invoice.pdf --schema examples/invoice.schema.json --ocr-only
prettypipeline run invoice.pdf --schema examples/invoice.schema.json --device mps
prettypipeline run invoice.pdf --schema examples/invoice.schema.json --max-length 2048
```

`--schema` is any JSON Schema. Swap the file to extract a different document type — nothing is hardcoded.

Device is auto-detected (`cuda` → `mps` → `cpu`). Override with `--device`.

On Apple Silicon, Unlimited-OCR can start looping (it hardcodes CUDA). Use `--max-length 2048` for short docs. NVIDIA CUDA is the model's native path.

## Output

```json
{
  "data": { "date": "…", "vendor": "…", "line_items": [], "total": 0 },
  "needs_review": [{ "field": "tax", "reason": "null" }],
  "ocr_text": "…"
}
```

## License

MIT. Unlimited-OCR is also MIT.
