Metadata-Version: 2.4
Name: cheap-ocr
Version: 0.0.1
Summary: Cheap, fast single-GPU PDF OCR (PP-DocLayoutV3 + GLM-OCR via vLLM)
Author: Mathis Lucka, Miriam Kümmel
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: aiohttp>=3.13.0
Requires-Dist: numpy>=1.26.0
Requires-Dist: opencv-python-headless>=4.10.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: pypdfium2>=4.30.0
Requires-Dist: typer>=0.12.0
Requires-Dist: adlfs>=2026.5.0 ; extra == 'azure'
Requires-Dist: azure-identity>=1.25.3 ; extra == 'azure'
Requires-Dist: azure-storage-blob>=12.30.0 ; extra == 'azure'
Requires-Dist: fsspec>=2026.6.0 ; extra == 'azure'
Requires-Dist: fsspec>=2026.6.0 ; extra == 'gcs'
Requires-Dist: gcsfs>=2026.6.0 ; extra == 'gcs'
Requires-Dist: accelerate>=1.13.0 ; extra == 'gpu'
Requires-Dist: huggingface-hub>=0.27.0 ; extra == 'gpu'
Requires-Dist: sentencepiece>=0.2.0 ; extra == 'gpu'
Requires-Dist: transformers>=5.3.0 ; extra == 'gpu'
Requires-Dist: vllm==0.21.0 ; sys_platform == 'linux' and extra == 'gpu'
Requires-Dist: modal>=1.5.0 ; extra == 'modal'
Requires-Dist: fsspec>=2026.6.0 ; extra == 's3'
Requires-Dist: s3fs>=2026.6.0 ; extra == 's3'
Requires-Dist: onnx>=1.17.0 ; extra == 'trt'
Requires-Dist: onnxruntime-gpu>=1.20.0 ; sys_platform == 'linux' and extra == 'trt'
Requires-Dist: tensorrt>=10.0.0,<11 ; sys_platform == 'linux' and extra == 'trt'
Maintainer: Blue Guardrails
Maintainer-email: Blue Guardrails <info@blueguardrails.com>
Requires-Python: >=3.12, <3.15
Project-URL: Documentation, https://github.com/blue-guardrails/cheap-ocr#readme
Project-URL: Repository, https://github.com/blue-guardrails/cheap-ocr
Provides-Extra: azure
Provides-Extra: gcs
Provides-Extra: gpu
Provides-Extra: modal
Provides-Extra: s3
Provides-Extra: trt
Description-Content-Type: text/markdown

# cheap-ocr

`cheap-ocr` was born from the need to OCR a large corpus of PDF documents with complex layouts quickly and at low cost. It converts PDF documents to markdown and json, including documents with complex layouts and tables. It uses  `pp-doclayout-v3` for layout detection and GLM-OCR to recognize text. The throughput is 20 pages per second on our internal benchmarks. It runs on a single L40S GPU (or similar).

<img src="assets/readme-example.png" alt="A PDF page with cheap-ocr layout bounding boxes next to the rendered Markdown output." width="900">

## Benchmarks

We use 100 PDF documents from the [European Medicines Agency](https://www.ema.europa.eu/en/medicines/download-medicine-data) to benchmark throughput. The documents are typical business documents with a mix of tables, text, and some figures. All metrics are reported end-to-end including download and upload. Warm-up (model loading and initialization) is excluded from our measurements. We compare `cheap-ocr` against the official GLM-OCR SDK. The benchmarks ran on a single L40S for both solutions.

| Metric | GLM-OCR SDK | cheap-ocr |
| --- | ---: | ---: |
| Pages/sec | 5.29 | 14.26 |
| Output tokens/sec | 4,316 | 11,801 |
| Page regions/sec | 75 | 205.4 |


## Features

| Feature | Description |
| --- | --- |
| Fast & cheap | Fastest and cheapest VLM-based OCR that we know of. |
| VLM-based OCR | State-of-the-art text recognition that handles difficult text extraction well. |
| Layout detection | Recognizes bounding boxes for headings, paragraphs, images, tables, and more. |
| OCR from cloud storage | Processes PDFs from and writes outputs to S3 buckets, Azure storage containers, or Google Cloud buckets. |
| One-line deployment | Runs the full pipeline on Modal with a single CLI command. |
| Bare-metal deployment | Runs on any machine with an NVIDIA GPU. |


## Use on Modal (easiest, scroll down for bare metal use)

Install `cheap-ocr` and log into Modal or register a new account.

```bash
uv install cheap-ocr[modal,gpu]
modal login
```

Provide a source folder with PDFs and a target folder where you want to store the outputs.
Use the `--limit` parameter to limit the number of PDFs to process.

```bash
# run with local files
# startup takes a few minutes because models need to be loaded and CUDA graphs compiled
uv run cheap-ocr modal --source input_folder/with_pdfs/ --target output_folder/for_results/

# to run cloud storage to cloud cloud storage, install cheap-ocr[s3,azure,gcs]
# in this setup, you can detach from the process
uv run cheap-ocr modal --source "s3://in" --target "s3://out" --detach
```

## Use on bare metal GPU

Follow these instructions, to run `cheap-ocr` on any GPU machine.

Clone the repository.

```bash
git clone https://github.com/blue-guardrails/cheap-ocr.git
cd cheap-ocr
```

Now build the docker image to run it or directly install the project into a virtual environment.

```bash
# docker
docker build -t cheap-ocr .

# Local in/out: mount the folders and reuse the cached weights.
docker run --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -v "$PWD/input:/data/in" -v "$PWD/output:/data/out" \
  cheap-ocr --source /data/in --target /data/out

# Cloud in/out: no mounts — pass URIs plus credentials as env (-e).
docker run --gpus all \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY \
  cheap-ocr --source s3://bucket/in --target s3://bucket/out --limit 100


# Virtual environment (uv creates .venv automatically if needed)
uv sync --python 3.12 --frozen --no-dev --extra gpu --extra trt --extra s3   # add --extra gcs/azure as needed

uv run cheap-ocr --source ./input --target ./output
uv run cheap-ocr --source s3://bucket/in --target s3://bucket/out --limit 100
```

## Use in code

```python
from cheap_ocr import EngineConfig, OcrConfig, DocumentEngine

engine = DocumentEngine().start()          # loads layout model, boots vLLM

# Startup knobs via EngineConfig, per-run knobs via OcrConfig (or per call);
# unset fields fall back to CHEAP_OCR_* env vars, then the built-in defaults.
engine = DocumentEngine(
    EngineConfig(gpu_memory_utilization=0.7),
    run_config=OcrConfig(pdf_dpi=150),
).start()

result = await engine.ocr(pdf_bytes)            # one PDF, in memory
print(result.markdown, result.tokens.total_tokens)

async for doc in engine.stream(documents, config):   # many, as each finishes
    ...

summary = await engine.run("s3://in", "s3://out")    # storage-to-storage, resumable
```

## Outputs

For each `foo.pdf` under the source, cheap-ocr writes one directory named after
the relative stem in the target:

- `foo/content.md` — GLM-OCR-formatted Markdown
- `foo/layout.json` — per-page region JSON
- `foo/stats.json` — per-document timings, token usage, region counts and completion marker

A run-level `_run_summary.json` is written too, with document/page/token totals,
throughput, and per-batch status.

## Configuration

Most settings resolve the same way: **explicit value → `CHEAP_OCR_*` environment
variable → built-in default**. "Explicit" means a CLI flag (`--pdf-dpi 150`,
`--max-num-seqs 512`, `--gpu H100`) or a constructor argument in code
(`DocumentEngine(EngineConfig(...))`, `OcrConfig(...)`). Plain CLI routing flags are
called out below. OCR crops are always sent to vLLM as inline base64 data URLs.

The tables below list every operational flag on `cheap-ocr` and
`cheap-ocr modal` (except `--help`, which prints the command help and exits).

**Common CLI flags** (both commands):

| Flag | Default | Explanation |
| --- | --- | --- |
| `--source` | `./input` | Local folder/file or cloud URI containing PDFs. Plain CLI flag. |
| `--target` | `./output` | Local folder or cloud URI for OCR outputs. Plain CLI flag. |
| `--limit` | all incomplete PDFs | Maximum number of incomplete PDFs to process in this run. Plain CLI flag. |
| `--log-level` | `INFO` | Log level (`DEBUG`, `INFO`, `WARNING`, ...); also reads `CHEAP_OCR_LOG_LEVEL`. |

**Run settings** (`OcrConfig`, both commands):

| Flag | Default | Explanation |
| --- | --- | --- |
| `--force/--no-force` | off | Reprocess documents whose outputs are already complete. |
| `--pdf-dpi` | `100` | PDF render DPI. 100 is throughput-tuned; raise for accuracy. |
| `--pdf-max-side` | `3500` | Longest rendered page side in pixels; larger pages are downscaled. |
| `--page-chunk-size` | `8` | Pages streamed through render → layout → crop → OCR as one unit. |
| `--jpeg-quality` | `90` | JPEG quality for OCR crops before base64 encoding. |
| `--max-tokens-text` | `2048` | Maximum generated tokens for text regions. |
| `--max-tokens-formula` | `2048` | Maximum generated tokens for formula regions. |
| `--max-tokens-table` | `4096` | Maximum generated tokens for table regions. |
| `--temperature` | `0.0` | OCR generation temperature. |
| `--top-p` | `0.00001` | OCR generation nucleus-sampling `top_p`. |
| `--max-inflight-pdfs` | `16` local, `32` on Modal | Documents rendered, laid out, and cropped concurrently. Modal uses 32 only when neither flag nor env var is set. |
| `--ocr-request-concurrency` | `1024` | Concurrent OCR requests to the vLLM server; keep near `--max-num-seqs`. |
| `--storage-check-workers` | `64` | Threads reading completion markers when resuming against cloud targets. |

**Engine settings** (`EngineConfig`, both commands; startup-scoped):

| Flag | Default | Explanation |
| --- | --- | --- |
| `--vllm-port` | `8000` | Local port for the OpenAI-compatible vLLM server. |
| `--vllm-health-timeout` | `900` | Seconds to wait for the vLLM server to become healthy. |
| `--gpu-memory-utilization` | `0.60` | vLLM's share of GPU memory; remaining headroom is for the co-located layout model. |
| `--max-model-len` | `8192` | vLLM maximum model context length. |
| `--max-num-seqs` | `1024` | vLLM maximum concurrent sequences. OOM-safe to raise within the reserved KV pool. |
| `--max-num-batched-tokens` | `16384` | vLLM batched-token budget. |
| `--api-server-count` | `4` | vLLM API server processes; scale with available CPU cores. |
| `--speculative-config` | `{"method": "mtp", "num_speculative_tokens": 3}` | JSON passed to vLLM `--speculative-config`; an empty string disables it. |
| `--vllm-extra-args` | empty | Extra shell-style arguments appended to `vllm serve`. |
| `--layout-backend` | `onnx` | Layout detector backend: `onnx` (ONNX Runtime + TensorRT FP16) or `transformers` (PyTorch). |
| `--layout-batch-size` | `4` | Layout detection batch size; small batches interleave best with OCR. |
| `--layout-threshold` | backend default | Layout detection score threshold: `0.5` for `onnx`, `0.3` for `transformers`. |
| `--trt-layout-cache` | `/root/.cache/vllm/trt_layout` | TensorRT layout engine cache directory. |
| `--trt-builder-opt-level` | `1` | TensorRT builder optimization level. |
| `--crop-encode-workers` | `8` | Threads for crop/resize/JPEG/base64 preparation. |

**Batch policy** (`BatchPolicy`, both commands):

| Flag | Default | Explanation |
| --- | --- | --- |
| `--batch-docs` | `64` | Maximum PDFs per GPU batch. |
| `--batch-bytes-mb` | `512` | Approximate maximum input size per GPU batch, in MB (`CHEAP_OCR_BATCH_MAX_BYTES` is bytes). |
| `--batch-pages` | unlimited | Maximum PDF pages per GPU batch. |
| `--batch-retries` | `3` | Retries for a failed or preempted GPU batch. |
| `--batch-retry-backoff-seconds` | `10` | Initial retry backoff seconds for failed batches. |

**Modal-only flags** (`cheap-ocr modal`):

| Flag | Default | Explanation |
| --- | --- | --- |
| `--detach/--no-detach` | off | Submit a detached cloud run and return immediately. Requires cloud source and cloud target. Plain CLI flag. |
| `--app-name` | `cheap-ocr` | Modal app name. |
| `--gpu` | `A100-40GB` | Modal GPU type. |
| `--cpu-cores` | `16` | Modal CPU request. |
| `--cpu-limit` | none | Optional Modal CPU limit. |
| `--scaledown-window` | `900` | Modal container scaledown window in seconds. |
| `--startup-timeout` | `1200` | Modal container startup timeout in seconds. |
| `--timeout` | `21600` | Modal call timeout in seconds. |
| `--max-containers` | `1` | Maximum Modal GPU containers. |
| `--modal-secrets` | none | Comma-separated Modal secret names mounted into workers (`CHEAP_OCR_MODAL_SECRET_NAMES`). |


## Development

Install the development environment with uv:

```bash
uv sync --frozen --group dev
```

Common checks are available through the `Makefile`:

```bash
make lint          # Ruff lint checks
make format-check  # verify Ruff formatting
make types         # Pyright, including the Modal extra
make unit          # GPU-free test suite
make check         # lint, formatting, types, and tests
```

GitHub Actions workflows:

- `.github/workflows/ci.yml` runs lint, formatting, type checks, and tests on pull requests and pushes to `main`. Tests run on Python 3.12, 3.13, and 3.14.
- `.github/workflows/zizmor.yml` runs `zizmor` against the GitHub Actions workflows and uploads results to GitHub code scanning.
- `.github/workflows/pypi_release.yml` builds and publishes releases to PyPI using trusted publishing when a `v*` tag is pushed.

All workflow actions are pinned to full commit SHAs. When updating actions, use the newest available action version and update both the SHA and the version comment.

### Release flow

Releases are published from tags through PyPI trusted publishing; no PyPI token is stored in GitHub.

One-time setup:

1. Create a GitHub environment named `pypi`.
2. Configure a PyPI trusted publisher for this repository with workflow `.github/workflows/pypi_release.yml` and environment `pypi`.

To publish a release:

1. Ensure `main` is green.
2. Choose the version, e.g. `1.2.3`.
3. Create and push a matching tag:

   ```bash
   git tag v1.2.3
   git push origin v1.2.3
   ```

The release workflow strips the leading `v`, updates the package version for the build, builds the wheel and source distribution, smoke-tests both artifacts, and publishes them with `uv publish --trusted-publishing always`.

## Acknowledgements

This work is only a thin layer on strong foundations:

- [GLM OCR SDK](https://github.com/zai-org/GLM-OCR)
- [PaddlePaddle/PP-DocLayoutV3](https://huggingface.co/PaddlePaddle/PP-DocLayoutV3)
- [bndos/pp-doclayout-v3-trt](https://huggingface.co/bndos/pp-doclayout-v3-trt)

## License

Apache-2.0.
