Metadata-Version: 2.4
Name: dumb-pdf2md
Version: 0.1.0
Summary: Lightweight, model-free PDF-to-Markdown conversion
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
License-File: THIRD_PARTY_NOTICES.md
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: pdfplumber==0.11.10
Requires-Dist: pdftext==0.7.1
Description-Content-Type: text/markdown

# Dumb PDF2MD

[简体中文](docs/README.zh-CN.md)

dumb-pdf2md is a lightweight, typed Python library that converts text-based
PDF documents into structured, LLM-friendly Markdown. It reconstructs document
structure from the PDF text layer, geometry, and typography without layout
models, OCR models, GPU resources, or external services.

## Benchmarks

READoc selectable-text holdout (12 real-world PDFs, 161 pages):

| Metric | dumb-pdf2md | MarkItDown 0.1.6 | Docling 2.115.0 | PyMuPDF4LLM 1.28.0 | pdftext 0.7.1 | pdfplumber 0.11.10 |
|---|---:|---:|---:|---:|---:|---:|
| Text token F1 | 0.852 | 0.756 | 0.899 | 0.863 | 0.893 | 0.691 |
| Reading-order bigram F1 | 0.734 | 0.627 | 0.831 | 0.793 | 0.818 | 0.562 |
| Table-content F1 (6-PDF subset) | 0.452 | 0.326 | 0.890 | 0.685 | N/A | N/A |
| Conversion time | 188.7 ms/page | 53.5 ms/page | 917.1 ms/page | 143.7 ms/page | 24.2 ms/page | 53.6 ms/page |
| Peak RSS | 248.5 MiB | 115.1 MiB | 4.44 GiB | 667.7 MiB | 122.6 MiB | 202.6 MiB |
| Uses OCR model | No | No | Yes | No | No | No |
| Uses layout model | No | No | Yes | Yes | No | No |
| Installed size with dependencies | 112.5 MiB | 168.8 MiB | 5.25 GiB | 233.2 MiB | 70.0 MiB | 50.2 MiB |
| License | Apache-2.0 | MIT | MIT | AGPL-3.0 + PolyForm NC / commercial | Apache-2.0 | MIT |

`pdftext` and `pdfplumber` are the two base extraction libraries used by
dumb-pdf2md, not end-to-end Markdown converters. Their columns score native
text output without dumb-pdf2md's structure analysis or Markdown rendering,
so table-content F1 is not applicable to those baseline modes. `pdfplumber`
does provide table-finding APIs; dumb-pdf2md combines their candidates with
`pdftext` geometry, custom aligned-table detection, cell normalization,
deduplication, and Markdown rendering.

Model rows describe the exact benchmark configuration. Docling ran on CPU with
EasyOCR and its layout/table models enabled, without forcing full-page OCR on
pages that already contain text. PyMuPDF4LLM uses its installed
`pymupdf-layout` dependency. MarkItDown's optional OCR plugin and cloud
integrations were not enabled. The PyMuPDF4LLM license row includes the license
of its layout dependency.

Quality metrics are higher-is-better; time and memory are lower-is-better.
Performance uses three isolated process runs per tool, excludes import time,
and reports median conversion time and maximum peak RSS.

Installed size is the `site-packages` footprint in a clean Python 3.13
environment after installing the package and all transitive dependencies. It
excludes Python itself, package caches, and models downloaded at runtime.

Performance host: Intel Xeon Platinum 8488C, 16 vCPUs (8 cores/16 threads),
64 GB RAM, Ubuntu 24.04.4 LTS (x86_64), and CPython 3.13.3. Results are
directly comparable only on the same environment.

READoc was not used for parser tuning. See
[benchmark reproduction](benchmarks/README.md), the
[base-engine results](benchmarks/results/readoc-base-engines.json), the
[MarkItDown results](benchmarks/results/readoc-markitdown.json), the
[Docling results](benchmarks/results/readoc-docling.json), the
[performance results](benchmarks/results/readoc-performance.json), and the
[evaluation methodology](docs/EXTERNAL_HOLDOUT.md).

### Quality-resource map

```mermaid
quadrantChart
    title Overall extraction quality versus runtime memory
    x-axis Higher peak RSS --> Lower peak RSS
    y-axis Lower composite F1 --> Higher composite F1
    quadrant-1 Best quality-resource balance
    quadrant-2 High quality, high resource use
    quadrant-3 High resource use, weaker structure
    quadrant-4 Lightweight, weaker structure
    "MarkItDown": [0.96, 0.570]
    "pdfplumber": [0.82, 0.418]
    "★ Dumb PDF2MD": [0.77, 0.679]
    "PyMuPDF4LLM": [0.51, 0.781]
    "Docling": [0.02, 0.873]
```

Licenses:

- ★ Dumb PDF2MD: Apache-2.0
- MarkItDown: MIT
- pdfplumber: MIT
- PyMuPDF4LLM: AGPL-3.0 + PolyForm Noncommercial / commercial
- Docling: MIT

The horizontal coordinates reverse and normalize `log10(peak RSS)`, so lower
memory use moves a tool to the right while Docling's 4.44 GiB result does not
compress the other tools at the origin. They show relative resource tiers, not
raw memory values; the exact measurements are in the table above. Composite F1
is the equal-weight mean of text-token, reading-order-bigram, and table-content
F1; the table component uses the six-PDF subset with ground-truth tables. For
this visualization only, pdfplumber receives zero for the table component
because its benchmark mode does not produce Markdown tables; the source table
keeps this result as N/A because no table-quality measurement was made.

## Positioning

dumb-pdf2md fills the gap between raw text extraction and model-based document
understanding:

- It adds headings, lists, multi-column reading order, table reconstruction,
  deduplication, and Markdown rendering on top of lightweight PDF engines.
- It has higher text, reading-order, and table F1 than MarkItDown on this
  holdout, while remaining fully local and model-free.
- It uses substantially less memory and disk space than Docling, and less
  memory than PyMuPDF4LLM.
- Its Apache-2.0 runtime has no model-weight license, GPU requirement, model
  download, or external service dependency.

The tradeoff is deliberate: Docling is more accurate on complex structures
and supports scanned PDFs, while raw extractors are faster when structure is
not needed. dumb-pdf2md targets selectable-text PDFs where useful Markdown
structure matters but OCR/model deployment cost does not make sense.

## Features

- Accepts file paths, bytes-like objects, and readable binary streams.
- Runs locally on CPU without downloading layout or OCR models.
- Preserves headings, paragraphs, lists, footers, and reading order.
- Converts ruled and structurally aligned tables to Markdown tables.
- Handles common multi-column layouts and CJK text.
- Produces clean Markdown suitable for LLM context, indexing, and RAG
  pipelines.
- Returns page counts and non-fatal diagnostics alongside Markdown.
- Provides immutable result models and a documented exception hierarchy.
- Supports concurrent callers without taking ownership of caller-provided
  streams.

dumb-pdf2md is designed for PDFs that contain extractable text. Scanned or
image-only PDFs require OCR before conversion. The library does not currently
extract images, preserve inline text styling, or expose page-level bounding
boxes and metadata.

## Requirements

- Python 3.10 or later

## Installation

Install from the repository:

```bash
python -m pip install .
```

For development, install all dependency groups with
[uv](https://docs.astral.sh/uv/):

```bash
uv sync --all-groups
```

## Usage

Use `parse` when you need conversion metadata:

```python
from dumb_pdf2md import parse

result = parse("report.pdf")

print(result.markdown)
print(f"Pages: {result.page_count}")

for diagnostic in result.diagnostics:
    print(diagnostic.code, diagnostic.page, diagnostic.message)
```

Use `to_markdown` when you only need the generated Markdown:

```python
from dumb_pdf2md import to_markdown

markdown = to_markdown("report.pdf")
```

Both functions also accept PDF data and readable binary streams:

```python
from io import BytesIO

from dumb_pdf2md import parse

with open("report.pdf", "rb") as file:
    pdf_data = file.read()

from_bytes = parse(pdf_data)

stream = BytesIO(pdf_data)
from_stream = parse(stream)
assert not stream.closed
```

Caller-owned streams remain open after conversion.

## Parser Options

Document heuristics are enabled by default and can be controlled with
`ParserOptions`:

```python
from dumb_pdf2md import ParserOptions, parse

result = parse(
    "report.pdf",
    options=ParserOptions(
        promote_bold_headings=False,
        detect_columns=False,
    ),
)
```

- `promote_bold_headings` promotes qualifying short bold lines to headings.
- `detect_columns` detects common column layouts and restores reading order.

## Error Handling

All public conversion errors inherit from `PdfParserError`:

```python
from dumb_pdf2md import PdfInputError, PdfParserError, parse

try:
    result = parse("report.pdf")
except PdfInputError as error:
    print(f"Invalid PDF input: {error}")
except PdfParserError as error:
    print(f"PDF conversion failed: {error}")
```

`PdfInputError` reports invalid, empty, missing, or unreadable input.
`PdfConversionError` reports extraction failures or an empty conversion
result. Wrapped failures retain the original exception in `__cause__`.

## Development

Run the complete release checkpoint:

```bash
scripts/release-check.sh
```

Or run individual checks:

```bash
uv run ruff check .
uv run mypy
uv run pytest
uv build
```

The test suite covers the public API, parsing algorithms, resource cleanup,
concurrent use, distribution artifacts, and a reproducible golden PDF corpus.

## Documentation

- [API reference](docs/API.md)
- [Architecture](docs/ARCHITECTURE.md)
- [Contributor guide](CONTRIBUTING.md)
- [Benchmark reproduction](benchmarks/README.md)
- [Release verification](docs/RELEASE_VERIFICATION.md)
- [Changelog](CHANGELOG.md)
- [Third-party notices](THIRD_PARTY_NOTICES.md)

## License

Licensed under the [Apache License 2.0](LICENSE).
