Metadata-Version: 2.4
Name: pdf-jsonl-smith
Version: 0.1.0
Summary: Convert born-digital PDF text blocks to BigQuery-ready JSONL.
Author-email: Eiichi YAMAMOTO <info@yeiichi.com>
License-Expression: MIT
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: pymupdf
Provides-Extra: dev
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: python-semantic-release; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Provides-Extra: docs
Requires-Dist: furo; extra == 'docs'
Requires-Dist: sphinx; extra == 'docs'
Description-Content-Type: text/markdown

# pdf-jsonl-smith

`pdf-jsonl-smith` is a thin, boring wrapper around PyMuPDF for converting
text-based PDF files into BigQuery-ready newline-delimited JSON.

It is intended for upstream ELT pipelines that need flat text-unit records,
not document understanding.

## Install

```bash
uv pip install -e ".[dev]"
```

## CLI

```bash
pdf2jsonl input.pdf -o text_units.jsonl
```

Optional metadata:

```bash
pdf2jsonl input.pdf \
  --output text_units.jsonl \
  --source-uri gs://bucket/input.pdf \
  --schema-version 0.1
```

By default, the CLI prints a short completion summary to stdout:

```text
Wrote 42 records to text_units.jsonl from input.pdf.
```

Use `--quiet` to suppress the summary.

## Python API

```python
from pdf_jsonl_smith import extract_pdf_to_jsonl

count = extract_pdf_to_jsonl(
    input_path="input.pdf",
    output_path="text_units.jsonl",
    source_uri="gs://bucket/input.pdf",
)
```

The function returns the number of JSONL records written.

## Output

The output is UTF-8 JSONL: one standalone JSON object per text unit. Records are
flat and BigQuery-friendly.

For v0.1, extraction starts from PyMuPDF text blocks. Adjacent blocks on the
same page may be merged when their geometry looks like wrapped lines from the
same paragraph. This is intentionally conservative and does not try to infer
document semantics.

Fields:

```text
schema_version STRING
doc_id STRING
source_uri STRING
filename STRING
page_number INT64
unit_type STRING
unit_index INT64
text STRING
bbox_x0 FLOAT64
bbox_y0 FLOAT64
bbox_x1 FLOAT64
bbox_y1 FLOAT64
char_count INT64
word_count INT64
extractor STRING
extracted_at TIMESTAMP
warning STRING
```

## Supported

* Born-digital PDFs.
* PyMuPDF block-based text extraction.
* Conservative same-page wrapped-line merging.
* UTF-8 JSONL output.
* BigQuery-friendly flat records.
* Images ignored.

## Not Supported

* Scanned PDFs / OCR.
* Table reconstruction.
* Perfect reading order.
* Cross-page paragraph reconstruction.
* Semantic heading detection.
* Forms.
* Image text.

## Development

```bash
uv sync --dev
uv run pytest
uv run ruff check
uv run sphinx-build -b html docs/source docs/build/html
uv build
```

## Project Infrastructure

This project is maintained as a `uv`-based Python package.

Releases use Python Semantic Release with zero-version releases enabled for the
initial `0.x` series. The GitHub Actions release workflow runs on `main`,
creates releases from conventional commits, builds distributions with `uv`, and
publishes to PyPI using trusted publishing.

Documentation sources live in `docs/source`, build output goes under
`docs/build`, and the site uses Sphinx with the Furo theme. It is intended to be
built by Read the Docs through the RTD GitHub App integration.

## License

MIT
