Metadata-Version: 2.4
Name: pagepilot
Version: 0.3.0
Summary: Standalone PDF page-indexing SDK: layout heuristics, tree index, summaries, retrieval, and citation-answer chat.
Author: Parth
License: MIT
Project-URL: Homepage, https://github.com/vaibhavGala262/PagePilot
Project-URL: Repository, https://github.com/vaibhavGala262/PagePilot
Project-URL: Issues, https://github.com/vaibhavGala262/PagePilot/issues
Keywords: pdf,indexing,rag,ocr,tree,qa
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pymupdf>=1.24
Requires-Dist: ocrmypdf>=16
Requires-Dist: pydantic>=2
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: build>=1; extra == "dev"
Requires-Dist: twine>=5; extra == "dev"
Dynamic: license-file

# PagePilot

Standalone, provider-agnostic PDF **page-indexing SDK**. Layout heuristics
build a nested tree index of a document; retrieval and citation-answer chat
run on an **injected** LLM callable — PagePilot never imports a provider SDK,
so it works with any model/tool (OpenAI, Gemini, Groq, Copilot, LiteLLM, …).

Vectorless, reasoning-based retrieval: the LLM navigates a table-of-contents
tree the way a human reader would, then answers with **exact page (and
document) citations**. No vector DB, no chunking required — but chunks with
section paths are a first-class export for hybrid/vector pipelines.

## Install

```sh
pip install pagepilot          # from PyPI (0.3.0+)
# or, for development:
python -m venv .venv && .\.venv\Scripts\activate
pip install -e ".[dev]"
```

## Quick start

```python
from pagepilot import PagePilotClient
from pagepilot.llm import ChatCompletion

def llm(messages, *, model=None, max_tokens=None):
    return my_provider.chat(messages, model=model, max_tokens=max_tokens)

client = PagePilotClient(ChatCompletion(llm, chat_model="your/model"),
                         summary_concurrency=1)   # gentle by default

entry = client.submit_document("report.pdf")       # llm-free structure, or:
# entry = client.submit_document("report.pdf", mode="skip-summaries")  # never calls the LLM
doc_id = entry["doc"]["id"]

# OpenAI-shaped answer dict + cited result
answer = client.chat("What was the 2023 margin?", doc_id=doc_id)
print(answer["choices"][0]["message"]["content"])

# Typed Answer with page + document sources
typed = client.answer([{"role": "user", "content": "…"}], doc_id=doc_id)
print(typed.content, [c for c in typed.citations])
```

Multi-turn conversations are **context-aware**: page and document selection
receive the earlier turns, so follow-ups ("…and the second part?") resolve
correctly.

### Multi-document chat

```python
ids = [client.submit_document(d)["doc"]["id"] for d in ("a.pdf", "b.pdf")]
answer = client.chat("Compare the risk factors across these filings.",
                     doc_id=ids)
print([f"{s.doc_name} p.{s.page}" for s in client.answer([], doc_id=ids).sources])
```

The client picks relevant documents by reasoning over the library index, then
retrieves pages per document and cites the correct source document.

### Change detection

```python
r1 = client.submit_document("report.pdf")   # {"status": "ingested"}
r2 = client.submit_document("report.pdf")   # {"status": "unchanged"} (same sha256)
r3 = client.submit_document("report.pdf", force=True)   # re-index
```

### Chunks & export (for vector pipelines)

```python
from pagepilot import chunk_document

chunks = chunk_document(client.raw_tree(doc_id), source=doc_id)
for c in chunks[:3]:
    print(c.section_path, c.page, c.content[:60])

print(client.export(doc_id, fmt="json"))       # json | csv | markdown | tree
```

### Index health (zero LLM calls)

```python
from pagepilot import assess_library
for r in assess_library(client.storage_path):
    print(r["name"], r["coverage"], r["problems"])
```

Or via CLI: `python scripts/assess_index.py --storage <store>` (exit code 1 on
failures — CI-safe).

## What it does

1. **Detect** — per-page text-layer analysis (`pagepilot.detect`)
2. **OCR** — optional, via OCRmyPDF + Tesseract for scanned PDFs
   (`pagepilot.ocr`)
3. **Layout extract** — pure heuristics (no LLM): columns, body-font stats,
   header/footer stripping, TOC recognition, heading candidates, doc title,
   embedded-PDF-bookmark merge, graceful degradation to coarse page groups
   (`pagepilot.layout`)
4. **Tree** — node walk / validation / pruning; typed `TreeNode` view
   (`pagepilot.tree`, `pagepilot.models`)
5. **Summaries** — bottom-up, gentle concurrency (`pagepilot.summarize`)
6. **Library** — corpus-level document selection for multi-doc chat
   (`pagepilot.library`)
7. **Retrieval** — one-shot page pick from the tree, history-aware
   (`pagepilot.retrieve`)
8. **Chat** — cited answers (page + document), OpenAI-shaped output, streaming
   surface (`pagepilot.chat`)
9. **Chunk/export** — hierarchy-aware chunks + JSON / CSV / Markdown / tree
   serializers (`pagepilot.chunk`, `pagepilot.export`)
10. **Assess** — zero-LLM index health reports (`pagepilot.assess`)
11. **Storage** — JSON per-doc dirs under a user storage root
    (`pagepilot.storage`)

## Examples

```sh
python examples/quickstart.py path/to/report.pdf other.pdf
python examples/export_chunks.py report.pdf out/
python examples/index_health.py path/to/report.pdf
python -m pagepilot.demo path/to.pdf     # deterministic fake-LLM demo
```

Examples run against an injected fake LLM — no keys, no network.

## Design notes

- **No provider coupling** — the LLM seam (`llm.py`) is the only
  network-ward interface; providers own their retry/quota/header logic.
- **Cheap ingest** — structure extraction is entirely heuristic; summaries are
  the only per-ingest LLM cost, or zero with `mode="skip-summaries"`.
- **Honest degradation** — documents that defeat heuristics fall back to page
  groups, never fabricated subsection structure.
- **Traceable answers** — every statement carries `[document p.page]` citation
  from the actually-read pages.

## Development

```sh
python -m venv .venv
.\.venv\Scripts\activate        # Windows PowerShell
python -m pip install -e ".[dev]"
python -m pytest -q
```

## License

MIT — see `LICENSE`.
