Metadata-Version: 2.5
Name: pandion
Version: 0.1.0
Summary: Retrieval pipelines you can inspect, step by step
Project-URL: Homepage, https://rag.codewithnk.com
Project-URL: Documentation, https://github.com/nadeem4/rag-playground/blob/main/docs/pandion.md
Project-URL: Source, https://github.com/nadeem4/rag-playground/tree/main/packages/pandion
Project-URL: Issues, https://github.com/nadeem4/rag-playground/issues
Author: Nadeem Khan
License-Expression: MIT
License-File: LICENSE
Keywords: chunking,embeddings,pdf,rag,reranking,retrieval,search
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: <3.14,>=3.13
Requires-Dist: pydantic>=2.9
Requires-Dist: pyyaml>=6.0.3
Provides-Extra: anthropic
Requires-Dist: anthropic>=1.7.0; extra == 'anthropic'
Provides-Extra: docling
Requires-Dist: docling>=2.129.0; extra == 'docling'
Requires-Dist: onnxruntime>=1.30.0; extra == 'docling'
Provides-Extra: lancedb
Requires-Dist: lancedb<0.39,>=0.38; extra == 'lancedb'
Requires-Dist: pyarrow>=25.0.1; extra == 'lancedb'
Provides-Extra: local
Requires-Dist: sentence-transformers>=6.1.0; extra == 'local'
Requires-Dist: torch>=2.14.0; extra == 'local'
Provides-Extra: openai
Requires-Dist: openai>=3.17.0; extra == 'openai'
Provides-Extra: pdfium
Requires-Dist: pypdfium2>=5.13.0; extra == 'pdfium'
Description-Content-Type: text/markdown

# Pandion

Retrieval pipelines you can inspect, step by step.

Pandion parses your documents, cleans and chunks them, embeds and stores them, then
searches, reranks and answers questions. Every step is a strategy you can swap, and every
step tells you what it did. Each result is cached under a hash of its settings, so the
same files with the same settings are never processed twice.

Pandion is named after the osprey, which spots a fish under the water and dives to
retrieve it. It is the engine of [RAG Playground](https://rag.codewithnk.com), a web app
that shows every step of a pipeline on a PDF.

## Install

Pandion needs Python 3.13.

```bash
pip install "pandion[pdfium,lancedb,local]"
```

The base install is small. Parsers, stores and models are extras, and a step whose extra
is missing tells you which one to install:

| Extra | What it brings | Needed by |
|---|---|---|
| `pdfium` | pypdfium2 | the fast `pdfium` parser |
| `docling` | Docling and its OCR engine | the `docling` parser, for layout, tables and scans |
| `lancedb` | LanceDB and pyarrow | the index, and the `dense`, `bm25` and `hybrid_rrf` search |
| `local` | sentence-transformers and torch | the local embedders, and the `mmr` and `cross_encoder` rerankers |
| `anthropic` | the Anthropic SDK | `llm_rewrite`, `llm_rerank` and `chat` with Claude |
| `openai` | the OpenAI SDK | the same steps with OpenAI, OpenRouter or a compatible server |

`pandion install pipeline.yaml` installs the extras a pipeline file needs.

## Quick start

```python
from pandion import Pipeline

p = Pipeline(
    path="./my-index",
    parse="pdfium",
    chunk={"use": "recursive_character", "chunk_size": 400, "chunk_overlap": 40},
    embed="bge-small-en-v1.5",
    store="lancedb",
    search="hybrid_rrf",
    answer="search",
)
p.index.run("report.pdf")
print(p.ask("What does the report recommend?").text)
```

The embedder downloads once, on first use. After that, a second `p.index.run` on the
same file finds every step already stored and reuses it.

## What you can do

- **Run one step on its own**, with no pipeline and no cache:
  `pandion.parse(pdf, use="pdfium")`, `pandion.clean(...)`, `pandion.chunk(...)`.
- **Write a pipeline three ways**: as a YAML file (`Pipeline.from_file("pipeline.yaml")`),
  as a dict per step, or with classes such as `from pandion.parse import Docling`. The
  three give the same pipeline and the same cache.
- **Build and update an index**: `p.index.run(files)` indexes them, and
  `p.index.sync(folder)` adds, changes and removes files to match a folder.
- **Search**: `p.retrieve(question)` runs the rewrite, search and rerank steps and returns
  each step's result as well as the hits.
- **Ask**: `p.ask(question)` returns the pieces found, or an answer from a chat model
  that cites them.

## Learn more

- [The guide](https://github.com/nadeem4/rag-playground/blob/main/docs/pandion.md): every
  step, setting and error.
- [Examples](https://github.com/nadeem4/rag-playground/tree/main/examples/pandion): index,
  sync, search, ask, and the three ways to write a pipeline.
- [RAG Playground](https://rag.codewithnk.com): try the same pipelines in the browser.

## License

MIT. See [LICENSE](https://github.com/nadeem4/rag-playground/blob/main/LICENSE).
