Metadata-Version: 2.5
Name: ragfold
Version: 0.1.0
Summary: Unified interface for RAG and information-extraction engines with built-in benchmarks.
Project-URL: Homepage, https://github.com/mihailorama/ragfold
Project-URL: Documentation, https://github.com/mihailorama/ragfold/tree/main/docs
Project-URL: Repository, https://github.com/mihailorama/ragfold
Project-URL: Issues, https://github.com/mihailorama/ragfold/issues
Project-URL: Changelog, https://github.com/mihailorama/ragfold/blob/main/CHANGELOG.md
Author-email: Mihailorama <mihailorama@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: document-ai,information-extraction,rag,retrieval-augmented-generation,vector-search
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.10
Provides-Extra: adaptive-chunking
Provides-Extra: agentic-file-search
Provides-Extra: all
Requires-Dist: chromadb>=0.5; extra == 'all'
Requires-Dist: cocoindex>=0.1; extra == 'all'
Requires-Dist: cohere>=5.0; extra == 'all'
Requires-Dist: colpali-engine>=0.3; extra == 'all'
Requires-Dist: faiss-cpu>=1.8; extra == 'all'
Requires-Dist: haystack-ai>=2.0; extra == 'all'
Requires-Dist: headroom-ai>=0.1; extra == 'all'
Requires-Dist: lightrag-hku>=1.0; extra == 'all'
Requires-Dist: llama-index-core>=0.10; extra == 'all'
Requires-Dist: openai>=1.0; extra == 'all'
Requires-Dist: pgvector>=0.2; extra == 'all'
Requires-Dist: promptfoo>=0.1; extra == 'all'
Requires-Dist: psycopg[binary]>=3.1; extra == 'all'
Requires-Dist: qdrant-client>=1.9; extra == 'all'
Requires-Dist: raganything>=1.0; extra == 'all'
Requires-Dist: rank-bm25>=0.2.2; extra == 'all'
Requires-Dist: sentence-transformers>=3.0; extra == 'all'
Requires-Dist: txtai>=7.0; extra == 'all'
Requires-Dist: voyageai>=0.2; extra == 'all'
Provides-Extra: bm25
Requires-Dist: rank-bm25>=0.2.2; extra == 'bm25'
Provides-Extra: chroma
Requires-Dist: chromadb>=0.5; extra == 'chroma'
Provides-Extra: cocoindex
Requires-Dist: cocoindex>=0.1; extra == 'cocoindex'
Provides-Extra: cohere
Requires-Dist: cohere>=5.0; extra == 'cohere'
Provides-Extra: colpali
Requires-Dist: colpali-engine>=0.3; extra == 'colpali'
Provides-Extra: colqwen2
Requires-Dist: colpali-engine>=0.3; extra == 'colqwen2'
Provides-Extra: dev
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: dse
Provides-Extra: evaluation
Provides-Extra: faiss
Requires-Dist: faiss-cpu>=1.8; extra == 'faiss'
Provides-Extra: haystack
Requires-Dist: haystack-ai>=2.0; extra == 'haystack'
Provides-Extra: headroom
Requires-Dist: headroom-ai>=0.1; extra == 'headroom'
Provides-Extra: lightrag
Requires-Dist: lightrag-hku>=1.0; extra == 'lightrag'
Provides-Extra: llamaindex
Requires-Dist: llama-index-core>=0.10; extra == 'llamaindex'
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == 'openai'
Provides-Extra: pgvector
Requires-Dist: pgvector>=0.2; extra == 'pgvector'
Requires-Dist: psycopg[binary]>=3.1; extra == 'pgvector'
Provides-Extra: pixelrag
Provides-Extra: promptfoo
Requires-Dist: promptfoo>=0.1; extra == 'promptfoo'
Provides-Extra: qdrant
Requires-Dist: qdrant-client>=1.9; extra == 'qdrant'
Provides-Extra: rag-anything
Requires-Dist: raganything>=1.0; extra == 'rag-anything'
Provides-Extra: rerank-cohere
Requires-Dist: cohere>=5.0; extra == 'rerank-cohere'
Provides-Extra: rerank-cross-encoder
Requires-Dist: sentence-transformers>=3.0; extra == 'rerank-cross-encoder'
Provides-Extra: sentence-transformers
Requires-Dist: sentence-transformers>=3.0; extra == 'sentence-transformers'
Provides-Extra: test
Requires-Dist: mypy>=1.8; extra == 'test'
Requires-Dist: pytest-asyncio>=0.21; extra == 'test'
Requires-Dist: pytest>=7.0; extra == 'test'
Requires-Dist: ruff>=0.4; extra == 'test'
Provides-Extra: txtai
Requires-Dist: txtai>=7.0; extra == 'txtai'
Provides-Extra: voyage
Requires-Dist: voyageai>=0.2; extra == 'voyage'
Description-Content-Type: text/markdown

# Ragfold

[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![CI](https://github.com/mihailorama/ragfold/actions/workflows/ci.yml/badge.svg)](https://github.com/mihailorama/ragfold/actions/workflows/ci.yml)

**Compare RAG and information-extraction engines behind one interface.** Ragfold
does for retrieval/extraction tools what `docfold` does for document parsers:
standard adapters, clean fallbacks, built-in benchmarks, and a light default CI
that needs no GPU, model downloads, API keys, or network.

```bash
pip install -e ".[dev]"
ragfold list-engines
ragfold compare examples/corpus.json examples/queries.json --engines text-rag,bm25 --top-k 3
```

## Engine Comparison

> Research-based estimates from public docs, package metadata, and adapter
> behavior. See [docs/benchmarks.md](docs/benchmarks.md). Costs are estimates,
> not procurement quotes.

| Engine | ragfold | Modality | OCR-free | Type | License | Retrieval method | Rerank | Speed | Cost |
|---|:---:|---|:---:|---|---|---|:---:|---|---|
| `text-rag` | yes | text | yes | local | MIT | lexical TF-IDF | no | fast | free |
| `bm25` | yes | text | yes | local | MIT adapter / optional package license | lexical BM25 | no | fast | free |
| `sentence-transformers` | gated | text | yes | local | Apache-2.0 library, model-dependent | dense single-vector | optional | medium | free infra cost |
| `openai` | gated | text | yes | SaaS | commercial terms | dense single-vector | no | fast | paid tokens |
| `cohere-embed` | gated | text | yes | SaaS | commercial terms | dense single-vector | optional via Cohere Rerank | fast | paid tokens/search |
| `voyage` | gated | text | yes | SaaS | commercial terms | dense single-vector | no | fast | paid tokens |
| `colpali` | gated | visual | yes | local VLM | code/model-dependent | late-interaction | no | slow | free infra cost |
| `colqwen2` | gated | visual | yes | local VLM | code/model-dependent | late-interaction | no | slow | free infra cost |
| `pixelrag` | gated | visual | yes | local/VLM | research/model-dependent | visual embedding | no | slow | free infra cost |
| `dse` | gated | visual | yes | local | research/model-dependent | screenshot embedding | no | medium | free infra cost |
| `lightrag` | gated | text | yes | local/framework | MIT | graph-hybrid RAG | framework-dependent | medium | infra/provider-dependent |
| `rag-anything` | gated | multimodal | no | local/VLM framework | MIT | multimodal document RAG | framework-dependent | slow | infra/provider-dependent |
| `agentic-file-search` | gated | text/documents | no | SaaS/agentic | project-dependent | agentic file search | no | slow | paid tokens |
| `llamaindex` | gated | text | yes | framework | framework-dependent | framework retriever | framework-dependent | varies | varies |
| `haystack` | gated | text | yes | framework | framework-dependent | framework retriever | framework-dependent | varies | varies |
| `txtai` | gated | text | yes | framework | framework-dependent | framework retriever | framework-dependent | varies | varies |

`text-rag` and `bm25` run in core. Every model, cloud, GPU, visual, service, and
framework adapter is optional and reports `available=no` until its extra,
credentials, and runtime are configured.

## How to Choose

| Situation | Start with |
|---|---|
| Need a no-dependency baseline for CI or unit tests | `text-rag` |
| Keyword-heavy corpora, legal clauses, IDs, exact phrases | `bm25` |
| Semantic text retrieval on local hardware | `sentence-transformers` |
| Managed embeddings with low operational overhead | `openai`, `cohere-embed`, or `voyage` |
| Need reranking after first-stage retrieval | Cross-encoder or Cohere Rerank adapters |
| PDFs where layout matters and OCR should be avoided | `colpali` or `colqwen2` |
| Screenshot/page-image retrieval experiments | `pixelrag` or `dse` |
| Graph/hybrid RAG experiment from the LightRAG family | `lightrag` |
| Multimodal document RAG over text, images, tables, equations | `rag-anything` |
| Dynamic file exploration with citations instead of pre-built vectors | `agentic-file-search` |
| Better chunks before retrieval | `RecursiveChunker` or `AdaptiveChunker` |
| Compress retrieved passages before sending to an LLM | `NoopCompressor` or `HeadroomCompressor` |
| Already invested in a RAG framework | `llamaindex`, `haystack`, or `txtai` |
| Need persistent vector search | FAISS, Qdrant, Chroma, or pgvector vector-store extras |

## Why Ragfold

| Challenge | Without Ragfold | With Ragfold |
|---|---|---|
| Try a new retriever | Rewrite indexing and result parsing | Swap an engine name |
| Keep CI light | Mock the whole RAG layer | Core lexical engines run offline |
| Compare quality | Build ad hoc notebooks | `ragfold compare` gives a Markdown report |
| Track costs | Token math lives in spreadsheets | Reports include token-cost columns |
| Handle missing extras | Import failures crash sweeps | Unavailable engines list and skip cleanly |
| Batch queries | Write concurrency glue | `EngineRouter.process_batch(..., concurrency=N)` |

## Install Extras

| Extra | Installs | Use when |
|---|---|---|
| `bm25` | `rank-bm25` | You want the external BM25 implementation instead of pure Python fallback |
| `sentence-transformers` | local embedding/reranking stack | You can download/load local models |
| `openai` | OpenAI SDK | You have `OPENAI_API_KEY` and want OpenAI embeddings |
| `cohere` | Cohere SDK | You have `COHERE_API_KEY` for Embed/Rerank |
| `voyage` | Voyage AI SDK | You have `VOYAGE_API_KEY` |
| `colpali`, `colqwen2` | ColPali engine stack | You can run visual document retrievers |
| `lightrag` | LightRAG package | You want HKUDS LightRAG as a gated framework retriever |
| `rag-anything` | RAG-Anything package | You want multimodal document RAG experiments |
| `agentic-file-search` | source-installed agentic search package | You want tool-using document search |
| `adaptive-chunking` | adaptive chunking source/package | You want per-document chunking strategy selection |
| `headroom` | Headroom compression package | You want context compression after retrieval |
| `cocoindex` | CocoIndex package | You want incremental indexing/corpus preparation |
| `promptfoo` | Promptfoo package | You want benchmark export for LLM/RAG eval workflows |
| `faiss`, `qdrant`, `chroma`, `pgvector` | vector-store clients | You need persistent or accelerated vector search |
| `llamaindex`, `haystack`, `txtai` | framework clients | You want thin wrappers over existing retrievers |
| `dev` | pytest, ruff, mypy | Local development and CI |

Examples:

```bash
pip install ragfold
pip install "ragfold[bm25,sentence-transformers,faiss]"
pip install "ragfold[openai,cohere,voyage]"
pip install "ragfold[lightrag,rag-anything,adaptive-chunking,headroom]"
```

## Python API

```python
import asyncio

from ragfold import EngineRouter
from ragfold.engines.bm25 import BM25Engine
from ragfold.engines.text_rag import TextRagEngine


async def main():
    corpus = [
        {"id": "policy", "text": "Refunds are available for 30 days."},
        {"id": "security", "text": "Accounts require multi-factor authentication."},
    ]
    router = EngineRouter([BM25Engine(), TextRagEngine()])

    result = await router.retrieve(corpus, "refund policy", engine_hint="bm25", top_k=1)
    print(result.passages[0].document_id)

    comparison = await router.compare(
        corpus,
        [{"id": "q1", "query": "refund policy", "relevant_ids": ["policy"]}],
        top_k=1,
    )
    print(comparison.keys())


asyncio.run(main())
```

Optional LightRAG runtime, with provider/model functions configured by the
caller:

```python
from lightrag.llm.openai import gpt_4o_mini_complete, openai_embed

from ragfold.engines.github_rag import LightRAGEngine


engine = LightRAGEngine(
    working_dir="./rag_storage",
    llm_model_func=gpt_4o_mini_complete,
    embedding_func=openai_embed,
)
```

## CLI

```bash
ragfold list-engines
ragfold compare examples/corpus.json examples/queries.json --engines text-rag,bm25 --top-k 1
ragfold bench examples --engines text-rag,bm25
```

`corpus.json` is a list of objects with `id` and `text`. `queries.json` is a list
of objects with `id`, `query`, `relevant_ids`, and optional `answers`.

## Evaluation

Ragfold reports retrieval and answer metrics using the same convention as
docfold: predicted value first, reference value second, and higher is better.

| Metric | What it measures |
|---|---|
| Recall@k | Share of gold documents found in top-k |
| Precision@k | Share of top-k results that are gold |
| Hit@k | Whether any gold document appears in top-k |
| MRR | Reciprocal rank of first relevant result |
| nDCG@k | Ranking quality with early hits rewarded |
| MAP | Mean average precision over queries |
| Answer EM/F1 | End-to-end exact match and token overlap when gold answers exist |
| Token cost | Provider-reported or adapter-estimated cost, reported separately |

Reference-free faithfulness is intentionally a gated `slow` hook because it
requires an LLM judge.

## Pipeline Stages

`EngineRouter` can apply optional lifecycle stages consistently across
`retrieve()`, `compare()`, and `process_batch()`:

```python
from ragfold import EngineRouter
from ragfold.compression import NoopCompressor
from ragfold.engines.text_rag import TextRagEngine
from ragfold.preprocessing import RecursiveChunker

router = EngineRouter(
    [TextRagEngine()],
    chunker=RecursiveChunker(chunk_size=300, chunk_overlap=50),
    compressor=NoopCompressor(),
)
```

Heavy integrations such as `AdaptiveChunker` and `HeadroomCompressor` stay gated
behind optional extras and injected runtime objects.

## Architecture

```
Corpus + Queries
      |
      v
EngineRouter  -- optional reranker --> RetrievalResult / RagAnswer
      |
      +-- lexical: text-rag, bm25
      +-- dense: sentence-transformers, OpenAI, Cohere, Voyage
      +-- visual: ColPali, ColQwen2, PixelRAG, DSE
      +-- github RAG forks: LightRAG, RAG-Anything, agentic-file-search
      +-- framework: LlamaIndex, Haystack, txtai
      +-- vector stores: in-memory, FAISS, Qdrant, Chroma, pgvector
      +-- stages: chunkers, indexers, context compressors, eval exporters
```

## Development

```bash
pip install -e ".[dev]"
pytest -m "not slow"
```

All heavy/cloud/GPU/model tests must be marked `slow` and must not run in the
default CI path.
