Metadata-Version: 2.4
Name: raglan-retrieval
Version: 0.1.0
Summary: A lightweight, highly configurable RAG retrieval engine. Framework-free, protocol-driven.
Project-URL: Homepage, https://github.com/Gushuchun/RAGLAN
Project-URL: Documentation, https://github.com/Gushuchun/RAGLAN/tree/master/docs
Project-URL: Repository, https://github.com/Gushuchun/RAGLAN
Project-URL: Issues, https://github.com/Gushuchun/RAGLAN/issues
Project-URL: Changelog, https://github.com/Gushuchun/RAGLAN/blob/master/CHANGELOG.md
Author: Gushuchun
License: MIT
License-File: LICENSE
Keywords: bm25,llm,rag,reranker,retrieval,search,vector-search
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.10
Provides-Extra: all
Requires-Dist: asyncpg>=0.29.0; extra == 'all'
Requires-Dist: chromadb>=0.5.0; extra == 'all'
Requires-Dist: cohere>=5.0.0; extra == 'all'
Requires-Dist: dashscope>=1.17.0; extra == 'all'
Requires-Dist: litellm>=1.0.0; extra == 'all'
Requires-Dist: openai>=1.0.0; extra == 'all'
Requires-Dist: qdrant-client>=1.13.0; extra == 'all'
Requires-Dist: sentence-transformers>=3.0.0; extra == 'all'
Requires-Dist: tiktoken>=0.5; extra == 'all'
Provides-Extra: chromadb
Requires-Dist: chromadb>=0.5.0; extra == 'chromadb'
Provides-Extra: cohere
Requires-Dist: cohere>=5.0.0; extra == 'cohere'
Provides-Extra: dashscope
Requires-Dist: dashscope>=1.17.0; extra == 'dashscope'
Provides-Extra: dev
Requires-Dist: build>=1.0; extra == 'dev'
Requires-Dist: hypothesis>=6.100; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: numpy; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-xdist>=3.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Provides-Extra: huggingface
Requires-Dist: sentence-transformers>=3.0.0; extra == 'huggingface'
Provides-Extra: litellm
Requires-Dist: litellm>=1.0.0; extra == 'litellm'
Provides-Extra: openai
Requires-Dist: openai>=1.0.0; extra == 'openai'
Provides-Extra: pgvector
Requires-Dist: asyncpg>=0.29.0; extra == 'pgvector'
Provides-Extra: qdrant
Requires-Dist: qdrant-client>=1.13.0; extra == 'qdrant'
Provides-Extra: tiktoken
Requires-Dist: tiktoken>=0.5; extra == 'tiktoken'
Description-Content-Type: text/markdown

# Raglan

<p align="center">
  <a href="https://pypi.org/project/raglan-retrieval/"><img src="https://img.shields.io/pypi/v/raglan.svg" alt="PyPI"></a>
  <a href="https://github.com/Gushuchun/RAGLAN/actions"><img src="https://img.shields.io/github/actions/workflow/status/Gushuchun/RAGLAN/ci.yml?branch=master" alt="CI"></a>
  <a href="https://github.com/Gushuchun/RAGLAN/actions"><img src="https://img.shields.io/badge/coverage-≥85%25-green" alt="Coverage"></a>
  <a href="https://github.com/Gushuchun/RAGLAN/blob/master/LICENSE"><img src="https://img.shields.io/github/license/Gushuchun/RAGLAN" alt="License"></a>
  <a href="https://pypi.org/project/raglan-retrieval/"><img src="https://img.shields.io/badge/python-≥3.10-blue" alt="Python"></a>
</p>

A lightweight, highly configurable RAG retrieval engine. Framework-free, protocol-driven.

## Why Raglan?

Standard RAG has one step: vector search → top-K results. Real-world retrieval needs more:

| Problem | Standard RAG | Raglan |
|---------|-------------|--------|
| Single query misses aspects | "return policy" finds policy text, misses refund flow | Auto-generates 3 query variants, searches in parallel |
| Dense-only retrieval | Poor short-keyword / jargon matching | BM25 sparse + vector dense, RRF hybrid fusion |
| Small chunks lack context | "refund takes 7 days" without surrounding doc | Child-chunk match → expanded to full parent context |
| Vector similarity != semantic match | "how to return" vs "I don't want this" look close | Cross-Encoder pair-wise reranking filters false positives |
| One failure kills the pipeline | Embedding service timeout → empty results | Each stage degrades independently, rest continues |

## Six-Stage Pipeline

```
User query
  │
  ▼
┌─────────────────────────────────┐
│ Stage 1: QueryExpander           │  ← LLM generates entity extraction + 3 variants
└──────────────┬──────────────────┘
               │ original + variants = parallel search
               ▼
┌─────────────────────────────────┐
│ Stage 2: Retrievers (parallel)   │
│  · Dense: pgvector / Qdrant / ChromaDB / ...
│  · Sparse: BM25 full-text        │
└──────────────┬──────────────────┘
               │ multi-source results
               ▼
┌─────────────────────────────────┐
│ Stage 3: Fusion                  │  ← RRF / Weighted / RoundRobin
└──────────────┬──────────────────┘
               │ fused candidates
               ▼
┌─────────────────────────────────┐
│ Stage 4: Reranker (optional)     │  ← Cross-Encoder / Cohere Rerank
│ Filters scores below threshold   │
└──────────────┬──────────────────┘
               │ top-N candidates
               ▼
┌─────────────────────────────────┐
│ Stage 5: ContextBuilder          │  ← Parent expansion / window / passthrough
│ Greedy fill up to max_tokens     │
└──────────────┬──────────────────┘
               │
               ▼
         Final Top-N results
```

## Installation

```bash
pip install raglan-retrieval
```

With optional providers:

```bash
pip install raglan-retrieval[openai]        # OpenAI embedder + expander
pip install raglan-retrieval[pgvector]      # Postgres + pgvector retriever
pip install raglan-retrieval[huggingface]   # HF embedder + Cross-Encoder reranker
pip install raglan-retrieval[qdrant]        # Qdrant retriever
pip install raglan-retrieval[chromadb]      # ChromaDB retriever
pip install raglan-retrieval[cohere]        # Cohere reranker
pip install raglan-retrieval[dashscope]     # Alibaba DashScope embedder
pip install raglan-retrieval[litellm]       # LiteLLM multi-provider expander
pip install raglan-retrieval[all]           # Everything
```

## 5-Minute Quickstart

```python
from raglan import Raglan
from raglan.retrievers import BM25Retriever
import asyncio


async def main():
    # Step 1: Set up a retriever with your data
    bm25 = BM25Retriever()

    async def chunks():
        yield [
            ("doc1", "Return policy: items can be returned within 30 days.", None),
            ("doc2", "Refund process: refunds are issued to the original payment method.", None),
            ("doc3", "Shipping: orders ship within 2 business days.", None),
        ]

    await bm25.index(chunks())

    # Step 2: Build the pipeline (all stages optional beyond retrievers)
    rag = Raglan.builder().with_retrievers([bm25]).build()

    # Step 3: Search
    results, trace = await rag.search("how to return my order")

    for r in results:
        print(f"[{r.score:.3f}] {r.content}")

    print(f"Pipeline took {trace.total_ms:.1f}ms")


asyncio.run(main())
```

### Adding vector search and reranking

```python
from raglan.context_builders import ParentExpander
from raglan.embedders import OpenAIEmbedder
from raglan.expanders import OpenAIExpander
from raglan.rerankers import CrossEncoderReranker
from raglan.retrievers import ConfigurablePgvectorRetriever

rag = (
    Raglan.builder()
    .with_expander(OpenAIExpander(model="gpt-4o-mini"))
    .with_embedder(OpenAIEmbedder(model="text-embedding-3-small"))
    .with_retrievers(
        [
            ConfigurablePgvectorRetriever(
                connection_string="postgresql://...",
                table="kb.chunks",
                embedding_column="embedding",
                parent_id_column="parent_id",
            ),
            BM25Retriever(),
        ]
    )
    .with_reranker(
        CrossEncoderReranker(
            model_name="ms-marco-TinyBERT-L2-v2",
            min_score=0.5,
        )
    )
    .with_context_builder(
        ParentExpander(
            loader=my_parent_loader,
            max_tokens=6000,
        )
    )
    .with_fallback_mode("degrade")
    .build()
)

results, trace = await rag.search("damaged order return policy")
```

## Design Philosophy

1. **Framework-Free** — No dependency on LangChain, LlamaIndex, or any specific vector database or LLM provider. Protocols define the interfaces; you provide the implementations.

2. **Graceful Degradation** — Every stage is independent. Query expansion fails? Use the original query. Cross-Encoder not installed? Skip reranking. BM25 unavailable? Pure vector search still works.

3. **Fully Configurable** — Every stage's parameters, weights, models, and thresholds are configurable via the Builder or `from_dict()`.

4. **Production-Ready** — Extracted from production systems processing thousands of support emails daily. Handles extreme text, multilingual, and high-concurrency scenarios.

## Comparison with Existing RAG Frameworks

| Tool | Positioning | vs Raglan |
|------|------------|-----------|
| **LangChain RAG** | Full-stack LLM framework's RAG module | LC binds to LangChain ecosystem; Raglan is zero-dependency, direct use |
| **LlamaIndex** | Data→LLM full pipeline | LI has many concepts (Node, Index, QueryEngine); Raglan has one: Search |
| **RAGatouille** | ColBERT-specific RAG | RAGatouille focuses on ColBERT; Raglan is general retrieval + reranking |
| **Cohere Rerank** | Commercial API reranking | Cohere charges, sends data to cloud; Raglan runs locally |
| **rerankers (answer.ai)** | Unified reranking API | rerankers only does reranking; Raglan covers retrieval→fusion→reranking |

## Project Structure

```
raglan/
├── README.md
├── docs/
│   ├── architecture.md      # Architecture design
│   ├── configuration.md     # Full configuration reference
│   ├── pipeline.md          # Six-stage deep dive
│   └── examples.md          # Multi-scenario examples
├── raglan/
│   ├── __init__.py
│   ├── raglan.py            # Raglan facade + Builder
│   ├── pipeline.py          # Pipeline engine + stage dispatch
│   ├── protocols.py         # User-implementable abstract interfaces
│   ├── types.py             # ScoredChunk, SearchResult, Filter, etc.
│   ├── exceptions.py        # Exception hierarchy
│   ├── observability.py     # Metrics collector
│   ├── expanders/           # Query expansion (OpenAI, LiteLLM, Identity)
│   ├── embedders/           # Embedding (OpenAI, HuggingFace, DashScope)
│   ├── retrievers/          # Search backends (BM25, pgvector, Qdrant, ChromaDB)
│   ├── fusion/              # Result fusion (RRF, Weighted, RoundRobin)
│   ├── rerankers/           # Reranking (CrossEncoder, Cohere)
│   ├── context_builders/    # Context assembly (Parent, Window, Passthrough)
│   ├── middleware/           # Timeout, Retry, CircuitBreaker, Logging
│   └── resilience/          # RateLimiter, RetryBudget, HealthChecker
└── tests/
```

## License

MIT
