Metadata-Version: 2.4
Name: lexigram-ai-rag
Version: 0.1.2
Summary: Retrieval-Augmented Generation (RAG) pipeline for the Lexigram Framework
Project-URL: Homepage, https://github.com/dbtinoy-/lexigram
Project-URL: Repository, https://github.com/dbtinoy-/lexigram
Project-URL: Documentation, https://docs.lexigram.dev
Project-URL: Issues, https://github.com/dbtinoy-/lexigram/issues
Project-URL: Changelog, https://github.com/dbtinoy-/lexigram/blob/main/CHANGELOG.md
Author-email: Lexigram Framework Team <team@lexigram.dev>
Maintainer-email: Lexigram Framework Team <team@lexigram.dev>
License: MIT
License-File: LICENSE
Keywords: ai,async,framework,lexigram,python,rag,retrieval-augmented-generation
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: AsyncIO
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: lexigram-contracts>=0.1.0
Requires-Dist: lexigram>=0.1.1
Requires-Dist: typing-extensions>=4.0.0
Provides-Extra: all
Requires-Dist: aiohttp>=3.9; extra == 'all'
Requires-Dist: beautifulsoup4>=4.12; extra == 'all'
Requires-Dist: flashrank>=0.2.0; extra == 'all'
Requires-Dist: llmlingua>=0.2.0; extra == 'all'
Requires-Dist: pypdf2>=3.0; extra == 'all'
Requires-Dist: torch>=2.0; extra == 'all'
Requires-Dist: transformers>=4.30; extra == 'all'
Provides-Extra: clip
Requires-Dist: torch>=2.0; extra == 'clip'
Requires-Dist: transformers>=4.30; extra == 'clip'
Provides-Extra: compression
Requires-Dist: llmlingua>=0.2.0; extra == 'compression'
Provides-Extra: dev
Requires-Dist: mypy>=1.0.0; extra == 'dev'
Requires-Dist: ruff>=0.8.0; extra == 'dev'
Provides-Extra: pdf
Requires-Dist: pypdf2>=3.0; extra == 'pdf'
Provides-Extra: reranking
Requires-Dist: flashrank>=0.2.0; extra == 'reranking'
Provides-Extra: test
Requires-Dist: lexigram-testing>=0.1.1; extra == 'test'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'test'
Requires-Dist: pytest-cov>=4.0.0; extra == 'test'
Requires-Dist: pytest-mock>=3.10.0; extra == 'test'
Requires-Dist: pytest>=8.0.0; extra == 'test'
Provides-Extra: web
Requires-Dist: aiohttp>=3.9; extra == 'web'
Requires-Dist: beautifulsoup4>=4.12; extra == 'web'
Description-Content-Type: text/markdown

# lexigram-ai-rag

Retrieval-Augmented Generation (RAG) pipeline for the Lexigram Framework

---

## Overview

RAG (Retrieval-Augmented Generation) pipeline for the Lexigram Framework. Provides a multi-stage, fully configurable pipeline covering ingestion, query processing, retrieval, context optimisation, synthesis, quality assurance, and post-processing — all wired through the DI container via `RAGModule`. Zero-config usage starts with sensible defaults.


> Full documentation: [docs.lexigram.dev](https://docs.lexigram.dev)
## Install

```bash
uv add lexigram-ai-rag
# Optional extras
uv add "lexigram-ai-rag[pdf,web]"
```

## Quick Start

```python
from lexigram import Application
from lexigram.di.module import Module, module

from lexigram.ai.rag import RAGModule
from lexigram.ai.rag.config import RAGConfig

@module(imports=[
    RAGModule.configure(
        RAGConfig(
            vector_store_type="pgvector",
            collection_name="my_docs",
            top_k=5,
            enable_citations=True,
        )
    )
])
class AppModule(Module):
    pass

app = Application(modules=[AppModule])
if __name__ == "__main__":
    app.run()
```

## Configuration

> **Zero-config usage:** Call `RAGModule.configure()` with no arguments to use defaults.

### Option 1 — YAML file

```yaml
# application.yaml
ai_rag:
  vector_store_type: "pgvector"
  collection_name: "my_docs"
  top_k: 5
  embedding_model: "text-embedding-ada-002"
  enable_citations: true
```

### Option 2 — Profiles + Environment Variables *(recommended)*

```bash
export LEX_AI_RAG__VECTOR_STORE_TYPE=pgvector
# Environment variables for each field
```

### Option 3 — Python

```python
from lexigram.ai.rag.config import RAGConfig
from lexigram.ai.rag import RAGModule

config = RAGConfig(
    vector_store_type="pgvector",
    collection_name="my_docs",
    top_k=5,
)
RAGModule.configure(config)
```

### Config reference

| Field | Default | Env var | Description |
|-------|---------|---------|-------------|
| `vector_store_type` | `pgvector` | `LEX_AI_RAG__VECTOR_STORE_TYPE` | Backend: `pgvector`, `chroma`, `qdrant`, `mock` |
| `collection_name` | `default` | `LEX_AI_RAG__COLLECTION_NAME` | Collection / index name |
| `vector_dimension` | `1536` | `LEX_AI_RAG__VECTOR_DIMENSION` | Embedding dimension |
| `top_k` | `5` | `LEX_AI_RAG__TOP_K` | Documents to retrieve per query |
| `similarity_threshold` | `0.7` | `LEX_AI_RAG__SIMILARITY_THRESHOLD` | Minimum similarity score to include |
| `use_hybrid_search` | `True` | `LEX_AI_RAG__USE_HYBRID_SEARCH` | Combine semantic + keyword search |
| `embedding_provider` | `openai` | `LEX_AI_RAG__EMBEDDING_PROVIDER` | Embedding provider |
| `embedding_model` | `None` | `LEX_AI_RAG__EMBEDDING_MODEL` | Embedding model |
| `chunking_strategy` | `recursive` | `LEX_AI_RAG__CHUNKING_STRATEGY` | `recursive`, `semantic`, or `token` |
| `chunk_size` | `512` | `LEX_AI_RAG__CHUNK_SIZE` | Tokens per chunk |
| `chunk_overlap` | `50` | `LEX_AI_RAG__CHUNK_OVERLAP` | Token overlap between chunks |
| `enable_citations` | `True` | `LEX_AI_RAG__ENABLE_CITATIONS` | Include source citations in responses |
| `citation_style` | `inline` | `LEX_AI_RAG__CITATION_STYLE` | `inline`, `footnote`, or `numbered` |
| `enable_query_expansion` | `True` | `LEX_AI_RAG__ENABLE_QUERY_EXPANSION` | Expand queries before retrieval |
| `enable_hyde` | `False` | `LEX_AI_RAG__ENABLE_HYDE` | Hypothetical Document Embeddings |
| `synthesis_strategy` | `hybrid` | `LEX_AI_RAG__SYNTHESIS_STRATEGY` | `direct`, `extractive`, `abstractive`, `hybrid` |
| `tenancy.enabled` | `False` | — | Enable per-tenant pipeline isolation |

## Module Factory Methods

| Method | Description |
|--------|-------------|
| `RAGModule.configure(config)` | Production pipeline |
| `RAGModule.stub()` | In-memory / no-op pipeline for tests |

## Key Features

- **Multi-stage pipeline**: Ingestion, query processing, retrieval, context optimization, synthesis, quality assurance, post-processing
- **Chunking strategies**: recursive, semantic, token, fixed_size, sliding_window
- **Retrieval**: Vector search, BM25 keyword search, knowledge graph traversal
- **Reranking**: Cross-encoder and LLM-based rerankers
- **Synthesis**: Direct, extractive, abstractive, and hybrid synthesizers
- **HyDE support**: Hypothetical Document Embeddings for query expansion
- **Citations**: Inline, footnote, or numbered citation styles
- **Quality assurance**: Faithfulness check and hallucination detection

## Multi-Tenancy

`lexigram-ai-rag` supports per-tenant pipeline isolation. When tenancy is
enabled, the provider registers a `TenantScopedRAGPipeline` — a caching
wrapper that builds a dedicated `RAGPipelineProtocol` per tenant, with a
tenant-resolved `collection_name`.

### Configuration

```python
from lexigram.ai.rag.config import RAGConfig, RAGTenancyConfig, RAGModule

config = RAGConfig(
    tenancy=RAGTenancyConfig(enabled=True),
    collection_name="my_docs",
)
RAGModule.configure(config)
```

### Per-Tenant Collection Name

Use `RAGConfig.with_collection()` to create configs scoped to different
collection names — handy for dynamic per-tenant pipeline construction:

```python
tenant_config = RAGConfig().with_collection("tenant_a_collection")
```

### Components

| Component | Role |
|-----------|------|
| `RAGTenancyConfig` | Dataclass with `enabled` flag |
| `TenantScopedRAGPipeline` | Caches per-tenant pipelines (LRU eviction) |
| `TemplatedTenantCollectionResolver` | Resolves logical → physical collection name per tenant |

## Testing

```python
async with Application.boot(modules=[RAGModule.stub()]) as app:
    # your test code
    ...
```

## Key Source Files

| File | What it contains |
|------|-----------------|
| `src/lexigram/ai/rag/module.py` | `RAGModule.configure()` and `RAGModule.stub()` |
| `src/lexigram/ai/rag/config.py` | `RAGConfig`, `RAGTenancyConfig`, `PipelineConfig`, all stage configs |
| `src/lexigram/ai/rag/di/provider.py` | `RAGProvider` — registers pipeline and supporting services |
| `src/lexigram/ai/rag/pipeline/` | Stage executor and pipeline runner |
| `src/lexigram/ai/rag/tenancy/` | `TenantScopedRAGPipeline` factory + resolver |
| `src/lexigram/ai/rag/exceptions.py` | Full exception hierarchy |
| `src/lexigram/ai/rag/protocols.py` | Package-local protocol extensions |
| `src/lexigram/ai/rag/types.py` | RAG-specific domain types |
