Metadata-Version: 2.4
Name: iris-vector-rag
Version: 0.11.1
Summary: RAG pipelines with InterSystems IRIS as the vector database. Six strategies (basic, rerank, CRAG, GraphRAG, multi-query RRF, ColBERT), unified API, RAGAS evaluation.
Author-email: Thomas Dyar <thomas.dyar@intersystems.com>
Maintainer-email: Thomas Dyar <thomas.dyar@intersystems.com>
License: MIT
Project-URL: Homepage, https://github.com/intersystems-community/iris-vector-rag
Project-URL: Documentation, https://github.com/intersystems-community/iris-vector-rag/tree/main/docs
Project-URL: Repository, https://github.com/intersystems-community/iris-vector-rag
Project-URL: Bug Tracker, https://github.com/intersystems-community/iris-vector-rag/issues
Project-URL: Changelog, https://github.com/intersystems-community/iris-vector-rag/blob/main/CHANGELOG.md
Keywords: rag,vector-search,langchain,ragas,enterprise-ai,iris,intersystems,retrieval-augmented-generation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Database
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: openai>=1.0.0
Requires-Dist: anthropic>=0.25.0
Requires-Dist: sentence-transformers>=5.1.1
Requires-Dist: transformers>=4.41.0
Requires-Dist: numpy>=1.21.0
Requires-Dist: pandas>=1.3.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: requests>=2.28.0
Requires-Dist: tqdm>=4.64.0
Requires-Dist: typing-extensions>=4.0.0
Requires-Dist: psutil>=7.0.0
Requires-Dist: intersystems-irispython>=5.1.2
Requires-Dist: langchain-openai>=0.3.24
Requires-Dist: langchain>=0.3.25
Requires-Dist: python-dotenv>=1.1.0
Requires-Dist: matplotlib>=3.10.3
Requires-Dist: ragas>=0.2.15
Requires-Dist: seaborn>=0.13.2
Requires-Dist: plotly>=6.1.2
Requires-Dist: jaydebeapi>=1.2.3
Requires-Dist: docker>=6.1.3
Requires-Dist: tiktoken>=0.5.0
Requires-Dist: torch>=2.0.0
Provides-Extra: embedding
Requires-Dist: torch>=2.0.0; extra == "embedding"
Requires-Dist: sentence-transformers>=5.1.1; extra == "embedding"
Requires-Dist: transformers>=4.41.0; extra == "embedding"
Requires-Dist: iris-vector-graph>=1.82.0; extra == "embedding"
Provides-Extra: hybrid-graphrag
Requires-Dist: iris-vector-graph>=1.82.0; extra == "hybrid-graphrag"
Provides-Extra: iris-llm
Provides-Extra: colbert
Requires-Dist: iris-vector-graph>=1.82.0; extra == "colbert"
Requires-Dist: intersystems-irispython>=5.1.2; extra == "colbert"
Requires-Dist: pylate>=1.4.0; extra == "colbert"
Requires-Dist: scikit-learn>=1.3.0; extra == "colbert"
Provides-Extra: dspy
Requires-Dist: dspy-ai>=2.0.0; extra == "dspy"
Provides-Extra: evaluation
Requires-Dist: scipy>=1.17.1; extra == "evaluation"
Requires-Dist: scikit-learn>=1.3.0; extra == "evaluation"
Requires-Dist: datasets>=4.8.4; extra == "evaluation"
Requires-Dist: spacy>=3.6.0; extra == "evaluation"
Requires-Dist: statsmodels>=0.14.6; extra == "evaluation"
Requires-Dist: pingouin>=0.5.3; extra == "evaluation"
Requires-Dist: jinja2>=3.1.0; extra == "evaluation"
Requires-Dist: markdown>=3.10.2; extra == "evaluation"
Requires-Dist: nltk>=3.9.4; extra == "evaluation"
Provides-Extra: dev
Requires-Dist: uv>=0.1.0; extra == "dev"
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-mock>=3.10.0; extra == "dev"
Requires-Dist: pytest-dotenv>=0.5.2; extra == "dev"
Requires-Dist: iris-devtester>=1.11.1; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: isort>=5.12.0; extra == "dev"
Requires-Dist: flake8>=6.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"
Requires-Dist: build>=0.10.0; extra == "dev"
Requires-Dist: twine>=4.0.0; extra == "dev"
Provides-Extra: mcp
Requires-Dist: mcp>=0.1.0; extra == "mcp"
Requires-Dist: fastapi>=0.100.0; extra == "mcp"
Requires-Dist: uvicorn>=0.23.0; extra == "mcp"
Requires-Dist: websockets>=11.0.0; extra == "mcp"
Provides-Extra: api
Requires-Dist: fastapi>=0.104.0; extra == "api"
Requires-Dist: uvicorn[standard]>=0.24.0; extra == "api"
Requires-Dist: python-multipart>=0.0.6; extra == "api"
Requires-Dist: websockets>=12.0; extra == "api"
Requires-Dist: redis>=5.0.0; extra == "api"
Requires-Dist: bcrypt>=4.0.0; extra == "api"
Requires-Dist: python-jose[cryptography]>=3.3.0; extra == "api"
Provides-Extra: all
Requires-Dist: uv>=0.1.0; extra == "all"
Requires-Dist: pytest>=7.0.0; extra == "all"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "all"
Requires-Dist: pytest-cov>=4.0.0; extra == "all"
Requires-Dist: iris-devtester>=1.11.1; extra == "all"
Requires-Dist: black>=23.0.0; extra == "all"
Requires-Dist: isort>=5.12.0; extra == "all"
Requires-Dist: flake8>=6.0.0; extra == "all"
Requires-Dist: mypy>=1.0.0; extra == "all"
Requires-Dist: pre-commit>=3.0.0; extra == "all"
Requires-Dist: build>=0.10.0; extra == "all"
Requires-Dist: twine>=4.0.0; extra == "all"
Requires-Dist: mcp>=0.1.0; extra == "all"
Requires-Dist: fastapi>=0.104.0; extra == "all"
Requires-Dist: uvicorn[standard]>=0.24.0; extra == "all"
Requires-Dist: websockets>=12.0; extra == "all"
Requires-Dist: python-multipart>=0.0.6; extra == "all"
Requires-Dist: redis>=5.0.0; extra == "all"
Requires-Dist: bcrypt>=4.0.0; extra == "all"
Requires-Dist: python-jose[cryptography]>=3.3.0; extra == "all"
Requires-Dist: dspy-ai>=2.0.0; extra == "all"
Requires-Dist: scipy>=1.17.1; extra == "all"
Requires-Dist: scikit-learn>=1.3.0; extra == "all"
Requires-Dist: datasets>=4.8.4; extra == "all"
Requires-Dist: statsmodels>=0.14.6; extra == "all"
Requires-Dist: tqdm>=4.64.0; extra == "all"
Dynamic: license-file

# IRIS Vector RAG

Production-ready RAG (Retrieval-Augmented Generation) pipelines powered by InterSystems IRIS vector search.

**Author:** Thomas Dyar (thomas.dyar@intersystems.com)

## Quick Start

```bash
# 1. Clone and install
git clone https://github.com/intersystems-community/iris-vector-rag.git
cd iris-vector-rag
pip install -e .

# 2. Start IRIS
docker compose up -d

# 3. Configure
cp .env.example .env
# Edit .env — add your OPENAI_API_KEY

# 4. Query
python -c "
from iris_vector_rag import create_pipeline
from iris_vector_rag.core.models import Document

pipeline = create_pipeline('basic')
pipeline.load_documents(documents=[
    Document(page_content='RAG combines retrieval with generation for accurate AI.', metadata={'source': 'intro.pdf'}),
    Document(page_content='Vector search finds similar content using embeddings.', metadata={'source': 'vectors.pdf'}),
])
result = pipeline.query('What is RAG?', top_k=5, generate_answer=True)
print(result['answer'])
"
```

## Pipelines

All pipelines share the same interface — switch with one line:

```python
from iris_vector_rag import create_pipeline

pipeline = create_pipeline('basic')           # Vector similarity search
pipeline = create_pipeline('basic_rerank')    # + cross-encoder reranking
pipeline = create_pipeline('crag')            # + self-correction + web fallback
pipeline = create_pipeline('graphrag')        # + knowledge graph + entity reasoning
pipeline = create_pipeline('multi_query_rrf') # + query expansion + rank fusion
pipeline = create_pipeline('pylate_colbert')  # + ColBERT late interaction
```

| Pipeline | Method | Best For |
|----------|--------|----------|
| `basic` | Vector similarity | General Q&A, getting started |
| `basic_rerank` | Vector + reranking | Higher accuracy, medical/legal |
| `crag` | Vector + evaluation + web | Fact-checking, current events |
| `graphrag` | Vector + text + graph + RRF | Complex relationships, research |
| `multi_query_rrf` | Query expansion + fusion | Comprehensive coverage |
| `pylate_colbert` | ColBERT embeddings | Fine-grained matching |

## Response Format

All pipelines return the same structure (LangChain/RAGAS compatible):

```python
result = pipeline.query("What is diabetes?", top_k=5)

result["answer"]                # LLM-generated answer
result["retrieved_documents"]   # List[Document]
result["contexts"]              # List[str] — for RAGAS evaluation
result["sources"]               # Source citations
result["metadata"]              # Timing, pipeline type, method used
```

## Configuration

Environment variables (loaded automatically from `.env`):

```
OPENAI_API_KEY=sk-...          # Required for answer generation
IRIS_HOST=localhost             # IRIS SuperServer host
IRIS_PORT=1972                  # IRIS SuperServer port
IRIS_NAMESPACE=USER             # IRIS namespace
IRIS_USERNAME=_SYSTEM           # IRIS username
IRIS_PASSWORD=SYS               # IRIS password
```

## Evaluate with RAGAS

Compare pipelines side-by-side using real RAGAS metrics:

```bash
python examples/compare_pipelines.py --pipelines basic,basic_rerank
```

Or in code:

```python
from iris_vector_rag import create_pipeline
from ragas import evaluate, EvaluationDataset, SingleTurnSample
from ragas.metrics import faithfulness, context_precision, context_recall

pipeline = create_pipeline('basic')
pipeline.load_documents(documents=docs)
result = pipeline.query("What is diabetes?", top_k=3, generate_answer=True)

sample = SingleTurnSample(
    user_input="What is diabetes?",
    response=result["answer"],
    retrieved_contexts=result["contexts"],
    reference="Diabetes is a chronic condition...",
)
scores = evaluate(EvaluationDataset(samples=[sample]),
                  metrics=[faithfulness, context_precision, context_recall])
```

## Optional Extras

```bash
pip install iris-vector-rag[colbert]     # ColBERT/PyLate support
pip install iris-vector-rag[dspy]        # DSPy prompt optimization
pip install iris-vector-rag[evaluation]  # RAGAS evaluation framework
pip install iris-vector-rag[api]         # REST API server (FastAPI + Redis)
```

## MCP Server

Use as an AI tool server via Model Context Protocol:

```bash
docker build -f Dockerfile.mcp -t iris-vector-rag-mcp .
docker run -p 3000:3000 --env-file .env iris-vector-rag-mcp
```

## Development

```bash
pip install -e ".[dspy,evaluation]"
pytest tests/unit/                    # Fast, no IRIS needed
pytest tests/unit/ tests/contract/    # Full suite, needs IRIS running
```

## Architecture

```
iris_vector_rag/
├── pipelines/      # 6 RAG implementations (basic, crag, graphrag, etc.)
├── core/           # Base classes, models, connection management
├── storage/        # IRIS vector store, schema management
├── embeddings/     # Embedding generation and caching
├── services/       # Entity extraction, storage adapters
├── config/         # Configuration management
├── mcp/            # MCP server implementation
└── api/            # Optional REST API (FastAPI)
```

## License

MIT
