Metadata-Version: 2.4
Name: dastavej-rag
Version: 0.1.0
Summary: A simple multi-user Retrieval-Augmented Generation (RAG) framework for Python.
Author: Yash Hanamghar
License-Expression: MIT
Keywords: rag,retrieval-augmented-generation,llm,qdrant,embeddings,pdf,gemini,ai,nlp,dastavej
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pymupdf>=1.24
Requires-Dist: qdrant-client>=1.12
Requires-Dist: sentence-transformers>=3.0
Requires-Dist: google-genai>=1.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Dynamic: license-file

# Dastavej RAG

A simple Python framework for building multi-user Retrieval-Augmented Generation (RAG) applications.

`dastavej-rag` provides PDF ingestion, text chunking, embeddings, Qdrant vector storage, user-isolated semantic retrieval, duplicate-document detection, source tracking, and LLM-based question answering.

## Features

- PDF ingestion with PyMuPDF
- Plain-text ingestion
- Automatic text chunking with overlap
- Sentence Transformer embeddings
- Qdrant vector storage
- Persistent local vector database
- User-isolated retrieval using `user_id`
- Duplicate PDF detection using SHA-256
- Semantic similarity search
- Gemini LLM integration
- Custom LLM support
- Source and page tracking
- Document deletion
- User-data deletion
- Structured `RAGResponse`
- Structured `Source` objects

## Installation

```bash
pip install dastavej-rag
```

## Gemini Setup

Set your Gemini API key:

```bash
export GEMINI_API_KEY="your-api-key"
```

Never commit API keys to source control.

## Quick Start

```python
import dastavej_rag as dr

rag = dr.RAG(
    llm=dr.GeminiLLM()
)

result = rag.add(
    "company_policy.pdf",
    user_id="user_123"
)

print(result)

response = rag.ask(
    "What is the leave policy?",
    user_id="user_123"
)

print(response.answer)

for source in response.sources:
    print(source.source)
    print(source.page)
    print(source.score)
```

## Add Plain Text

```python
rag.add(
    "Qdrant is a vector database.",
    user_id="user_123"
)
```

You can also use the explicit API:

```python
rag.add_text(
    text="Qdrant is a vector database.",
    user_id="user_123"
)
```

## Retrieve Chunks

```python
results = rag.retrieve(
    query="What vector database is used?",
    user_id="user_123",
    top_k=3
)

for result in results:
    print(result["text"])
    print(result["score"])
```

## Delete a Document

```python
rag.delete_document(
    document_id="document-id",
    user_id="user_123"
)
```

## Delete User Data

```python
rag.delete_user_data(
    user_id="user_123"
)
```

## Custom LLM Providers

You can implement your own provider by extending `BaseLLM`:

```python
import dastavej_rag as dr


class MyLLM(dr.BaseLLM):

    def generate(self, prompt: str) -> str:
        # Call your preferred LLM here.
        return "Generated answer"


rag = dr.RAG(
    llm=MyLLM()
)
```

This allows Dastavej RAG to work with other LLM providers.

## RAG Pipeline

```text
PDF / Text
    ↓
Text Extraction
    ↓
Chunking
    ↓
Embeddings
    ↓
Qdrant
    ↓
user_id Filtering
    ↓
Semantic Retrieval
    ↓
Context Construction
    ↓
LLM
    ↓
RAGResponse
    ├── answer
    └── sources
```

## Multi-User Applications

Every stored chunk contains a `user_id`.

Retrieval and deletion operations filter using that `user_id`, allowing applications to logically isolate documents belonging to different users.

Authentication and authorization remain the responsibility of the application using `dastavej-rag`. Applications should provide a trusted `user_id` derived from their authenticated user/session rather than trusting arbitrary client input.

## Duplicate Detection

PDF files are hashed using SHA-256.

Uploading the same PDF again for the same user returns a duplicate result instead of embedding and storing the document again.

The same document may still be independently stored for another user.

## Local Storage

By default, Qdrant data is persisted locally under:

```text
.dastavej_qdrant/
```

Add this directory to `.gitignore`.

## Development

Install development dependencies:

```bash
pip install -e ".[dev]"
```

Run tests:

```bash
python -m pytest -v
```

## Current Version

`0.1.0`

Dastavej RAG is currently an alpha release.

## License

MIT
