Metadata-Version: 2.4
Name: slm-rag
Version: 0.1.3
Summary: A lightweight, CPU-optimized RAG library powered by a local Small Language Model (SLM)
Home-page: https://github.com/SuryaprakashCV/SLM-RAG
Author: Suryaprakash CV
Author-email: Suryaprakash CV <suryaprakash.c.v@gmail.com>
Project-URL: Homepage, https://github.com/SuryaprakashCV/SLM-RAG
Project-URL: Bug Tracker, https://github.com/SuryaprakashCV/SLM-RAG/issues
Project-URL: Source Code, https://github.com/SuryaprakashCV/SLM-RAG
Keywords: llm,slm,rag,retrieval-augmented-generation,qwen,llama-cpp,local-ai,cpu-inference,knowledge-retrieval,semantic-search
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: onnxruntime-genai>=0.2.0
Requires-Dist: huggingface_hub>=0.10.0
Requires-Dist: pyyaml
Dynamic: author
Dynamic: home-page
Dynamic: license-file

# SLM RAG

`slm_rag` is a lightweight, local Retrieval-Augmented Generation (RAG) library powered entirely by a Small Language Model (SLM) running on CPU. It allows developers to pass a list of document chunks, a user question, and arbitrary guidelines/instructions to answer queries locally with high privacy, low resource usage, and zero API costs.

---

## Key Features

- **Local & Private**: Runs completely on CPU / RAM. Zero API keys, zero network latency, and complete data privacy.
- **Resource Efficient**: Uses a 1.5B parameter model (`Qwen 2.5 1.5B Instruct ONNX`), consuming only **1.5 GB to 2.0 GB of RAM** and taking **1.1 GB of disk storage**.
- **Instruction Adherence**: Formats instructions directly into the system template to enforce constraints (e.g. style, safety, or formatting constraints like JSON).
- **Streaming Support**: Stream token-by-token output in real-time via a Python generator.
- **Agentic Tool Use**: Optional ReAct loop support. Pass custom tools (like Vector DB search) for the RAG agent to autonomously fetch missing context before answering.

---

## Installation

Install directly via `pip`:

```bash
pip install slm-rag
```

Or install locally for development:

```bash
# 1. Create a fresh virtual environment
python3 -m venv .venv
source .venv/bin/activate

# 2. Install the package in editable mode
pip install -e .
```

*Note: Requires `onnxruntime-genai`, `huggingface_hub`, and `pyyaml`.*

---

## Quick Start

```python
from slm_rag import SLMRag

# Initialize the RAG engine (auto-locates or downloads the model)
rag = SLMRag()

# Provide context chunks
chunks = [
    "NebulaCorp was founded in 2024 by Dr. Helena Vance. It specializes in quantum-resistant encryption algorithms.",
    "The flagship product of NebulaCorp is called 'AegisShield'. It is widely used by financial organizations.",
    "In early 2026, NebulaCorp announced a partnership with the European Space Agency."
]

# Run query with a strict instruction
answer = rag.answer(
    chunks=chunks,
    question="What is their flagship product?",
    instruction="Answer like a 17th-century pirate.",
    temperature=0.0
)

print(answer)
# Output: "Ahoy matey! AegisShield be the flagship product of NebulaCorp, savvy?"
```

### Streaming Example

```python
from slm_rag import SLMRag

rag = SLMRag()

chunks = [
    "The James Webb Space Telescope was launched on December 25, 2021.",
    "It is the largest and most powerful space telescope ever built.",
    "Its primary mirror is 6.5 meters in diameter, composed of 18 hexagonal gold-coated segments."
]

# Stream tokens as they are generated
for token in rag.answer(
    chunks=chunks,
    question="What is special about the James Webb Space Telescope?",
    instruction="Answer concisely in one paragraph.",
    stream=True
):
    print(token, end="", flush=True)
print()
```

---

## Configuration API

```python
SLMRag(
    model_path=None,   # Explicit path to an ONNX model directory (optional)
    cache_dir=None,    # Cache directory for auto-downloads
    n_ctx=8192,        # Context window size (default: 8192)
    n_threads=4        # Number of CPU threads (default: 4)
)
```

### Answering Queries

```python
rag.answer(
    chunks: list[str],              # Document text chunks
    question: str,                  # User query / question
    instruction: str,               # Instruction or constraint the model must follow
    temperature: float = 0.0,       # Generation temperature (0.0 for deterministic answers)
    max_tokens: int = 256,          # Maximum token limit for the response
    tools: list = None,             # Optional JSON schemas for tool use
    tool_executor: callable = None, # Optional callback function to execute tools
    max_iterations: int = 5,        # Max ReAct tool execution loops
    stream: bool = False            # If True, returns a generator that yields token strings
)
```

---

## Environment Variables

All constructor parameters can be overridden via environment variables:

| Variable | Description | Default |
|---|---|---|
| `SLM_RAG_CONFIG` | Path to a custom `config.yaml` file | — |
| `SLM_RAG_CACHE_DIR` | Override model download/cache directory | — |
| `SLM_RAG_N_THREADS` | Number of CPU threads | `4` |
| `SLM_RAG_N_CTX` | Context window size | `8192` |
| `SLM_RAG_MAX_TOKENS` | Default max tokens per answer | `256` |

---

## License

Apache License 2.0.
