Metadata-Version: 2.4
Name: ragkit-rag
Version: 0.1.0
Summary: A modular, production-oriented Retrieval-Augmented Generation (RAG) library.
Author: ragkit contributors
License: MIT
Keywords: rag,retrieval-augmented-generation,llm,qdrant,embeddings,nlp
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pydantic>=2.5
Requires-Dist: pydantic-settings>=2.1
Requires-Dist: python-dotenv>=1.0
Requires-Dist: openai>=1.30
Requires-Dist: sentence-transformers>=2.7
Requires-Dist: qdrant-client>=1.9
Requires-Dist: rank-bm25>=0.2.2
Requires-Dist: pypdf>=4.0
Requires-Dist: tenacity>=8.2
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-mock>=3.12; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Provides-Extra: ollama
Requires-Dist: ollama>=0.3.0; extra == "ollama"
Dynamic: license-file

# ragkit

A modular, production-oriented **Retrieval-Augmented Generation (RAG)** library for Python.

`ragkit` ingests documents, indexes them for both dense (vector) and keyword (BM25) search,
retrieves and reranks the most relevant chunks for a question, and generates a grounded,
citation-backed answer - all through a small public API:

```python
from ragkit import RAG

rag = RAG()
rag.ingest("documents/example.pdf")

response = rag.query("What is the refund policy?")
print(response.answer)
for source in response.sources:
    print(source.document, source.page, source.retrieval_score)
```

It is built as a standalone, installable library (not an application) so it can be `pip install`ed
into any Python project.

---

## Table of contents

1. [Architecture](#architecture)
2. [Installation](#installation)
3. [Environment configuration](#environment-configuration)
4. [Vector storage](#vector-storage)
5. [Basic usage](#basic-usage)
6. [Document ingestion](#document-ingestion)
7. [Querying](#querying)
8. [Sources & citations](#sources--citations)
9. [Configuration reference](#configuration-reference)
10. [Running ragkit with a local Ollama LLM](#running-ragkit-with-a-local-ollama-llm)
11. [Evaluation](#evaluation)
12. [Advanced usage](#advanced-usage)
13. [Project structure](#project-structure)
14. [Development & testing](#development--testing)
15. [Packaging / PyPI](#packaging--pypi)
16. [Design decisions & constraints](#design-decisions--constraints)

---

## Architecture

Each responsibility is its own module behind a small interface, so any one stage can be swapped
without touching the others:

```
Document
   |
   v
Loader (ingestion/loaders.py)          .pdf / .txt / .md -> Document
   |
   v
Cleaner (ingestion/cleaner.py)         whitespace/control-char normalization
   |
   v
Chunker (ingestion/chunker.py)         RecursiveCharacterChunker -> list[Chunk]
   |
   v
EmbeddingProvider (embeddings/)        SentenceTransformerEmbeddingProvider -> vectors
   |
   v
VectorStore (vectorstore/)             QdrantVectorStore
   |
   v
   +----------------------+
   |                      |
   v                      v
VectorRetriever      BM25Retriever (keyword/)
   |                      |
   +----------+-----------+
              v
       HybridRetriever (fusion: RRF or weighted)
              |
              v
       QueryRewriter (optional, LLM-based)
              |
              v
       Reranker (optional, cross-encoder)
              |
              v
       ContextBuilder (dedup, limit, cite)
              |
              v
       LLMProvider (generation/llm.py)   OpenAIProvider (OpenAI-compatible)
              |
              v
          RAGResponse (answer + sources + metadata)
```

`pipeline/rag.py`'s `RAG` class is the only module that knows about every other module - it wires
concrete implementations together (constructed from `Settings` by default, or injected directly for
tests/customization) and exposes `ingest()` / `query()` as the public surface.

**Baseline vs. advanced is a configuration difference, not two implementations.** Query rewriting,
hybrid retrieval, and reranking are each gated by a feature flag
(`ENABLE_QUERY_REWRITING`, `ENABLE_HYBRID_RETRIEVAL`, `ENABLE_RERANKING`). There is one pipeline; the
evaluation harness (`evaluation/evaluate.py`) runs it under different flag combinations - `baseline`,
`hybrid`, `hybrid_rerank`, `final` - and records results for comparison across them. See
[Evaluation](#evaluation).

## Installation

Requires Python >= 3.10.

```bash
git clone <this-repo>
cd rag
python -m venv .venv
# Windows: .venv\Scripts\activate      macOS/Linux: source .venv/bin/activate

pip install -e ".[dev]"
```

The default `EmbeddingProvider` and `Reranker` run locally via `sentence-transformers`
(downloads a small model to the local HF cache on first use - no API key needed for embeddings).
The vector store (Qdrant) also runs embedded, in-process - nothing separate to start (see
[Vector storage](#vector-storage)). By default the `LLMProvider` talks to an OpenAI-compatible Chat
Completions API (OpenAI itself, or any self-hosted server - vLLM, LM Studio, ... - that speaks the
same API) and needs `API_KEY`. To run generation locally with **zero API cost** via Ollama instead
(`pip install -e ".[dev,ollama]"`), see
[Running ragkit with a local Ollama LLM](#running-ragkit-with-a-local-ollama-llm).

## Environment configuration

Copy the example file and fill in real values:

```bash
cp .env.example .env
```

**No API keys, model names, ports, or other configuration are hardcoded anywhere in the source
code.** Everything is read once, centrally, in [`src/ragkit/config/settings.py`](src/ragkit/config/settings.py)
via `pydantic-settings`, and every other module receives a `Settings` object rather than calling
`os.getenv` itself. See [Configuration reference](#configuration-reference) for every variable.

## Vector storage

**No separate service required by default.** `ragkit` uses Qdrant's embedded local mode: the vector
database runs in-process and persists to disk at `VECTOR_DB_PATH` (default: `./.ragkit_data/qdrant`).
`pip install ragkit-rag`, configure `.env`, and `rag.ingest(...)` works immediately - nothing to start
beforehand. Collections are created automatically on first `ingest()` call, sized to match the
active embedding model's dimension.

Embedded mode is a single-process store, not built for concurrent multi-process access or large-
scale production workloads. For those, run Qdrant as a real server instead (self-hosted or managed)
and point `ragkit` at it by setting `VECTOR_DB_HOST` (and optionally `VECTOR_DB_PORT`) in `.env` -
this switches `QdrantVectorStore` from local mode to server mode, with no other code changes:

```env
VECTOR_DB_HOST=localhost
VECTOR_DB_PORT=6333
```

Verify a server is reachable:

```bash
curl http://localhost:6333/collections
# or open http://localhost:6333/dashboard in a browser
```

## Basic usage

```bash
python examples/basic_usage.py
```

```python
from pathlib import Path
from ragkit import RAG

rag = RAG()
result = rag.ingest("documents/example.pdf")
print(f"Indexed {result.chunk_count} chunks from {result.document_name}")

response = rag.query("What is the refund policy?")
print(response.answer)
```

## Document ingestion

`rag.ingest(path)` runs: load -> clean -> chunk -> embed -> upsert into Qdrant -> index into BM25.

Supported formats out of the box: `.pdf`, `.txt`, `.md` (registered in
[`ingestion/loaders.py`](src/ragkit/ingestion/loaders.py); adding a format means adding one loader
function, nothing else changes).

Every `Chunk` carries full provenance: `document_id`, `document_name`, `source`, `chunk_id`,
`chunk_index`, and `page_number` (populated for PDFs, `None` for plain text/markdown).

Chunking is **structure-aware and recursive**
([`ingestion/chunker.py`](src/ragkit/ingestion/chunker.py)): it prefers to split on paragraph
boundaries, falling back to sentence and then word boundaries only for oversized pieces, and
carries `CHUNK_OVERLAP` characters of trailing context from one chunk into the next. Configurable
via `CHUNK_SIZE` / `CHUNK_OVERLAP`.

Re-ingesting the same file (by path) replaces its previous chunks rather than duplicating them -
`document_id` is derived deterministically from the file's absolute path.

## Querying

`rag.query(question, top_k=None)` runs: query rewriting (optional) -> retrieval (dense or hybrid,
per config) -> reranking (optional) -> context construction -> grounded generation.

The system prompt ([`generation/prompts.py`](src/ragkit/generation/prompts.py)) instructs the model
to answer only from retrieved context, cite sources with `[n]` markers, explicitly say when the
context is insufficient rather than guess, and distinguish stated facts from inference.

## Sources & citations

`RAGResponse` is a structured object, not a bare string:

```python
response.answer     # str
response.sources    # list[Source]: document, page, chunk_id, retrieval_score, text
response.metadata    # dict: latency per stage, rewritten query, active feature flags
```

`ContextBuilder` ([`generation/context_builder.py`](src/ragkit/generation/context_builder.py))
deduplicates retrieved chunks, orders them by score, stops once `MAX_CONTEXT_CHARS` is reached, and
assigns each included chunk a citation number (`[1]`, `[2]`, ...) matching `response.sources`, so
every claim the model cites can be traced back to the exact chunk that supports it.

## Configuration reference

All variables, with defaults, live in [`.env.example`](.env.example). Summary:

| Variable | Default | Purpose |
|---|---|---|
| `LLM_PROVIDER` | `openai` | `openai` (any OpenAI-compatible endpoint) or `ollama` (local, see [next section](#running-ragkit-with-a-local-ollama-llm)). |
| `API_KEY` | *(none)* | LLM API key. Required only when `LLM_PROVIDER=openai`, and only for `query()`/generation, not `ingest()`. |
| `MODEL` | `gpt-4o-mini` | Chat model name, OpenAI-compatible. Used when `LLM_PROVIDER=openai`. |
| `OPENAI_BASE_URL` | *(none)* | Point at a self-hosted OpenAI-compatible endpoint instead of api.openai.com. |
| `OLLAMA_BASE_URL` | `http://localhost:11434` | Ollama server address. Used when `LLM_PROVIDER=ollama`. |
| `OLLAMA_MODEL` | `llama3.2:3b` | Local model name (must already be `ollama pull`ed). Used when `LLM_PROVIDER=ollama`. |
| `EMBEDDING_MODEL` | `sentence-transformers/all-MiniLM-L6-v2` | Local embedding model. |
| `RERANKER_MODEL` | `cross-encoder/ms-marco-MiniLM-L-6-v2` | Local cross-encoder reranker model. |
| `VECTOR_DB_PATH` | `./.ragkit_data/qdrant` | On-disk path for Qdrant's embedded local mode (default, no server needed). |
| `VECTOR_DB_HOST` / `VECTOR_DB_PORT` | *(none)* / `6333` | Set `VECTOR_DB_HOST` to use a real Qdrant server instead of local mode. |
| `VECTOR_DB_COLLECTION` | `documents` | Qdrant collection name. |
| `CHUNK_SIZE` / `CHUNK_OVERLAP` | `500` / `100` | Chunking parameters (characters). |
| `TOP_K` | `5` | Chunks included in the final context/answer. |
| `RETRIEVAL_CANDIDATE_K` | `20` | Candidates pulled before fusion/reranking narrows to `TOP_K`. |
| `ENABLE_QUERY_REWRITING` | `false` | Turn on LLM-based query rewriting. |
| `ENABLE_HYBRID_RETRIEVAL` | `true` | Combine vector + BM25 retrieval. |
| `ENABLE_RERANKING` | `true` | Cross-encoder rerank of candidates. |
| `HYBRID_FUSION_STRATEGY` | `rrf` | `rrf` or `weighted` - see [`retrieval/hybrid.py`](src/ragkit/retrieval/hybrid.py). |
| `HYBRID_ALPHA` | `0.5` | Vector-vs-keyword weight, `weighted` strategy only. |
| `MAX_CONTEXT_CHARS` | `6000` | Hard cap on context sent to the LLM. |
| `LOG_LEVEL` | `INFO` | Python logging level for the `ragkit` logger namespace. |

Settings can also be overridden programmatically (bypassing `.env` entirely), which is how
`evaluation/evaluate.py` runs the same pipeline under different presets:

```python
from ragkit import RAG, Settings
rag = RAG(settings=Settings(enable_hybrid_retrieval=False, enable_reranking=False))
```

## Running ragkit with a local Ollama LLM

`LLM_PROVIDER=ollama` runs the entire pipeline - including evaluation - with **zero paid
API calls**: generation happens against a small model running locally via
[Ollama](https://ollama.com), on the same machine, fully offline after the model is pulled once.
Retrieval already runs locally (Qdrant + local embeddings + local reranker); this is what makes
generation local too.

### 1. Install Ollama

- **Windows**: `winget install --id Ollama.Ollama -e`, or download the installer from
  [ollama.com/download](https://ollama.com/download).
- **macOS / Linux**: see [ollama.com/download](https://ollama.com/download).

The installer sets up a background server listening on `http://localhost:11434` and normally
starts it automatically. Verify it's running:

```bash
curl http://localhost:11434/api/version
# {"version":"0.32.15"}  (or similar)
```

If that fails, start it manually with `ollama serve` (or launch the Ollama app).

### 2. Pick and pull a model sized for your hardware

**This project defaults to `llama3.2:3b` (~2GB download), chosen specifically for a CPU-only
machine with 16GB RAM** - it's small enough to leave headroom for the local embedding model
(~90MB), the local reranker (~90MB), Qdrant, and everything else running on the machine, while
still following instructions well enough to produce grounded, citation-following answers. Do not
default to a 7B+ model on hardware like this - it will be slow and can crowd out RAM needed by the
rest of the pipeline.

```bash
ollama pull llama3.2:3b
```

| Model | Approx. download | When to use it |
|---|---|---|
| `llama3.2:1b` | ~1.3GB | Tightest memory budget, or noticeably faster answers; weaker instruction-following. |
| `llama3.2:3b` (default) | ~2.0GB | Recommended default for 16GB RAM / CPU-only. |
| `qwen2.5:3b` | ~1.9GB | Comparable alternative to `llama3.2:3b`, worth trying if you want a second opinion. |

Do **not** pull a 7B/8B/14B+ model on this kind of hardware unless you know you have the RAM and
patience for it - this project intentionally does not default to one.

### 3. Configure ragkit to use it

In `.env`:

```env
LLM_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llama3.2:3b
```

No `API_KEY` needed for this path.

### 4. Run ragkit

Nothing else changes - `RAG()` reads `LLM_PROVIDER` from `.env` and picks `OllamaProvider`
automatically:

```bash
python examples/basic_usage.py
```

If Ollama isn't running or the model hasn't been pulled, you get an actionable
`ConfigurationError` (not a generic connection error) telling you exactly what to run:

```python
from ragkit import RAG
from ragkit.generation.llm import OllamaProvider

OllamaProvider(model="llama3.2:3b", base_url="http://localhost:11434").check_availability()
# raises ConfigurationError with a concrete fix if Ollama or the model isn't ready
```

`evaluation/evaluate.py` runs this same check automatically before doing any work.

### 5. Run the evaluation harness locally

```bash
pip install -e ".[dev,ollama]"

python evaluation/evaluate.py --all
```

This runs every preset against the fixed corpus/question set and records raw results (answers,
retrieved contexts, per-question latency) - see [Evaluation](#evaluation). Quality scoring
(faithfulness, relevancy, precision/recall) is not currently wired up; it will be reintroduced via
LangSmith.

### Resource considerations

- **CPU inference is slower than GPU or a hosted API** - expect single answers to take several
  seconds to tens of seconds depending on context length and machine speed, and a full evaluation
  run (one generation call per question, per preset) to take meaningfully longer than the
  equivalent OpenAI run. This is expected, not a bug.
- Don't run multiple large local models at once, and don't add a second Ollama model into the mix
  unless you've confirmed you have the RAM for both models plus everything else.
- Qdrant runs embedded (see [Vector storage](#vector-storage)) and Ollama runs natively on the
  host - neither needs a separate service or virtualization overhead, keeping the whole local
  setup lightweight.

## Evaluation

Evaluation is reproducible and dataset-driven, not a one-off manual check. By default this uses
whatever `LLM_PROVIDER` is set to in `.env` - see [the previous section](#running-ragkit-with-a-local-ollama-llm)
to run it entirely locally via Ollama with no API key, or configure `LLM_PROVIDER=openai` to use a
hosted model instead:

```bash
pip install -e ".[dev]"          # add ",ollama" too if using LLM_PROVIDER=ollama

python evaluation/evaluate.py --preset baseline
python evaluation/evaluate.py --preset hybrid
python evaluation/evaluate.py --preset hybrid_rerank
python evaluation/evaluate.py --preset final
# or: python evaluation/evaluate.py --all
```

Each run: (re-)ingests the fixed corpus in `evaluation/datasets/corpus/` (a synthetic "Acme
Gadgets" customer-policy knowledge base - refunds, shipping, warranty, privacy, subscriptions),
answers every question in `evaluation/datasets/eval_dataset.json`, and writes:

- `evaluation/results/<preset>_<timestamp>.json` - full detail: every question, answer, retrieved
  contexts, ground truth, and per-stage latency.
- `evaluation/results/summary.json` - aggregate latency appended across every run, for comparing
  presets over time.

Quality scoring (faithfulness, relevancy, precision/recall) is not currently wired up in this
harness - it will be reintroduced via LangSmith. Until then, `evaluation/offline_analysis.py`
provides a zero-LLM-cost proxy: it exercises the real retrieval pipeline for all four presets and
scores what was retrieved against a hand-authored answer key (see the script's docstring).

`eval_dataset.json` deliberately spans six question categories (simple factual, multi-sentence,
requiring multiple chunks of context, confusingly similar policies, **unanswerable from the
corpus**, and exact numeric/date facts) so results reflect genuine retrieval difficulty rather than
trivially-answerable questions.

**No results are manufactured or hardcoded, and the dataset is not modified to inflate metrics** -
this repository does not commit a results file with claimed numbers; run the evaluation yourself to
get honest numbers for your machine, models, and API access, and compare `summary.json` across
presets to see what each retrieval upgrade (hybrid retrieval, reranking, query rewriting) actually
changed for this corpus.

## Advanced usage

```bash
python examples/advanced_usage.py
```

Covers: programmatic `Settings` overrides for A/B-ing retrieval configurations, and dropping in a
fully custom `Reranker` (or any other interface: `EmbeddingProvider`, `VectorStore`, `Retriever`,
`QueryRewriter`, `LLMProvider`) via constructor injection on `RAG(...)` without modifying `ragkit`
itself:

```python
from ragkit import RAG
from ragkit.retrieval.reranker import Reranker

class MyReranker(Reranker):
    def rerank(self, query, results):
        ...  # your logic

rag = RAG(reranker=MyReranker())
```

## Project structure

```
rag/
├── src/ragkit/
│   ├── config/          # Settings (pydantic-settings) - the only place reading env vars
│   ├── ingestion/        # loaders, cleaner, chunker
│   ├── embeddings/       # EmbeddingProvider interface + local (sentence-transformers) impl
│   ├── vectorstore/      # VectorStore interface + Qdrant impl
│   ├── retrieval/        # vector / keyword / hybrid retrievers, query rewriter, reranker
│   ├── generation/       # LLMProvider interface + OpenAI/Ollama impls, prompts, context builder
│   ├── pipeline/         # RAG - the public orchestrator
│   ├── models/           # Document, Chunk, RAGResponse, Source
│   ├── exceptions.py      # RAGKitError hierarchy
│   ├── logging_config.py
│   └── utils/            # Timer
├── tests/
│   ├── unit/              # hermetic, mocked - no network/API required
│   └── integration/       # full pipeline wiring (fakes) + optional live Qdrant/LLM test
├── evaluation/
│   ├── datasets/          # eval_dataset.json + corpus/*.txt
│   ├── evaluate.py        # preset-driven evaluation harness
│   └── results/           # generated JSON/CSV run output (gitignored)
├── examples/
│   ├── basic_usage.py
│   └── advanced_usage.py
├── .env.example
├── pyproject.toml
└── README.md
```

## Development & testing

```bash
pip install -e ".[dev]"

# Unit tests: fast, no external services
pytest tests/unit -q

# Integration tests (fakes-based tier runs by default; live tiers are opt-in)
pytest tests/integration -q
RAGKIT_RUN_LIVE_INTEGRATION_TESTS=1 pytest tests/integration -q   # needs Qdrant + real API_KEY
RAGKIT_RUN_LIVE_OLLAMA_TESTS=1 pytest tests/integration -q        # needs Qdrant + Ollama + pulled model

# Everything with coverage
pytest --cov=ragkit --cov-report=term-missing
```

Unit tests cover configuration validation, document loading, cleaning, chunking (size limits,
overlap, page boundaries, provenance), embeddings (via a fake model), the vector store (via a fake
Qdrant client), retrieval (vector, BM25, hybrid fusion - both RRF and weighted), query rewriting
(including graceful fallback on LLM failure), reranking, context construction (dedup/limit/
citations), response model serialization, LLM provider selection (`OllamaProvider` vs.
`OpenAIProvider`, based on `LLM_PROVIDER`), and `OllamaProvider` itself (via a fake `ollama.Client`
- unreachable server, model not pulled, model ready, generation success/failure). No unit test
requires network access, an API key, or a running Ollama server.

## Packaging / PyPI

The package uses a modern `src/` layout and `pyproject.toml` (PEP 621 metadata,
`setuptools.build_meta` backend):

```bash
pip install build twine
python -m build                 # produces dist/*.whl and dist/*.tar.gz
python -m twine check dist/*     # validate metadata before publishing
python -m twine upload dist/*    # publish to PyPI (needs PyPI credentials)
```

Once published:

```bash
pip install ragkit-rag
```

```python
from ragkit import RAG
```

The PyPI distribution is named `ragkit-rag` (the name `ragkit` was already taken by an unrelated
project), but the importable package stays `ragkit` - `pip install ragkit-rag` and
`from ragkit import RAG` are both correct, together.

Only the public surface (`RAG`, `RAGResponse`, `Source`, `IngestResult`, `Settings`,
`get_settings`, and the `RAGKitError` hierarchy) is exported from the top-level `ragkit` package;
internal modules remain importable for advanced customization but are not part of the promised
stable API.

## Design decisions & constraints

- **No hardcoded secrets, models, or endpoints.** Every configurable value flows through
  `ragkit.config.settings.Settings`, loaded once from `.env` / environment variables.
- **Every external dependency sits behind an interface**: `EmbeddingProvider`, `VectorStore`,
  `Retriever`, `Reranker`, `QueryRewriter`, `LLMProvider`. Swapping an implementation never requires
  touching the pipeline or public API - `LLMProvider` ships two implementations
  (`OpenAIProvider`, `OllamaProvider`), selected purely via `LLM_PROVIDER` in `.env`.
- **Vector database runs embedded by default**, no separate service needed - `pip install ragkit-rag`
  works standalone. A real Qdrant server (self-hosted or managed) is opt-in via `VECTOR_DB_HOST`,
  for production workloads that need concurrent access or scale past a single process.
- **Query rewriting is not assumed to help** - it ships disabled by default and is measured, not
  assumed, in evaluation.
- **Errors are typed** (`RAGKitError` subclasses per stage) and carry actionable messages; library
  users are never handed a raw, unexplained stack trace for common failure modes (missing API key,
  unreachable Qdrant, empty/unsupported document, embedding/LLM failure).
- **No evaluation data is fabricated.** The evaluation harness produces honest, reproducible
  numbers from a fixed dataset; comparing presets shows what each retrieval upgrade actually
  changed, for better or worse.
