Metadata-Version: 2.5
Name: ragfresh
Version: 0.1.0
Summary: Detect stale, drifted, and ghost documents in RAG vector indexes
Project-URL: Homepage, https://github.com/nac7/ragfresh
Project-URL: Repository, https://github.com/nac7/ragfresh
Project-URL: Issues, https://github.com/nac7/ragfresh/issues
Project-URL: Changelog, https://github.com/nac7/ragfresh/blob/main/CHANGELOG.md
Author-email: Nachiket Lele <lelenachiket991@gmail.com>
License-Expression: MIT
License-File: LICENSE
Keywords: data-quality,embeddings,llm,rag,retrieval,vector-database
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.9
Provides-Extra: all
Requires-Dist: chromadb>=0.4; extra == 'all'
Requires-Dist: psycopg[binary]>=3.1; extra == 'all'
Provides-Extra: chroma
Requires-Dist: chromadb>=0.4; extra == 'chroma'
Provides-Extra: dev
Requires-Dist: chromadb>=0.4; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: psycopg[binary]>=3.1; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.16; extra == 'dev'
Provides-Extra: pgvector
Requires-Dist: psycopg[binary]>=3.1; extra == 'pgvector'
Description-Content-Type: text/markdown

# ragfresh

[![CI](https://github.com/nac7/ragfresh/actions/workflows/ci.yml/badge.svg)](https://github.com/nac7/ragfresh/actions/workflows/ci.yml)

Detect stale, drifted, and ghost documents in RAG vector indexes.

Embeddings get created once. Source documents keep changing. Nothing tells you
your index has drifted from reality until a user gets an answer grounded in a
document that was edited — or deleted — weeks ago. `ragfresh` closes that gap
with a lightweight sidecar ledger and a CI-gateable check.

## Staleness classes

- **unchanged** — source content hash matches what was indexed
- **changed, not reindexed** — the source doc was edited since it was indexed
- **deleted, still indexed** — the source doc is gone but the vector store
  still serves it (ghost citation risk — the worst class)
- **not yet indexed** — present in the source, absent from the ledger/store

## Quickstart

```bash
pip install ragfresh

ragfresh init ./docs --db ragfresh.db
# ... time passes, docs change ...
ragfresh check ./docs --db ragfresh.db
```

Exits non-zero when `changed, not reindexed` or `deleted, still indexed` is
non-empty, so it's usable as a CI gate.

`check` only detects staleness. To resolve it -- re-embed changed/missing
docs and remove ghost entries from both the store and the ledger -- run:

```bash
ragfresh reindex ./docs --db ragfresh.db --chroma-collection my-index
```

`reindex` uses a local embedding model (no API calls) and requires a
writable store -- Chroma or pgvector; the default ledger-only store is
read-only. Preview what it would do without writing anything:

```bash
ragfresh reindex ./docs --db ragfresh.db --chroma-collection my-index --dry-run
```

## Sources & stores

- Sources: filesystem, git (`ragfresh.sources.GitSource`)
- Stores:
  - Chroma — `pip install ragfresh[chroma]`, then `--chroma-collection NAME [--chroma-path DIR]`
  - pgvector (any Postgres table with a doc_id column and a `vector`-typed
    embedding column) — `pip install ragfresh[pgvector]`, then
    `--pg-dsn DSN [--pg-table documents] [--pg-id-column id] [--pg-embedding-column embedding] [--pg-document-column TEXT_COL]`
    (the embedding/document columns are only needed for `reindex`, not `check`)
  - Neither flag given → the ledger itself stands in as the index (read-only)

## Semantic drift scoring

Hash-based staleness treats any byte-level change as "changed" — a typo fix
and a rewritten paragraph look identical. `--score-drift` classifies each
`changed_not_reindexed` doc as **cosmetic** or **meaningful** using cosine
distance between embeddings of the old and new text:

```bash
ragfresh init ./docs --db ragfresh.db          # snapshots content by default
ragfresh check ./docs --db ragfresh.db --score-drift
#   [CHANGED] report.md (meaningful, score=0.812)
```

Requires `ragfresh[chroma]` (reuses its bundled local ONNX MiniLM model —
downloaded once, then fully offline, no API calls) and content snapshots
from `init` (pass `--no-store-content` at init time to opt out and disable
this later). Tune sensitivity with `--drift-threshold` (default `0.15`).

By default drift scoring uses chromadb's bundled MiniLM model. If your
production embeddings diverge enough from that model that drift scores
wouldn't track them well, point `--embedding-model` (on `check` and
`reindex`) at any sentence-transformers model name instead (requires the
`sentence-transformers` package).

## Chunked indexes

Most production RAG pipelines chunk a doc into several vectors (ids like
`report.md#0`, `report.md#1`, ...), not one vector per doc. By default
ragfresh assumes indexed ids equal doc_ids -- pass `--chunk-separator` on
`check` to fold chunk ids back to their source doc:

```bash
ragfresh check ./docs --db ragfresh.db --chroma-collection my-index --chunk-separator "#"
```

A doc counts as indexed if *any* of its chunks are present. ragfresh
doesn't chunk on your behalf -- it only needs to know how to reverse
whatever chunking scheme you already use, via a separator (CLI) or a
`chunk_id_to_doc_id` callable (library API, `ragfresh.check.check`).
**Detection-only:** `reindex` doesn't support chunked stores yet -- it
writes one vector per doc_id, so re-embedding into a chunked index would
require ragfresh to own the chunking strategy, which it deliberately
doesn't.

## Development

```bash
pip install -e ".[dev]"
pytest
ruff check .
mypy
```

## Status

v0.1.0. Filesystem + git sources; SQLite ledger; Chroma + pgvector stores
(both readable and writable, for `check` and `reindex`); CLI with drift
scoring (pluggable embedding model), chunk-aware detection, and `reindex`
(flat indexes only, with `--dry-run`); staleness-injection benchmark
(`benchmarks/staleness_injection/`); CI across 3 OSes × 3 Python versions
plus a lint/type-check job (ruff + mypy). 44 tests passing.

Known gap: `reindex` doesn't support chunked indexes (writes one vector
per doc_id; `check` handles chunking, `reindex` deliberately doesn't own a
chunking strategy).

## License

MIT
