Metadata-Version: 2.4
Name: attune-rag
Version: 1.2.1
Summary: Lightweight, LLM-agnostic RAG pipeline with pluggable corpora. Works with Claude, Gemini, or any LLM.
Author-email: Patrick Roebuck <admin@smartaimemory.com>
License:                                  Apache License
                                   Version 2.0, January 2004
                                http://www.apache.org/licenses/
        
           Licensed under the Apache License, Version 2.0 (the "License");
           you may not use this file except in compliance with the License.
           You may obtain a copy of the License at
        
               http://www.apache.org/licenses/LICENSE-2.0
        
           Unless required by applicable law or agreed to in writing, software
           distributed under the License is distributed on an "AS IS" BASIS,
           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
           See the License for the specific language governing permissions and
           limitations under the License.
        
        Copyright 2026 Patrick Roebuck / Smart AI Memory
        
Project-URL: Homepage, https://attune-rag.dev
Project-URL: Repository, https://github.com/Smart-AI-Memory/attune-rag
Project-URL: Documentation, https://attune-rag.dev
Project-URL: Changelog, https://github.com/Smart-AI-Memory/attune-rag/blob/main/CHANGELOG.md
Project-URL: Issues, https://github.com/Smart-AI-Memory/attune-rag/issues
Keywords: rag,retrieval-augmented-generation,llm,claude,gemini,grounding,provenance,attune
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Text Processing :: Markup :: Markdown
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: structlog>=24.0
Requires-Dist: jinja2>=3.1.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Requires-Dist: jsonschema>=4.20
Provides-Extra: attune-help
Requires-Dist: attune-help<0.14,>=0.13.0; extra == "attune-help"
Provides-Extra: claude
Requires-Dist: anthropic<1.0,>=0.95; extra == "claude"
Requires-Dist: claude-agent-sdk>=0.1.63; extra == "claude"
Provides-Extra: gemini
Requires-Dist: google-genai<2.0,>=1.0; extra == "gemini"
Provides-Extra: author
Requires-Dist: attune-author>=0.25.0; extra == "author"
Provides-Extra: embeddings
Requires-Dist: model2vec<1.0,>=0.3; extra == "embeddings"
Provides-Extra: transformers
Requires-Dist: sentence-transformers<6.0,>=3.0; extra == "transformers"
Provides-Extra: all
Requires-Dist: attune-help<0.14,>=0.13.0; extra == "all"
Requires-Dist: anthropic<1.0,>=0.95; extra == "all"
Requires-Dist: google-genai<2.0,>=1.0; extra == "all"
Requires-Dist: model2vec<1.0,>=0.3; extra == "all"
Requires-Dist: sentence-transformers<6.0,>=3.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: ruff>=0.4.0; extra == "dev"
Requires-Dist: black>=24.0; extra == "dev"
Requires-Dist: pre-commit>=3.5; extra == "dev"
Requires-Dist: build>=1.0; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Requires-Dist: attune-help<0.14,>=0.13.0; extra == "dev"
Requires-Dist: attune-author>=0.25.0; extra == "dev"
Requires-Dist: anthropic<1.0,>=0.95; extra == "dev"
Requires-Dist: google-genai<2.0,>=1.0; extra == "dev"
Dynamic: license-file

# attune-rag

Lightweight, LLM-agnostic RAG pipeline with pluggable
corpora. Works with Claude, Gemini, or any LLM.

🌐 **Docs & guides: [attune-rag.dev](https://attune-rag.dev)**

## Proven retrieval — the numbers

Every figure is reproducible with `attune-rag-benchmark`. Bundled-corpus
retrieval has **required CI thresholds**; faithfulness is checked when
selected by the [quality gate](#quality-baselines).

| | Bundled corpus¹ | Unseen corpus, overall² | Unseen corpus, **hard paraphrases**³ |
|---|---|---|---|
| **precision@1** | **100%** | 73% (lightweight, zero torch) | 25% lightweight → **90%** transformer tier |
| **recall@3** | **100%** | 82% (lightweight, zero torch) | 25% lightweight → **100%** transformer tier |
| **faithfulness** | **0.97** mean | — | — |

Plus: configurable **abstention** drops out-of-corpus false answers from
**92% → 8%** — so the retriever stays quiet instead of confidently wrong.

The headline: on a corpus attune-rag has **never seen or been tuned on**,
asking questions worded *nothing* like the docs, the transformer tier goes
from a keyword baseline of **1-in-4** to **9-in-10** top-1 correct, and
**finds the right doc in the top 3 every single time** (recall@3 100%).

<sub>¹ bundled attune-help corpus: required `P@1 ≥ 0.975 / R@3 = 1.00`,
with `faithfulness ≥ 0.9698` when selected and measured; actuals shown.
² lightweight keyword on an unseen
corpus (`corpus_b`), no embeddings.  ³ pure-paraphrase stress test
(`corpus_c`, queries with almost no vocabulary overlap with the docs) —
measured in `docs/specs/transformer-retriever/`.</sub>

## Two ways to run it

- 🪶 **Lightweight (default).** Keyword retrieval, optionally fused with
  **torch-free** static embeddings (`pip install attune-rag[embeddings]`).
  5 pure-Python deps, no LLM SDK, no torch, fully offline, ~1 ms/query —
  the dependency-light path that holds **100% / 100%** on a tuned corpus
  and **73% precision@1** on an unseen one.
- 🤖 **Transformer tier (opt-in).** Real sentence-transformers embeddings
  (`pip install attune-rag[transformers]`) for paraphrase-heavy or arbitrary
  corpora. On pure-paraphrase queries it lifts precision@1 **25% → 90%** and
  recall@3 **25% → 100%** — the generalization no torch-free retriever
  reaches. Heavyweight (pulls torch, ~GB); embedding-primary; never a default.

- **No LLM SDK at install time; footprint scales with your setup.**
  The base install pulls **5 small pure-Python deps** (`structlog`,
  `jinja2`, `pyyaml`, `rich`, `jsonschema`) — no LLM SDK, no torch.
  Retrieval tiers add only what they need: `[embeddings]` adds
  torch-free `model2vec`; `[transformers]` adds the torch stack. You
  pay for exactly the setup you choose.
- **Pluggable corpus.** Use attune-help (the default), any
  markdown directory, or your own `CorpusProtocol`.
- **Returns a prompt string + citation records** by default
  — `pipeline.run()` never opens a network connection. You
  call your own LLM however you like. Optional provider
  adapters ship convenience wrappers.
- **Opt-in retrieval ladder.** Keyword retrieval by default;
  add a torch-free static-embedding `HybridRetriever`
  (`[embeddings]`) or a `TransformerRetriever` (`[transformers]`)
  for paraphrase-heavy corpora, plus configurable **abstention**
  (`min_score=`) to suppress confident out-of-corpus answers.
  Every rung is opt-in and fail-safe; the keyword default is
  unchanged.

## Why attune-rag

attune-rag publishes **measured quality thresholds** in the active
[quality baseline JSON](docs/specs/release-quality-baseline/thresholds.json)
and [performance baseline JSON](docs/specs/downstream-validation/perf-thresholds.json).
CI requires `P@1 ≥ 0.975` and `R@3 = 1.00`; completed, selected
faithfulness measurements must reach `0.9698`. Performance regressions
block on two selected CPU metrics; other timing metrics remain advisory.
The [quality-baseline section](#quality-baselines) describes selection,
failure handling, and installed-artifact checks.

### vs LangChain / LlamaIndex

| | attune-rag | LangChain | LlamaIndex |
|---|---|---|---|
| Required runtime deps | 5 (pure-Python) | many (transitively, ~30+) | many (~25+) |
| LLM SDK at install | none | bundled | bundled |
| Published quality regression thresholds | yes (P@1, R@3, faithfulness) | no | no |
| Published perf thresholds (wall + CPU) | yes | no | no |
| Citation primitives built-in | yes | add-on | add-on |
| "Get a string back, call your own LLM" | default | possible w/ effort | possible w/ effort |

LangChain and LlamaIndex are fantastic frameworks if you want
batteries-included orchestration. attune-rag is the alternative
when you want a RAG component you can drop into an existing app
without buying into a framework — and want the quality bar
quantified, not implied.

Beyond drop-in retrieval, attune-rag is the grounding foundation
for the `attune-*` family's content-quality discipline. The
`attune-author` polish/fact-check pipeline uses attune-rag's
retrieval + faithfulness primitives to verify generated help
content is grounded in source material before it's marked
authoritative. This library's own selected faithfulness measurements
are checked against `mean_faithfulness ≥ 0.9698`; downstream authoring
policies are separate.

### What attune-rag is **not**

Honest exclusions, so you can self-disqualify if you need any
of these:

- **Not an agent framework.** No multi-step chains, no tool-use
  orchestration, no agent loops.
- **Not a document-parsing toolkit.** Bring your markdown
  already-parsed; use `unstructured.io` or similar upstream.
- **Not a vector DB integration.** Keyword retrieval is the
  default; the optional `[embeddings]` / `[transformers]` tiers
  embed in-process (model2vec / sentence-transformers). There is
  no external vector-store integration — you wire your own if
  you need one.
- **Not a one-line-install batteries-included framework.** That's
  LangChain / LlamaIndex. attune-rag is for the case where that's
  too much.

## Install

```bash
pip install attune-rag                      # core — keyword retrieval, 5 pure-Python deps, no LLM SDK
# Retrieval tiers (opt-in):
pip install 'attune-rag[embeddings]'        # + torch-free static hybrid retrieval
pip install 'attune-rag[transformers]'      # + transformer retrieval tier (pulls torch, ~GB)
# Corpus & LLM adapters (opt-in):
pip install 'attune-rag[attune-help]'       # + bundled help corpus
pip install 'attune-rag[claude]'            # + Claude adapter
pip install 'attune-rag[gemini]'            # + Gemini adapter
# Convenience:
pip install 'attune-rag[all]'               # every extra, incl. the transformer tier (pulls torch, ~GB)
```

Extras compose, e.g. `pip install 'attune-rag[embeddings,claude]'`. The base
install stays dependency-light on purpose; only `[transformers]` (and
therefore `[all]`) pulls torch.

## Authentication

The faithfulness judge is subscription-first: inside a Claude Code
session (`CLAUDECODE=1`) with the `[claude]` extra installed, judge
calls route through your Claude subscription via the Agent SDK — no
`ANTHROPIC_API_KEY` needed. From a plain terminal (or in CI) it uses
`ANTHROPIC_API_KEY` as before. Override with
`FaithfulnessJudge(auth_mode="api"|"sub")`, `ATTUNE_RAG_AUTH_MODE`,
or `attune-rag-benchmark --auth-mode`. Note: RAG answer *generation*
(`ClaudeProvider`) is API-key-only — the subscription route covers
the judge.

## Prompt-cache TTL

`ClaudeProvider` marks the stable prompt prefix (and the first citation
document block) with Anthropic prompt caching. By default the cache window
is the 5-minute `ephemeral` tier. Set `ATTUNE_RAG_CACHE_TTL=1h` to extend it
to one hour — at the **same per-token price** — for workloads that issue
clusters of related queries within the hour (dashboards, benchmark sweeps):

```bash
export ATTUNE_RAG_CACHE_TTL=1h   # default: 5m
```

Leave it unset for one-off queries: the cache rarely survives long enough to
pay off, and the default wire shape is byte-identical to prior behavior.

## Quick start — Claude

```bash
pip install 'attune-rag[attune-help,claude]'
```

```python
import asyncio
from attune_rag import RagPipeline

async def main():
    pipeline = RagPipeline()  # defaults to AttuneHelpCorpus
    response, result = await pipeline.run_and_generate(
        "How do I run a security audit with attune?",
        provider="claude",
    )
    print(response)
    print("\nSources:", [h.entry.path for h in result.citation.hits])

asyncio.run(main())
```

## Quick start — Gemini

```bash
pip install 'attune-rag[attune-help,gemini]'
```

```python
response, result = await pipeline.run_and_generate(
    "...", provider="gemini", model="gemini-1.5-pro",
)
```

## Quick start — custom corpus, any LLM

```python
from pathlib import Path
from attune_rag import RagPipeline, DirectoryCorpus

pipeline = RagPipeline(corpus=DirectoryCorpus(Path("./my-docs")))
result = pipeline.run("How do I...?")

# Send result.augmented_prompt to whatever LLM you use.
# The pipeline itself does NOT call an LLM unless you use
# run_and_generate or call a provider adapter yourself.
```

> 📖 **Building a quality corpus.** See [`docs/USER_CORPUS_GUIDE.md`](docs/USER_CORPUS_GUIDE.md)
> for the corpus-authoring discipline that produced the bundled
> attune-help corpus's 100% / 100% baseline + 100% paraphrased R@3:
> frontmatter aliases, multi-token intent, the `MIN_ALIAS_OVERLAP`
> knob, stemmer traps, the override file pattern, and the
> strict-dominance measurement loop. The guide is the v0 forerunner
> of the v1.0.0 framework framing (`user-corpus-onboarding` spec).

## CLI

Everything above is also reachable from the terminal — your own
corpus, the retrieval tiers, and abstention included:

```bash
attune-rag query "how do I run a security audit?"   # bundled corpus
attune-rag query "..." --corpus-path ./my-docs      # your markdown corpus
attune-rag query "..." --retriever hybrid           # [embeddings] tier
attune-rag query "..." --retriever transformer      # [transformers] tier
attune-rag query "..." --min-score 5                # abstain below threshold
attune-rag query "..." --prompt-variant strict      # prompt template
attune-rag query "..." --provider claude            # full RAG + LLM call
attune-rag query "..." --json                       # hits as JSON
attune-rag corpus-info --corpus-path ./my-docs      # corpus stats
attune-rag providers                                # installed LLM extras
```

The bundled default corpus requires the `[attune-help]` extra; on a
bare `pip install attune-rag`, pass `--corpus-path`.

## Hybrid retrieval (optional)

`QueryExpander` and `LLMReranker` require the `[claude]` extra and an
`ANTHROPIC_API_KEY`. Both are opt-in and fail-safe — any API error
falls back to keyword-only order automatically.

```python
from attune_rag import RagPipeline, LLMReranker, QueryExpander

# Reranker only (recommended for precision):
pipeline = RagPipeline(reranker=LLMReranker())

# Expander + reranker (max coverage):
pipeline = RagPipeline(
    expander=QueryExpander(),
    reranker=LLMReranker(),
)
```

### Embedding / hybrid retrieval (`[embeddings]` extra)

`HybridRetriever` fuses the keyword retriever with static **model2vec**
embeddings via Reciprocal Rank Fusion. No torch, no GPU, no API key —
offline, millisecond encode. Install: `pip install attune-rag[embeddings]`.

```python
from attune_rag import RagPipeline, HybridRetriever

# Opt-in. KeywordRetriever stays the default.
pipeline = RagPipeline(retriever=HybridRetriever())
```

**When to use it:** on an **unstructured / arbitrary corpus** (raw markdown
with no curated summaries or aliases), embeddings recover the paraphrase
recall that token overlap misses — measured **+9pts recall@3** on an unseen
corpus. On a corpus that's already **keyword-tuned** (curated
summaries/aliases, like the bundled `.help/`), an equal blend trades away
top-1 precision, so the default weighting favors keyword (`keyword_weight=2.0`);
raise it to fully protect a tuned corpus, lower toward `1.0` to maximize the
embedding contribution. Falls back to keyword-only if the extra isn't
installed.

### Confidence-gated retrieval (`[embeddings]` extra)

The hybrid's RRF fusion always dilutes a confident keyword ranking — on
a keyword-**tuned** corpus that costs top-1 precision (measured: 5pts).
The **confidence gate** removes the trade: when the keyword top-1 score
clears a calibrated threshold, the keyword ranking returns untouched
(the embedding leg is never even encoded); below it, hits RRF-fuse for
paraphrase rescue.

```python
from attune_rag import DirectoryCorpus, RagPipeline

# One calibration derives the threshold AND wires the measured recipe:
pipeline = RagPipeline.calibrated(
    DirectoryCorpus("./my-docs"),
    queries=["question my docs answer", ...],   # in-corpus
    negatives=["question they don't", ...],     # out-of-corpus
    gated=True,
)
```

Measured (`scripts/measure_gated_mechanism.py`, torch-free,
deterministic, `docs/specs/confidence-gated-retrieval/`): a
keyword-tuned corpus holds **1.00/1.00** P@1/R@3 (ungated hybrid: 0.95)
while an unseen corpus's hard-paraphrase P@1 lifts to **0.70** vs 0.50
ungated. The recipe uses the retrieval-tuned
`minishlab/potion-retrieval-32M` model (~250 MB cached vs ~60 MB for the
base default — the upgrade is load-bearing) with an unfiltered keyword
leg and 1:1 fusion weights; pass `embedding=` to choose a different
model.

**One threshold, one decision.** The gate threshold and the abstention
`min_score` come from the **same calibration sweep** — one per-corpus
number answering "do I trust this keyword retrieval?", with the response
chosen by tier:

```text
keyword top-1 >= T  ->  trust it: keyword ranking returns untouched
keyword top-1 <  T  ->  keyword tier:  ABSTAIN (calibrated min_score)
                        gated tier:    RESCUE  (RRF-fuse with embeddings)
```

The gated tier does not abstain — an embedding-side confidence floor is
deliberately unshipped (unmeasured). If out-of-corpus protection matters
more than paraphrase rescue, stay on the calibrated keyword tier.

### Transformer retrieval (`[transformers]` extra) — heavyweight

`TransformerRetriever` ranks by a real **sentence-transformers** model
(default `BAAI/bge-small-en-v1.5`). This is the **heaviest** rung of the
opt-in ladder (`keyword` → `[embeddings]` static → `[transformers]`):
it pulls **torch (~GB)** and downloads a model once (then offline,
~10–300 ms/query). Install: `pip install attune-rag[transformers]`.

```python
from attune_rag import RagPipeline, TransformerRetriever

# Heavyweight opt-in — embedding-primary, for arbitrary corpora.
pipeline = RagPipeline(retriever=TransformerRetriever())
```

**When to use it:** only on an **arbitrary corpus where paraphrasing is
heavy** and the torch-free static embeddings fall short. It is
embedding-primary and **tanks a keyword-tuned corpus's top-1 precision**,
so it is never a default. Measured on two unseen corpora
(`docs/specs/transformer-retriever/`): hard-tier paraphrase **P@1
0.50 → 0.69 (corpus_b) and 0.70 → 0.90 (corpus_c), recall@3 → 1.00** —
~+20pts per corpus over the best torch-free config (the confidence-gated
blend above), a margin no torch-free retriever reaches. For a keyword-tuned corpus use
`KeywordRetriever`; for a lexically-aligned arbitrary corpus the lighter
`[embeddings]` `HybridRetriever` is usually enough.

Pass `query_prefix=""` for symmetric models (e.g. `all-MiniLM-L6-v2`); the
default prefix is tuned for BGE's asymmetric query instruction.

Measure the lift on **your** corpus before paying the torch install:

```bash
attune-rag-measure --corpus-path ./my-docs --queries ./queries.yaml \
    --retriever transformer
```

### Abstention — don't answer out-of-corpus queries

**The bundled corpus abstains by default.** `RagPipeline()` ships with a
calibrated `min_score=5` — measured against the SHA-locked gate set +
negatives (`scripts/measure_abstention_distributions.py`): keeps 100% of
gate-set recall, cuts the out-of-corpus false-answer rate 92% → 8%.

The threshold is an **absolute keyword score**, so 5 is only safe for
the corpus it was measured on — your own corpus keeps the conservative
class default (`min_score=2.0`) until you calibrate it from its OWN
data. The honest BYO path is one call:

```python
from attune_rag import DirectoryCorpus, RagPipeline

pipeline = RagPipeline.calibrated(
    DirectoryCorpus("./my-docs"),
    queries=["question my docs answer", ...],      # in-corpus
    negatives=["question they don't", ...],        # out-of-corpus
)
```

Explicit construction always wins (escape hatch):

```python
from attune_rag import RagPipeline, KeywordRetriever

pipeline = RagPipeline(retriever=KeywordRetriever(min_score=7))
```

The CLI equivalent recommends a threshold from your query sets:

```bash
python -m attune_rag.benchmark --calibrate-abstention
```

## Template editor primitives (`attune_rag.editor`)

Headless toolkit for tools that need to validate, lint, and refactor a
template corpus — used by the [`attune-gui`](https://pypi.org/project/attune-gui/)
template editor and the [`attune-author`](https://pypi.org/project/attune-author/)
`edit` CLI, but works standalone with any
[`CorpusProtocol`](https://github.com/Smart-AI-Memory/attune-rag).

| API | What it does |
|-----|---------------|
| `load_schema()` | Loads `template_schema.json` (the v1 frontmatter contract: required `type` enum + `name`; optional `tags`, `aliases`, `summary`, `source`, `hash`; `additionalProperties: true`). |
| `parse_frontmatter(text)` / `validate_frontmatter(data)` | Split a template into frontmatter + body and report typed `FrontmatterIssue`s — used by linters and editors. |
| `lint_template(text, rel_path, corpus)` | Returns `Diagnostic[]` for schema violations, broken `[[alias]]` references, and depth-marker sequence errors. 1-indexed line/col ranges. |
| `autocomplete_tags(corpus, prefix, limit)` / `autocomplete_aliases(corpus, prefix, limit)` | Prefix-match completions ranked by frequency (tags) or lexical proximity (aliases). Sub-ms on 1k templates. |
| `find_references(corpus, name, kind)` | Locate every alias/tag/path occurrence across body, frontmatter, and `cross_links.json`. |
| `plan_rename(corpus, old, new, kind)` | Build a `RenamePlan` (one `FileEdit` per affected file with unified-diff hunks) for `kind="alias"` or `"tag"`. Raises `RenameCollisionError` on existing alias targets. |
| `apply_rename(corpus, plan)` | Atomically apply the plan (tempfile-per-file + sequential rename + drift-detection rollback). Returns the list of affected paths. |

Schema, lint, and rename are pure functions over `CorpusProtocol` — no I/O,
no global state. All three pieces are tested as a unit and used live by the
attune-gui editor's `/api/corpus/<id>/lint`, `/autocomplete`, and
`/refactor/rename/{preview,apply}` routes.

```python
from attune_rag import DirectoryCorpus
from attune_rag.editor import lint_template, plan_rename, apply_rename

corpus = DirectoryCorpus(Path("./templates"))

# Validate a template before saving
diagnostics = lint_template(
    text=Path("./templates/concepts/foo.md").read_text(),
    rel_path="concepts/foo.md",
    corpus=corpus,
)

# Rename an alias across the whole corpus
plan = plan_rename(corpus, old="oldname", new="newname", kind="alias")
print(f"Affects {len(plan.edits)} files")
affected = apply_rename(corpus, plan)
```

## Dashboard

```bash
attune-rag dashboard show    # live terminal dashboard
attune-rag dashboard render --out report.html  # HTML snapshot
```

## Quality baselines

The repository's active JSON baselines define the gates below. Historical
measurement notes explain their derivation; the JSON files supply the
current thresholds. The packaging and validation repairs described here
ship in [1.2.1](CHANGELOG.md#121--2026-10-19).

### Retrieval + faithfulness

| Metric | Threshold (current) | Source |
|---|---:|---|
| `precision_at_1` | **≥ 0.975** | keyword retrieval |
| `recall_at_3` | **= 1.00** | keyword retrieval |
| `mean_faithfulness` | **≥ 0.9698** | Claude judge; locked aggregate σ = 0.0052 |

Values and the SHA-256 lock for the 40-query set live in
[`thresholds.json`](docs/specs/release-quality-baseline/thresholds.json).
[The quality workflow](.github/workflows/benchmark.yml) requires retrieval
in every mode. With `ANTHROPIC_API_KEY` configured, it selects faithfulness
for pushes to `main`, manual runs, and PRs whose title contains
`[full-bench]` or whose diff touches retrieval, reranker, expander, pipeline,
prompts, or eval code. Otherwise it reports why faithfulness was skipped.

A completed faithfulness regression fails the gate. A classified transient
failure in the primary provider pass permits at most one workflow retry,
and only after retrieval passes its locked thresholds. If the provider
remains unavailable, the workflow discloses that state and still requires
retrieval to pass. Authentication failures, other local errors, malformed
or missing evidence, and measured regressions fail validation or gating;
they do not qualify for that retry. The checker rejects nonfinite values,
booleans, numeric strings, and quality values outside `[0, 1]`.

The [measurement notes](docs/specs/archive/release-quality-baseline/baseline-1.md)
record the historical runs behind the current quality lock.

### Per-hot-path latency

The active [performance JSON](docs/specs/downstream-validation/perf-thresholds.json)
contains eight measurements: CPU time and wall-clock time for four
benchmarks. [The per-PR workflow](.github/workflows/perf.yml) selects exactly
`keyword_retriever_retrieve.cpu` and `rag_pipeline_run.cpu` as blocking.

| Benchmark | CPU regression | Wall-clock regression |
|---|---|---|
| `keyword_retriever_retrieve` | blocking | advisory |
| `rag_pipeline_run` | blocking | advisory |
| `directory_corpus_load` | advisory | advisory |
| `llm_reranker_rerank` | advisory | advisory |

Both reranker axes exist in the locked baseline; the regular per-PR
measurement omits LLM benchmarks. It runs 30 trials of the local benchmarks.
Missing or invalid selected CPU evidence, a missing baseline, or a failed
measurement blocks the gate. Valid advisory-metric regressions remain
visible in the comment without blocking.

The recorded V2 baseline uses five invocations × 20 trials, with `sigma=2`
on Linux and CPython 3.11.15. It tracks intra-run and inter-run variance
separately and sets thresholds to `mean + 2 × inter_run_stdev`.
CPU timings still vary with execution conditions; they are not deterministic.
See the [measurement notes](docs/specs/downstream-validation/perf-baseline.md)
for the runner fingerprint and per-metric results.

### Installed distribution checks

Source tests and installed-artifact checks cover different failure modes.
The shared [distribution checker](scripts/check_distribution.py) copies the
current contents of Git-tracked files into a clean snapshot, honoring
deletions and excluding untracked build debris. It builds an sdist and then
a wheel from that sdist. Newly added files must be tracked to enter this
snapshot.

Run from the repository root with an empty output directory:

```bash
python -m pip install build ".[attune-help]"
python scripts/check_distribution.py --output-dir dist --report artifact-validation.json
```

Both distributions must contain byte-identical copies of the snapshot's
package `.py`/`.pyi` files and five required resources: the typing marker,
corpus alias and summary overrides, editor schema, and dashboard HTML.
The checker installs the wheel in a temporary environment, verifies import
origins, and runs separate source and wheel probes with the same prepared
dependencies. It compares ordered top-three paths and scores for all 40
locked queries and requires both probes to meet the retrieval thresholds.
It makes no generation or faithfulness calls. To check an existing pair
without rebuilding, supply `--sdist PATH --wheel PATH --report PATH`.

The JSON receipt records source and query hashes, dependency versions,
import origins, per-query results, and distribution checksums.
[Test CI](.github/workflows/tests.yml) runs this checker separately from
the source suite. [Publication](.github/workflows/publish.yml) verifies the
receipt's checksum and the exact sdist/wheel checksums before uploading
those same files to PyPI, without rebuilding them.

### Why this is the differentiator

The threshold JSON files and historical measurement notes are checked in,
so the selected metrics, cutoffs, and measurement methods can be inspected
alongside the code that enforces them.

## Bundled `.help/` corpus

The repo ships a polished `.help/` corpus that documents
attune-rag's own surface — 143 templates across 13 features ×
11 kinds (`concept`, `task`, `reference`, `quickstart`, `faq`,
`error`, `warning`, `tip`, `note`, `comparison`,
`troubleshooting`). Generated by
[`attune-author`](https://pypi.org/project/attune-author/) with
strict fact-check; queryable via `AttuneHelpCorpus` or as the
bundled default for `RagPipeline()`. See
[`.help/features.yaml`](.help/features.yaml) for the feature
map and [`.help/templates/`](.help/templates/) for the content.

The 13 features: `pipeline`, `retrieval`, `corpus`, `prompts`,
`provenance`, `providers`, `eval`, `benchmark`, `cli`, `editor`,
`dashboard`, `expander`, `reranker`.

### What faithfulness measures

Faithfulness scores how well an answer is **grounded in the retrieved
passages** — `1.0` means every claim in the answer is supported by a
cited source; lower scores mean some claims have no support in the
context. It catches hallucination in a way that `precision_at_k` and
`recall_at_k` can't: those only measure whether the *right documents*
were retrieved, not whether the *generated answer* actually used them.

attune-rag uses **Claude as the judge** via Anthropic's tool-use API
to produce a structured score in `[0.0, 1.0]` for each
`(query, answer, retrieved_context)` triple. The reported metric is
the mean over the golden query set. Aggregate σ ≈ `0.005` over 40
queries even though per-query judge non-determinism can swing 40+
percentage points on individual queries — averaging absorbs the noise.

The same discipline powers `attune-author`'s polish/fact-check
pipeline — generated help content is scored against retrieved
source passages before being marked authoritative. attune-rag's
faithfulness primitives aren't just instrumentation; they're the
contract the family's content-quality story is built on.

### Run faithfulness manually

```bash
pip install 'attune-rag[claude]'
export ANTHROPIC_API_KEY=sk-ant-...

# Retrieval metrics only (free, deterministic):
attune-rag-benchmark --queries queries.yaml --json out.json

# Add faithfulness (~1 Claude API call per query, costs tokens):
attune-rag-benchmark --queries queries.yaml --with-faithfulness --json out.json

# Compare extended-thinking on vs off (2× judge cost):
attune-rag-benchmark --queries queries.yaml --with-faithfulness --compare-thinking --json out.json
```

The judge implementation lives at
`attune_rag.eval.faithfulness.FaithfulnessJudge`. Note: `attune_rag.eval.*`
is currently INTERNAL and may move — the `attune-rag-benchmark
--with-faithfulness` CLI is the stable contract.

For the historical faithfulness methodology, the v1/v2 ground-truth
calibration runs, and the extended-thinking-vs-default decision record, see
[`docs/rag/faithfulness-thinking-calibration.md`](https://github.com/Smart-AI-Memory/attune-rag/blob/main/docs/rag/faithfulness-thinking-calibration.md).

## Embeddings — shipped (`[embeddings]` extra)

Local, CPU-only, offline embeddings shipped via
[`model2vec`](https://github.com/MinishLab/model2vec) static models (no
torch, no API key). Keyword retrieval remains the default; embeddings
layer in opt-in through `HybridRetriever` — see
[Embedding / hybrid retrieval](#embedding--hybrid-retrieval-embeddings-extra)
above.

**Measured impact** (`docs/specs/rag-strengthening/`): on an unseen,
unstructured corpus, hybrid lifts **recall@3 +9pts**; on the keyword-tuned
`.help/` corpus the default keyword path is unchanged. The benchmark can
compare either retriever:

```bash
attune-rag-benchmark --retriever keyword      # default
attune-rag-benchmark --retriever hybrid       # keyword + embeddings (RRF)
attune-rag-benchmark --retriever transformer  # [transformers] tier
```

See
[CHANGELOG.md](https://github.com/Smart-AI-Memory/attune-rag/blob/main/CHANGELOG.md)
for the decision record.

## Prompt caching (Claude only)

When using the Claude provider, `run_and_generate` automatically enables
[Anthropic prompt caching](https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching)
on the stable RAG context prefix (≥ 1 024 chars). This eliminates
repeated token costs on the corpus portion of the prompt when the same
context block is reused across calls.

No configuration needed — the provider handles the `cache_control`
header automatically.

## Public API

attune-rag's public surface is documented below and snapshot-tested
in [tests/unit/test_api_surface.py](tests/unit/test_api_surface.py).
Formal SemVer commitments have been in effect since 0.2.0 — see
[docs/POLICY.md](docs/POLICY.md) for the deprecation policy. Symbols
PUBLIC in a minor line stay PUBLIC through every patch of that line;
the snapshot test catches drift.

**Top-level (`from attune_rag import ...`):**

- Pipeline — `RagPipeline`, `RagResult`
- Corpus — `CorpusProtocol`, `RetrievalEntry`, `DirectoryCorpus`,
  `AttuneHelpCorpus`
- Retrieval — `KeywordRetriever`, `EmbeddingRetriever`,
  `HybridRetriever`, `TransformerRetriever`, `RetrievalHit`,
  `RetrieverProtocol`
- Provenance — `CitationRecord`, `CitedSource`, `ClaimCitation`,
  `format_citations_markdown`, `format_claim_citations_markdown`
- Prompting — `build_augmented_prompt`, `PROMPT_VARIANTS`
- Hybrid retrieval — `QueryExpander`, `LLMReranker`

**PUBLIC submodules** (importable by qualified path):

- `attune_rag.corpus` — exposes `AliasInfo`, `DuplicateAliasError`,
  `load_aliases_from_file` in addition to the top-level corpus names
- `attune_rag.corpus.attune_help` — `AttuneHelpCorpus`
- `attune_rag.corpus.help_adapter` — `HelpCorpusAdapter` Protocol
- `attune_rag.providers` — `LLMProvider`, `get_provider`,
  `list_available`
- `attune_rag.measure_corpus` — `measure(...)` function +
  `MeasureResult` dataclass for scoring a corpus against a query
  set. CLI via `python -m attune_rag.measure_corpus ...` or the
  `attune-rag-measure` console script. See
  [`docs/USER_CORPUS_GUIDE.md`](docs/USER_CORPUS_GUIDE.md) §6 for
  the worked example.
- `attune_rag.editor` — template-editor primitives (lint, schema,
  rename, autocomplete, references); see "Template editor primitives"
  above for the symbol list
- `attune_rag.editor.{rename,schema,lint,autocomplete,references}` —
  the individual editor submodules

**Console scripts:**

- `attune-rag` — CLI entry point (`attune_rag.cli:main`)
- `attune-rag-measure` — quality measurement
  (`attune_rag.measure_corpus:main`); CI-suitable via `--watermark-r3`
  (non-zero exit on fail)
- `attune-rag-benchmark` — retrieval + optional faithfulness
  benchmark (`attune_rag.benchmark:main`). The default golden query
  sets ship in the repo checkout, not the wheel — on a pip install,
  point `--queries` (and optionally `--negatives`) at your own sets.

Anything not listed above is INTERNAL and may change in any release.
The underscore-prefixed editor modules (`attune_rag.editor._rename`
etc.) shipped in 0.1.x are deprecation shims as of 0.2.0; they
re-export the new non-underscore names and emit `DeprecationWarning`.
They are removed at 1.0.0 (originally scheduled for 0.3.0 — see
[`docs/POLICY.md`](docs/POLICY.md) §3).

## Status

**0.9.x — the release line the 1.0.0 cut ships from.** The opt-in
retrieval ladder (0.5.x: `[embeddings]` static hybrid,
`[transformers]` dense tier, `min_score=` abstention), model tiers +
fable-5 handling (0.8.0), and the ranking correction that stopped the
content preview double-counting frontmatter aliases (0.9.0) are all
in. The current quality gates use P@1 ≥ 0.975, R@3 = 1.00, and
mean faithfulness ≥ 0.9698 when selected and measured; see
[Quality baselines](#quality-baselines) for the conditions.

SemVer commitments have been binding since 0.2.0 —
[`docs/POLICY.md`](docs/POLICY.md) §2; symbols PUBLIC in a minor
line stay PUBLIC through every patch of that line, and the snapshot
test catches drift.

Classifier is `4 - Beta` — the Production/Stable flip is a v1.0.0
deliverable, alongside a documented support window and 1.x
deprecation policy ([`docs/POLICY.md`](docs/POLICY.md) §8–§9).

Part of the attune ecosystem
([attune-ai](https://github.com/Smart-AI-Memory/attune-ai),
[attune-help](https://github.com/Smart-AI-Memory/attune-help);
the earlier [attune-author](https://github.com/Smart-AI-Memory/attune-author)
and [attune-gui](https://github.com/Smart-AI-Memory/attune-gui)
packages are archived — their roles were absorbed into attune-ai).

## License

Apache 2.0. See
[LICENSE](https://github.com/Smart-AI-Memory/attune-rag/blob/main/LICENSE).
