Metadata-Version: 2.4
Name: right-rag
Version: 0.2.3
Summary: Domain-structured RAG pipeline template
License-File: LICENSE
Requires-Python: >=3.13
Requires-Dist: tenacity>=9.1.4
Description-Content-Type: text/markdown

# right-rag

[![PyPI](https://img.shields.io/pypi/v/right-rag.svg)](https://pypi.org/project/right-rag/)
[![CI](https://github.com/sunmodza/right-rag/actions/workflows/ci.yml/badge.svg)](https://github.com/sunmodza/right-rag/actions/workflows/ci.yml)

`right-rag` is a domain-structured RAG template for documents where plain text
chunking loses important context: laws, policies, contracts, manuals, academic
papers, handbooks, and other hierarchical corpora.

It is built around one rule: model the document structure first, then choose the
retrieval method. Vector search, keyword search, SQL, graph retrieval, and custom
domain lookup are all implementation choices, not framework assumptions.

> Important: each project should adapt the `spec/` plugins to its own
> documents, domain structure, retrieval behavior, answer style, and evaluation
> rules.

## Why right-rag

Most RAG quality problems come from weak evidence modeling:

- rules are separated from their exceptions
- answers cannot be traced to exact source units
- metadata exists but is not usable by retrieval
- document versions and changes are flattened away
- the index technology dictates the design

right-rag keeps the core generic and lets each project define what valid
evidence means for its domain.

## What It Helps With

| Capability | Why it matters |
| --- | --- |
| Structure preservation | Keeps document hierarchy such as chapter, section, clause, rule, exception, table, and procedure step. |
| Evidence traceability | Every unit and chunk can carry source spans, citation data, file path, page, heading path, and domain attributes. |
| Temporal and change tracking | Unit state is replayable through changes, so evolving documents can be indexed incrementally instead of treated as unrelated blobs. |
| Domain-specific retrieval | Retrievers work from saver artifacts, so a project can use vector search, SQL filters, graph traversal, exact lookup, or custom rules. |
| Evaluation workflow | Query outputs can be checked against expected evidence, required answer text, forbidden text, and domain-specific evaluators. |
| Template-first customization | New projects start from `right-rag init` and replace only the plugins that need domain knowledge. |

## Where It Fits

right-rag is useful when answers must be grounded in auditable document
evidence:

- policy assistants that must include exceptions and current policy versions
- contract review tools that must cite exact clauses and party obligations
- legal or regulatory research where section hierarchy controls meaning
- technical manual assistants that must preserve ordered steps and warnings
- academic paper search where methods, metrics, tables, and limitations matter
- product knowledge bases that must distinguish current and legacy versions

## Architecture

```mermaid
flowchart LR
    A[Source documents] --> B[Document loaders]
    B --> C[Unit parsers]
    C --> D[Unit assemblers]
    D --> E[Rooted semantic units]
    E --> F[Chunk parsers]
    F --> G[Retrieval-ready chunks]
    E --> H[Unit savers]
    G --> I[Chunk savers]
    H --> J[Unit artifacts]
    I --> K[Chunk artifacts]
    J --> L[Retrievers]
    K --> L
    L --> M[Answer generator]
    M --> N[Answer with evidence]
    N --> O[Evaluators and reports]
```

```mermaid
flowchart TD
    subgraph Index
      A[Load documents] --> B[Parse units] --> C[Assemble one Unit root] --> D[Parse chunks] --> E[Save artifacts]
    end

    subgraph Query
      E[User query] --> F[Retrieve from artifacts] --> G[Generate answer]
    end

    subgraph Evaluate
      H[Evaluation cases] --> E
      G --> I[Evaluate] --> J[Report]
    end
```

## Install

```powershell
pip install right-rag
```

Or run without a permanent install:

```powershell
uvx --from right-rag right-rag --help
```

Requires Python 3.13+.

## Quick Start

```powershell
right-rag init my-rag
cd my-rag
```

Put documents in:

```text
data/documents/
```

Run the pipeline:

```powershell
right-rag index
right-rag booking
right-rag query "your question"
right-rag evaluate
```

`right-rag booking [ROOT_UNIT_ID]` opens an interactive, level-by-level Unit
browser. Choose a numbered Unit to descend, type a keyword to filter only its
direct children, or use `re:PATTERN` for a regular expression. The commands
`text`, `back`, `root`, `all`, and `quit` inspect and navigate the tree without
skipping levels.

The same workflows are available as a Python API, without starting a server:

```python
import re
from pathlib import Path

import rightrag

rightrag.add_document(Path("law.txt"), project="my-rag")
result = rightrag.index(project="my-rag")
answer = rightrag.query("What is prohibited?", project="my-rag")

roots = rightrag.search_units(parent_id=None, project="my-rag")
path = rightrag.booking(
    roots[0].id,
    ["Chapter 2", re.compile(r"Section\s+10")],
    project="my-rag",
)
```

The public functions are `add_document`, `index`, `query`, `booking`,
`evaluate`, `search_documents`, `search_units`, `search_chunks`, and `status`.
The `search_*` functions combine listing, exact filters, and regular-expression
search. `booking` walks direct children one level at a time; strings are
case-insensitive keywords and compiled patterns are regular expressions. It
does not skip levels or fall back to a global search.

Custom saver artifacts must implement `iter_units()`/`iter_chunks()` to support
browsing and `booking`. File-based plugins should use
`right_rag.runtime.project_path(...)`; the API scopes projects without changing
the process working directory.

Generated local artifacts:

- `output/units/`
- `output/chunks/`
- `.right-rag-state.json`
- `output/evaluation/`

The packaged plugins are intentionally generic starting points. Each project
should replace them with domain-specific plugins under `spec/` so parsing,
chunking, saving, retrieval, answer generation, and evaluation match the actual
dataset.
In practice, adapting right-rag means editing `spec/` first, not changing the
framework core.

## Project Template

`right-rag init` creates:

```text
my-rag/
  right_rag.toml
  data/documents/
  eval/cases.example.json
  output/
  spec/
    document_loaders/
    unit_parsers/
    unit_assemblers/
    chunk_parsers/
    unit_savers/
    chunk_savers/
    retrievers/
    answer_generators/
    evaluators/
    reporters/
```

## Documentation

- [Benefits And Use Cases](docs/benefits-and-use-cases.md): what right-rag helps with and why it works.
- [Template Guide](docs/template-guide.md): step-by-step guide for each plugin type.
- [Scenario Examples](docs/scenarios.md): examples for policies, contracts, manuals, academic papers, and product knowledge bases.
- [Code Architecture](docs/architecture.md): framework modules and index/query/evaluate lifecycle.
- [Artifacts And Retrieval](docs/artifacts-and-retrieval.md): saver artifact and retriever design.
- [Evaluation](docs/evaluation.md): how to build evaluation cases and reports.
- [Documentation Index](docs/README.md): recommended adaptation workflow.

## Extension Points

Each folder under `spec/` is a project-owned customization point. Start with
the document structure, then implement only the plugins your corpus needs.

| Folder | Purpose |
| --- | --- |
| `spec/document_loaders/` | Load source documents from files, APIs, stores, or custom extractors. |
| `spec/unit_parsers/` | Convert documents into a faithful `ParsedUnit` tree. |
| `spec/unit_assemblers/` | Arrange parsed Units into one domain-specific root tree. |
| `spec/chunk_parsers/` | Convert semantic `Unit` objects into retrieval-ready `Chunk` records. |
| `spec/unit_savers/` | Persist units and return unit artifacts. |
| `spec/chunk_savers/` | Persist chunks and return chunk artifacts. |
| `spec/retrievers/` | Retrieve evidence from artifacts. |
| `spec/answer_generators/` | Generate answers from retrieved evidence. |
| `spec/evaluators/` | Score query outputs. |
| `spec/reporters/` | Write evaluation reports. |

## Domain Modeling

Start by inspecting the source documents. Choose units that match the author's
real structure:

- laws: act -> chapter -> section -> clause -> subclause
- contracts: agreement -> article -> clause -> schedule item
- policies: policy -> section -> rule -> exception
- manuals: manual -> chapter -> procedure -> step
- academic papers: paper -> section -> subsection -> paragraph/table/figure note

Good unit parsers:

- preserve hierarchy with `ParsedUnit.subunits`
- set source spans when possible
- keep `fulltext` faithful to the source
- store provenance in attributes or metadata
- reject unsupported formats visibly
- avoid catch-all parsers

Good unit assemblers:

- use the existing Unit hierarchy and arrange parentless Units into one root
- preserve Unit ids, source spans, provenance, and source order
- create only structural parent Units needed by the domain
- keep synthetic structural Units empty instead of duplicating child text

Good chunk parsers:

- chunk inside a single unit
- keep rules with their exceptions and limitations
- carry citation and filter attributes
- prefer fewer useful chunks over many tiny fragments

Parser and assembler selection has no implicit fallback. Unsupported, failed,
or ambiguous inputs stop that document branch and remain visible in coverage.
Each pipeline stage reports every Spec as `In`, `Processed`, `Out`, and `%`,
followed by a `SUM`; overall document coverage and issues are persisted in
`.right-rag-state.json` and returned by `index()`/`status()`.
Default JSON savers also remove stale records at the end of a successful index
run, so deleted or newly unsupported Units are not returned by later queries.

## Evaluation

```powershell
right-rag evaluate eval/cases.example.json --output output/evaluation/report.md
```

The default evaluator is for smoke tests and regression checks. Real projects
should add domain-specific evaluators under `spec/evaluators/`.

## Development

```powershell
git clone https://github.com/sunmodza/right-rag.git
cd right-rag
uv sync
python -m compileall src/right_rag
python -m unittest discover -s tests
uv build
```

Run plugin discovery:

```powershell
uv run python -c "from right_rag.registry import build_registry; r=build_registry(); print(r.document_loaders, r.unit_parsers, r.unit_assemblers, r.chunk_parsers, r.unit_savers, r.chunk_savers)"
```

## License

`right-rag` is released under the MIT License. See [LICENSE](LICENSE).
