Metadata-Version: 2.4
Name: right-rag
Version: 0.1.3
Summary: Domain-structured RAG pipeline template
License-File: LICENSE
Requires-Python: >=3.13
Requires-Dist: docling>=2.110.0
Requires-Dist: prefect>=3.7.7
Requires-Dist: pydantic>=2.13.4
Description-Content-Type: text/markdown

# right-rag

[![PyPI](https://img.shields.io/pypi/v/right-rag.svg)](https://pypi.org/project/right-rag/)
[![CI](https://github.com/sunmodza/right-rag/actions/workflows/ci.yml/badge.svg)](https://github.com/sunmodza/right-rag/actions/workflows/ci.yml)

`right-rag` is a domain-structured RAG template for documents where plain text
chunking loses important context: laws, policies, contracts, manuals, academic
papers, handbooks, and other hierarchical corpora.

It is built around one rule: model the document structure first, then choose the
retrieval method. Vector search, keyword search, SQL, graph retrieval, and custom
domain lookup are all implementation choices, not framework assumptions.

> Important: each project should adapt the `spec/` plugins to its own
> documents, domain structure, retrieval behavior, answer style, and evaluation
> rules.

## Why right-rag

Most RAG quality problems come from weak evidence modeling:

- rules are separated from their exceptions
- answers cannot be traced to exact source units
- metadata exists but is not usable by retrieval
- document versions and changes are flattened away
- the index technology dictates the design

right-rag keeps the core generic and lets each project define what valid
evidence means for its domain.

## What It Helps With

| Capability | Why it matters |
| --- | --- |
| Structure preservation | Keeps document hierarchy such as chapter, section, clause, rule, exception, table, and procedure step. |
| Evidence traceability | Every unit and chunk can carry source spans, citation data, file path, page, heading path, and domain attributes. |
| Temporal and change tracking | Unit state is replayable through changes, so evolving documents can be indexed incrementally instead of treated as unrelated blobs. |
| Domain-specific retrieval | Retrievers work from saver artifacts, so a project can use vector search, SQL filters, graph traversal, exact lookup, or custom rules. |
| Evaluation workflow | Query outputs can be checked against expected evidence, required answer text, forbidden text, and domain-specific evaluators. |
| Template-first customization | New projects start from `right-rag init` and replace only the plugins that need domain knowledge. |

## Where It Fits

right-rag is useful when answers must be grounded in auditable document
evidence:

- policy assistants that must include exceptions and current policy versions
- contract review tools that must cite exact clauses and party obligations
- legal or regulatory research where section hierarchy controls meaning
- technical manual assistants that must preserve ordered steps and warnings
- academic paper search where methods, metrics, tables, and limitations matter
- product knowledge bases that must distinguish current and legacy versions

## Architecture

```mermaid
flowchart LR
    A[Source documents] --> B[Document loaders]
    B --> C[Unit parsers]
    C --> D[Semantic units]
    D --> E[Chunk parsers]
    E --> F[Retrieval-ready chunks]
    D --> G[Unit savers]
    F --> H[Chunk savers]
    G --> I[Unit artifacts]
    H --> J[Chunk artifacts]
    I --> K[Retrievers]
    J --> K
    K --> L[Answer generator]
    L --> M[Answer with evidence]
    M --> N[Evaluators and reports]
```

```mermaid
flowchart TD
    subgraph Index
      A[Load documents] --> B[Parse units] --> C[Parse chunks] --> D[Save artifacts]
    end

    subgraph Query
      E[User query] --> F[Retrieve from artifacts] --> G[Generate answer]
    end

    subgraph Evaluate
      H[Evaluation cases] --> E
      G --> I[Evaluate] --> J[Report]
    end
```

## Install

```powershell
pip install right-rag
```

Or run without a permanent install:

```powershell
uvx --from right-rag right-rag --help
```

Requires Python 3.13+.

## Quick Start

```powershell
right-rag init my-rag
cd my-rag
```

Put documents in:

```text
data/documents/
```

Run the pipeline:

```powershell
right-rag index
right-rag query "your question"
right-rag evaluate
```

Generated local artifacts:

- `output/units/`
- `output/chunks/`
- `.right-rag-state.json`
- `output/evaluation/`

The packaged plugins are intentionally generic starting points. Each project
should replace them with domain-specific plugins under `spec/` so parsing,
chunking, saving, retrieval, answer generation, and evaluation match the actual
dataset.
In practice, adapting right-rag means editing `spec/` first, not changing the
framework core.

## Project Template

`right-rag init` creates:

```text
my-rag/
  right_rag.toml
  data/documents/
  eval/cases.example.json
  output/
  spec/
    document_loaders/
    unit_parsers/
    chunk_parsers/
    unit_savers/
    chunk_savers/
    retrievers/
    answer_generators/
    evaluators/
    reporters/
```

## Documentation

- [Benefits And Use Cases](docs/benefits-and-use-cases.md): what right-rag helps with and why it works.
- [Template Guide](docs/template-guide.md): step-by-step guide for each plugin type.
- [Scenario Examples](docs/scenarios.md): examples for policies, contracts, manuals, academic papers, and product knowledge bases.
- [Code Architecture](docs/architecture.md): framework modules and index/query/evaluate lifecycle.
- [Artifacts And Retrieval](docs/artifacts-and-retrieval.md): saver artifact and retriever design.
- [Evaluation](docs/evaluation.md): how to build evaluation cases and reports.
- [Documentation Index](docs/README.md): recommended adaptation workflow.

## Extension Points

Each folder under `spec/` is a project-owned customization point. Start with
the document structure, then implement only the plugins your corpus needs.

| Folder | Purpose |
| --- | --- |
| `spec/document_loaders/` | Load source documents from files, APIs, stores, or custom extractors. |
| `spec/unit_parsers/` | Convert documents into a faithful `ParsedUnit` tree. |
| `spec/chunk_parsers/` | Convert semantic `Unit` objects into retrieval-ready `Chunk` records. |
| `spec/unit_savers/` | Persist units and return unit artifacts. |
| `spec/chunk_savers/` | Persist chunks and return chunk artifacts. |
| `spec/retrievers/` | Retrieve evidence from artifacts. |
| `spec/answer_generators/` | Generate answers from retrieved evidence. |
| `spec/evaluators/` | Score query outputs. |
| `spec/reporters/` | Write evaluation reports. |

## Domain Modeling

Start by inspecting the source documents. Choose units that match the author's
real structure:

- laws: act -> chapter -> section -> clause -> subclause
- contracts: agreement -> article -> clause -> schedule item
- policies: policy -> section -> rule -> exception
- manuals: manual -> chapter -> procedure -> step
- academic papers: paper -> section -> subsection -> paragraph/table/figure note

Good unit parsers:

- preserve hierarchy with `ParsedUnit.subunits`
- set source spans when possible
- keep `fulltext` faithful to the source
- store provenance in attributes or metadata
- reject unsupported formats visibly
- avoid catch-all parsers

Good chunk parsers:

- chunk inside a single unit
- keep rules with their exceptions and limitations
- carry citation and filter attributes
- prefer fewer useful chunks over many tiny fragments

## Evaluation

```powershell
right-rag evaluate eval/cases.example.json --output output/evaluation/report.md
```

The default evaluator is for smoke tests and regression checks. Real projects
should add domain-specific evaluators under `spec/evaluators/`.

## Development

```powershell
git clone https://github.com/sunmodza/right-rag.git
cd right-rag
uv sync
python -m compileall src/right_rag
python -m unittest discover -s tests
uv build
```

Run plugin discovery:

```powershell
uv run python -c "from right_rag.registry import build_registry; r=build_registry(); print(r.document_loaders, r.unit_parsers, r.chunk_parsers, r.unit_savers, r.chunk_savers)"
```

## License

`right-rag` is released under the MIT License. See [LICENSE](LICENSE).
