Metadata-Version: 2.4
Name: right-rag
Version: 0.1.1
Summary: Domain-structured RAG pipeline template
License-File: LICENSE
Requires-Python: >=3.13
Requires-Dist: docling>=2.110.0
Requires-Dist: prefect>=3.7.7
Requires-Dist: pydantic>=2.13.4
Description-Content-Type: text/markdown

# right-rag

[![PyPI](https://img.shields.io/pypi/v/right-rag.svg)](https://pypi.org/project/right-rag/)
[![CI](https://github.com/sunmodza/right-rag/actions/workflows/ci.yml/badge.svg)](https://github.com/sunmodza/right-rag/actions/workflows/ci.yml)

`right-rag` is a domain-structured RAG template for corpora where plain text
chunking loses too much context: laws, policies, contracts, manuals, academic
papers, handbooks, and other hierarchical documents.

Instead of starting with "split text every N tokens", right-rag starts with the
document's real structure: sections, clauses, rules, definitions, procedures,
tables, exceptions, and citations. You customize that structure through small
plugins under `spec/`.

## Why right-rag

Most RAG failures are not embedding failures. They are structure failures:

- the answer is split away from its exception
- the citation points to a broad page instead of the exact clause
- metadata is stored but not connected to retrieval
- a vector index becomes the design instead of one possible implementation

right-rag keeps the core pipeline generic and lets each domain define what its
evidence means.

## Architecture

```mermaid
flowchart LR
    A[Source documents] --> B[Document loaders]
    B --> C[Unit parsers]
    C --> D[Semantic units]
    D --> E[Chunk parsers]
    E --> F[Retrieval-ready chunks]
    D --> G[Unit savers]
    F --> H[Chunk savers]
    G --> I[Unit artifacts]
    H --> J[Chunk artifacts]
    I --> K[Retrievers]
    J --> K
    K --> L[Answer generator]
    L --> M[Answer with evidence]
    M --> N[Evaluators and reports]
```

Index and query are separate:

```mermaid
flowchart TD
    subgraph Index
      A[Load] --> B[Parse units] --> C[Parse chunks] --> D[Save artifacts]
    end

    subgraph Query
      E[User query] --> F[Retrievers use artifacts] --> G[Answer generator]
    end

    subgraph Evaluate
      H[Eval cases] --> E
      G --> I[Evaluator] --> J[Report]
    end
```

## Install

Requirements:

- Python 3.13+

Install from PyPI:

```powershell
pip install right-rag
```

Or run without a permanent install:

```powershell
uvx --from right-rag right-rag --help
```

## Quick Start

Create a new RAG project:

```powershell
right-rag init my-rag
cd my-rag
```

Add source documents:

```text
data/documents/
```

Run the default smoke pipeline:

```powershell
right-rag index
right-rag query "your question"
right-rag evaluate
```

Generated local artifacts:

- `output/units/`
- `output/chunks/`
- `.right-rag-state.json`
- `output/evaluation/`

The default unit and chunk parsers are smoke-test implementations only. For
production, write domain parsers under `spec/`.

## Project Template

`right-rag init` creates:

```text
my-rag/
  right_rag.toml
  data/documents/
  eval/cases.example.json
  output/
  spec/
    document_loaders/
    unit_parsers/
    chunk_parsers/
    unit_savers/
    chunk_savers/
    retrievers/
    answer_generators/
    evaluators/
    reporters/
```

You adapt the project by editing `spec/`, not by forking the framework.

## Extension Points

| Folder | Purpose |
| --- | --- |
| `spec/document_loaders/` | Load source documents from files, APIs, stores, or custom extractors. |
| `spec/unit_parsers/` | Convert documents into a faithful `ParsedUnit` tree. |
| `spec/chunk_parsers/` | Convert semantic `Unit` objects into retrieval-ready `Chunk` records. |
| `spec/unit_savers/` | Persist units and return artifacts that can be used later. |
| `spec/chunk_savers/` | Persist chunks and return artifacts that can be used later. |
| `spec/retrievers/` | Read artifacts and decide how to retrieve evidence. |
| `spec/answer_generators/` | Generate answers from retrieved evidence. |
| `spec/evaluators/` | Score query outputs for the domain. |
| `spec/reporters/` | Write evaluation reports. |

Artifacts are intentionally storage-agnostic. A saver can write JSON, SQL,
VectorDB, a graph, an external service, or something domain-specific. The
retriever should use artifact capabilities instead of depending on a storage
implementation.

## Domain Modeling

Start by looking at the source documents, not by picking a retrieval method.
Choose units that match the author's structure:

- laws: act -> chapter -> section -> clause -> subclause
- contracts: agreement -> article -> clause -> schedule item
- policies: policy -> section -> rule -> exception
- manuals: manual -> chapter -> procedure -> step
- academic papers: paper -> section -> subsection -> paragraph/table/figure note

Good unit parsers:

- preserve hierarchy with `ParsedUnit.subunits`
- set source spans when possible
- keep `fulltext` faithful to the source
- store provenance in attributes or metadata
- reject unsupported formats visibly
- avoid catch-all parsers

Good chunk parsers:

- chunk inside a single unit
- keep rules with their exceptions and limitations
- carry citation/filter attributes
- prefer fewer useful chunks over many tiny fragments

## Minimal Unit Parser

```python
from collections.abc import Iterable

from right_rag.interfaces import UnitParser
from right_rag.model import Document, ParsedUnit


class SectionParser(UnitParser):
    @property
    def parser_name(self) -> str:
        return "sections"

    def parser_can_handle_document(self, document: Document) -> bool:
        return "Section " in document.text

    def parse_document(self, document: Document) -> Iterable[ParsedUnit]:
        yield ParsedUnit(
            topic="Section 1",
            fulltext="Exact source text for section 1",
            source_start=0,
            source_end=31,
            attributes={
                "unit_type": "section",
                "role": "rule",
                "citation": "Section 1",
            },
        )
```

## Minimal Chunk Parser

```python
from right_rag.interfaces import ChunkParser
from right_rag.model import Chunk, Unit
from right_rag.pipeline_state import stable_hash


class SemanticChunkParser(ChunkParser):
    @property
    def parser_name(self) -> str:
        return "semantic"

    def parse_unit(self, unit: Unit):
        text = f"{unit.temporal_topic}\n\n{unit.fulltext}"
        yield Chunk(
            id=stable_hash({"unit_id": unit.id, "parser": self.parser_name}),
            document_id=unit.document_id,
            unit_id=unit.id,
            topic=unit.topic,
            fulltext=text,
            attributes={
                **unit.attributes,
                "parser_name": self.parser_name,
                "source_unit": unit.temporal_topic,
            },
        )
```

## Evaluation

Evaluation is built into the pipeline:

```powershell
right-rag evaluate eval/cases.example.json --output output/evaluation/report.md
```

The default evaluator is intentionally simple. It is useful for smoke tests and
regression checks. Real projects should add domain-specific evaluators under
`spec/evaluators/`.

## Development

Clone and test the framework:

```powershell
git clone https://github.com/sunmodza/right-rag.git
cd right-rag
uv sync
python -m compileall src/right_rag
python -m unittest discover -s tests
uv build
```

Run plugin discovery:

```powershell
uv run python -c "from right_rag.registry import build_registry; r=build_registry(); print(r.document_loaders, r.unit_parsers, r.chunk_parsers, r.unit_savers, r.chunk_savers)"
```

## Release

CI runs tests and builds the package on pushes to `main` and pull requests.

Publishing runs on version tags:

```powershell
git tag v0.1.0
git push origin v0.1.0
```

PyPI Trusted Publishing is configured for:

- project: `right-rag`
- workflow: `publish.yml`
- environment: `pypi`

## Design Rules

- Put domain logic in `spec/`.
- Keep `src/right_rag/pipeline.py` focused on orchestration.
- Return `ParsedUnit` from unit parsers.
- Let the library create `Unit`.
- Treat metadata as stored context, not control flow.
- Do not couple the framework to vector search, keyword search, SQL, or graph retrieval.
- Use stdlib before adding dependencies.
