RAGOps GitHub

Open source · offline · provider-independent

Release RAG changes with evidence.

Catch regressions in prompts, retrievers, embeddings, datasets, and evaluators before they ship. Compare a candidate with an accepted baseline and get an explainable PASS or BLOCK.

Five-minute proofNo account · no API key
pip install ragops==1.0.0
ragops demo --output ragops-demo
Read the guide

Creates local JSON, Markdown, and HTML evidence.

Recorded regressionGraph + ACL → lexical only
ACCEPTED BASELINEPASSMeets release policy
CANDIDATEBLOCK3 named gates failed
Versioned policyCompare
Citation coverage−25.00%
Citation precision−25.00%
Lexical groundedness−21.88%
Portable evidenceJSON · Markdown · HTML

One reviewable release loop

From recorded behavior to a decision.

RAGOps stays outside your application. Your model, retriever, and orchestration remain yours; the dependency-free core evaluates portable evidence against a versioned contract.

  1. 01Record

    Answers, citations, retrievals, latency, cost, and metadata.

  2. 02Evaluate

    Deterministic checks, critical findings, and optional external metrics.

  3. 03Compare

    Accepted baseline versus candidate with explicit tolerances.

  4. 04Gate

    Named PASS or BLOCK reasons for people and CI.

Open-source coreScenarios, traces, evaluators, comparison, reports, release gates
Optional adaptersAPI, browser workbench, providers, local control-plane alpha

Evidence, not decoration

Two experiments. Two bounded claims.

4-case reference deployment

Graph-assisted baseline passes. Lexical-only candidate is blocked.

Citation coverage
−25.00%
Citation precision
−25.00%
Lexical groundedness
−21.88%
Open reference comparison →

30-case synthetic harness benchmark

Baseline passes. Regressed and adversarial candidates are blocked.

Failure families
9
Critical findings
5
Blocked candidates
2
Open benchmark report →

Evidence boundary: these synthetic results validate the harness and the recorded architecture comparison. They do not establish semantic correctness, production security, customer adoption, or ROI.

Known limits

Every claim keeps its boundary.

01Lexical is not semantic

Lexical groundedness is overlap, not entailment or human judgment.

02Synthetic is not adoption

Fixture results do not prove usage, activation, retention, or ROI.

03Reference ACL is not identity

The role list is not enterprise SSO, RBAC, or a production boundary.

04Local alpha is not SaaS

The control plane is a development surface, not hosted infrastructure.

MIT License · zero hosted fonts · local-first

Make the next RAG change prove it is ready.

Run the demoStar or fork