Metadata-Version: 2.5
Name: ragops
Version: 2.3.1
Summary: Regression tests and explainable release gates for RAG and AI agents
Project-URL: Homepage, https://github.com/thangldw/ragops
Project-URL: Documentation, https://github.com/thangldw/ragops#readme
Project-URL: Issues, https://github.com/thangldw/ragops/issues
Project-URL: Changelog, https://github.com/thangldw/ragops/blob/main/CHANGELOG.md
Author: Thang Luu
License: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: ai-agents,evaluation,rag,regression-testing,release-gates
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Provides-Extra: api
Requires-Dist: fastapi>=0.115; extra == 'api'
Requires-Dist: uvicorn>=0.30; extra == 'api'
Provides-Extra: dev
Requires-Dist: coverage>=7.6; extra == 'dev'
Requires-Dist: fastapi>=0.115; extra == 'dev'
Requires-Dist: httpx>=0.27; extra == 'dev'
Requires-Dist: jsonschema>=4.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.6; extra == 'dev'
Provides-Extra: release
Requires-Dist: build>=1.2; extra == 'release'
Requires-Dist: cyclonedx-bom>=7.3; extra == 'release'
Requires-Dist: twine>=5.1; extra == 'release'
Description-Content-Type: text/markdown

# RAGOps

[English](#english) · [Tiếng Việt](#tiếng-việt) · [日本語](#日本語)

Offline, reproducible release gates for RAG systems and AI agents.

```mermaid
%%{init: {"theme":"base","themeVariables":{"background":"#FFFFFF","fontFamily":"Arial, sans-serif","lineColor":"#667085","primaryTextColor":"#172B4D"}}}%%
flowchart LR
    S["Scenario<br/>Kịch bản / シナリオ"]:::yellow
    B["Accepted baseline<br/>Mốc chuẩn / 基準"]:::blue
    C["Candidate traces<br/>Ứng viên / 候補"]:::purple
    E["Evaluators & policy<br/>Đánh giá / 評価"]:::green
    G{"PASS / WARN / BLOCK"}:::pink
    S --> E
    B --> E
    C --> E --> G
    classDef yellow fill:#FFF4A3,stroke:#C9A227,stroke-width:2px,color:#172B4D
    classDef blue fill:#D9EAFD,stroke:#4C78A8,stroke-width:2px,color:#172B4D
    classDef purple fill:#E9DDF7,stroke:#8064A2,stroke-width:2px,color:#172B4D
    classDef green fill:#DDF5E3,stroke:#4F9D69,stroke-width:2px,color:#172B4D
    classDef pink fill:#FFE1E6,stroke:#C96A7B,stroke-width:2px,color:#172B4D
```

## English

Product introduction: [thangldw.github.io/ragops/](https://thangldw.github.io/ragops/). Latest published release: [v2.3.1](https://github.com/thangldw/ragops/releases/tag/v2.3.1). Main may contain later source or documentation changes; they do not replace the published artifact or its recorded verification.

RAGOps compares recorded candidate behavior with an accepted baseline, applies a versioned release policy, and emits one canonical `PASS`, `WARN`, or `BLOCK` decision across JSON, Markdown, HTML, JUnit, SARIF, and GitHub Summary. The dependency-free core performs no network calls. Optional adapters import vendor exports; the API/workbench ships in the wheel through the `api` extra.

Requirements: Python 3.11+.

Current technical references: [architecture](docs/ARCHITECTURE.md), [contracts](docs/CONTRACTS.md), [governance](docs/governance.md), [operations](docs/OPERATIONS.md), [security](SECURITY.md), and [v2.3.1 release notes](docs/releases/v2.3.1.md).

The repository is also a skills-only plugin for ChatGPT, Codex, Claude Code and Cowork. It does not include a hosted MCP connector. See the [directory submission package](docs/submission/DIRECTORY_SUBMISSION.md), [privacy policy](PRIVACY.md), [terms](TERMS.md), and [support guidance](SUPPORT.md).

```bash
pip install ragops
ragops demo --profile engineer --output ./demo-output
ragops evidence verify --bundle ./demo-output/evidence
open ./demo-output/release-report.html
```

On Windows PowerShell, replace the final command with
`Start-Process ./demo-output/release-report.html`.

The credential-free demo intentionally returns `BLOCK`: citation coverage `1.0 -> 0.5`, citation precision `1.0 -> 0.5`, and lexical groundedness `1.0 -> 0.6`. It is synthetic benchmark evidence, not production adoption evidence.

Install `ragops[api]` and run `ragops serve` for the local authenticated API/workbench. Use `ragops adapter list` for Phoenix, Ragas, DeepEval, LangSmith, MLflow, Promptfoo, custom JSON, and installed entry-point adapters.

`lexical_groundedness` is a transparent lexical-overlap baseline, not a semantic judge: a correct paraphrase can score low and a wrong answer that reuses evidence vocabulary can score high. Japanese/Chinese text is compared as character bigrams and Vietnamese is NFC-normalized. For a stricter deterministic check (claim-level, exact numbers, negation) use `--evaluator strict_grounding`; measured accuracy and known failures are in [evaluator accuracy](docs/evaluator-accuracy.md). For semantic grounding, import Ragas/DeepEval scores through the adapters and calibrate against human labels.

Local CLI usage evidence is disabled by default. Set `RAGOPS_USAGE_LOG` to an explicit JSONL path to record only the UTC timestamp, RAGOps version, top-level command, and exit code; arguments and file paths are never recorded. Summarize the file with `ragops usage-report --events usage.jsonl`. Use the consent-aware `pilot-report` contract and the [production pilot runbook](docs/gtm/production-pilot.md) for production adoption evidence.

## Tiếng Việt

Trang giới thiệu: [thangldw.github.io/ragops/](https://thangldw.github.io/ragops/). Release đã phát hành mới nhất: [v2.3.1](https://github.com/thangldw/ragops/releases/tag/v2.3.1). Main có thể chứa source hoặc tài liệu mới hơn; chúng không thay thế artifact đã phát hành hay evidence kiểm chứng của artifact.

RAGOps so sánh trace ứng viên với baseline được chấp nhận, áp dụng release policy có version và xuất cùng một quyết định `PASS`, `WARN` hoặc `BLOCK` sang JSON/Markdown/HTML/JUnit/SARIF. Core không dependency, chạy offline; API/workbench là extra tùy chọn. Demo synthetic cố ý tạo `BLOCK`, không phải bằng chứng production adoption.

Yêu cầu: Python 3.11+. Dùng các lệnh ở phần English để cài đặt, kiểm tra scenario, lint và test.

## 日本語

紹介ページ: [thangldw.github.io/ragops/](https://thangldw.github.io/ragops/)。最新の公開 release は [v2.3.1](https://github.com/thangldw/ragops/releases/tag/v2.3.1) です。Main の新しい source/documentation change は公開 artifact やその検証記録を置換しません。

RAGOps は記録済み候補を承認済みベースラインと比較し、バージョン管理された方針から一つの `PASS`・`WARN`・`BLOCK` 判定を JSON、Markdown、HTML、JUnit、SARIF に出力します。依存関係のない core はオフラインで動作し、API/workbench は任意の extra です。デモは synthetic evidence であり、本番導入実績ではありません。

必要環境は Python 3.11 以上です。セットアップ、lint、テストには English セクションのコマンドを使用してください。

Released under the [Apache License 2.0](LICENSE).
