Metadata-Version: 2.4
Name: rai-audit-kit
Version: 0.1.8
Summary: Responsible AI (RAI) Audit Kit — evidence-grade audits for responsible, secure, and trustworthy AI systems
Author: Sai Teja Erukude
License: MIT License
        
        Copyright (c) 2026 Sai Teja Erukude
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: audit,fairness,responsible-ai,robustness,security,trustworthy-ai
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: rai-audit-core>=0.1.0
Requires-Dist: rai-audit-ml>=0.1.0
Provides-Extra: agents
Requires-Dist: rai-audit-agents>=0.1.0; extra == 'agents'
Provides-Extra: all
Requires-Dist: rai-audit-agents>=0.1.0; extra == 'all'
Requires-Dist: rai-audit-dl>=0.1.0; extra == 'all'
Requires-Dist: rai-audit-llm>=0.1.0; extra == 'all'
Provides-Extra: dl
Requires-Dist: rai-audit-dl>=0.1.0; extra == 'dl'
Provides-Extra: llm
Requires-Dist: rai-audit-llm>=0.1.0; extra == 'llm'
Description-Content-Type: text/markdown

# RAI Audit Kit

**RAI** = **Responsible AI**. A Python package suite for evidence-grade audits of
responsible, secure, and trustworthy AI systems.

Run fairness, data quality, robustness, compliance, image, medical imaging, LLM
safety, RAG security, and agent trace checks. Export HTML, Markdown, or JSON
reports and gate CI pipelines on risk thresholds.

**Author:** Sai Teja Erukude | **License:** MIT

## Why this exists

AI teams often run fairness, robustness, RAG, and agent security checks separately.
RAI Audit Kit brings them into one evidence and reporting workflow, so teams can
review findings consistently, preserve audit artifacts, and apply the same CI gates
across model types.

## What it looks like

<table>
  <tr>
    <td><strong>HTML audit report</strong><br>
      <a href="https://raw.githubusercontent.com/SaiTeja-Erukude/rai-audit/main/docs/images/html-report.png">
        <img src="https://raw.githubusercontent.com/SaiTeja-Erukude/rai-audit/main/docs/images/html-report.png" alt="HTML fairness audit report" width="560">
      </a>
    </td>
    <td><strong>Model card export</strong><br>
      <a href="https://raw.githubusercontent.com/SaiTeja-Erukude/rai-audit/main/docs/images/model-card.png">
        <img src="https://raw.githubusercontent.com/SaiTeja-Erukude/rai-audit/main/docs/images/model-card.png" alt="Markdown model card preview" width="560">
      </a>
    </td>
  </tr>
  <tr>
    <td><strong>LLM and RAG audit output</strong><br>
      <a href="https://raw.githubusercontent.com/SaiTeja-Erukude/rai-audit/main/docs/images/rag-audit.png">
        <img src="https://raw.githubusercontent.com/SaiTeja-Erukude/rai-audit/main/docs/images/rag-audit.png" alt="RAG security audit output" width="560">
      </a>
    </td>
    <td><strong>Agent trace finding</strong><br>
      <a href="https://raw.githubusercontent.com/SaiTeja-Erukude/rai-audit/main/docs/images/agent-trace-finding.png">
        <img src="https://raw.githubusercontent.com/SaiTeja-Erukude/rai-audit/main/docs/images/agent-trace-finding.png" alt="Agent trace prompt injection finding" width="560">
      </a>
    </td>
  </tr>
</table>

## Packages

| Package | Purpose |
|---------|---------|
| `rai-audit-core` | Audit engine, findings, reports, history, CI gates |
| `rai-audit-ml` | Tabular ML - fairness, drift, data quality, robustness |
| `rai-audit-dl` | Image, medical imaging, and scientific AI audits |
| `rai-audit-llm` | LLM and RAG safety, faithfulness, citation, and security audits |
| `rai-audit-agents` | Agent tool-use, memory, permission, and injection audits |
| `rai-audit-kit` | Meta-package - installs core + ml, unified CLI |

## Install

```bash
pip install rai-audit-kit          # core + tabular ML
pip install "rai-audit-kit[all]"   # all modules (dl, llm, agents)
```

## Quick start

```bash
rai-audit ml run --help
```

For repeatable audit workflows, generate and run a YAML configuration:

```bash
rai-audit init --project loan-model
rai-audit run --config audit.yaml
```

Configured runs write report artifacts and an evidence manifest with input,
environment, source-revision, and artifact hashes.

```python
from rai_audit.ml import ClassificationAudit

report = ClassificationAudit(
    y_true=y_true,
    y_pred=y_pred,
    sensitive_features=sensitive_df,
).run()

report.to_html("audit_report.html")
```

## Examples

- [Fairness audit walkthrough](https://github.com/SaiTeja-Erukude/rai-audit/blob/main/packages/rai-audit-ml/examples/ml_fairness_audit/example.py)
- [Batch drift monitoring](https://github.com/SaiTeja-Erukude/rai-audit/blob/main/packages/rai-audit-ml/examples/ml_drift_monitoring/batch_monitor.py)
- [MLflow and Airflow templates](https://github.com/SaiTeja-Erukude/rai-audit/tree/main/packages/rai-audit-ml/examples/mlops_integrations/)
- [Captured-response LLM and RAG audit suite](https://github.com/SaiTeja-Erukude/rai-audit/blob/main/packages/rai-audit-llm/examples/llm_audit_suite.yml)
- [Scientific image robustness audit](https://github.com/SaiTeja-Erukude/rai-audit/blob/main/packages/rai-audit-dl/examples/scientific_ai/microscopy_audit.py)
- [Agent trace with a webpage prompt-injection attempt](https://github.com/SaiTeja-Erukude/rai-audit/blob/main/packages/rai-audit-agents/examples/customer_support_trace.json)

## Development

```bash
pip install uv
uv sync
uv run pytest
```

See [CONTRIBUTING.md](https://github.com/SaiTeja-Erukude/rai-audit/blob/main/CONTRIBUTING.md)
for monorepo layout and release workflow.
