Metadata-Version: 2.4
Name: crust-score
Version: 0.1.0
Summary: CRUS-T: a normalized, programmatically computable runtime reliability score for agentic AI, per Ghosh Dastidar, 'Proportionate Governance in Practice'.
Author: Satyaki Ghosh Dastidar
License: MIT
Project-URL: Homepage, https://github.com/satyakigd/crust
Project-URL: Repository, https://github.com/satyakigd/crust
Keywords: agent evaluation,AI governance,model risk management,reliability scoring,agentic AI
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: pytest-cov>=4.1; extra == "dev"
Dynamic: license-file

# crust-score

A reference implementation of **CRUS-T score**: a normalized, programmatically
computable runtime reliability score for agentic AI, from

> Satyaki Ghosh Dastidar, *Proportionate Governance in Practice: An
> IDEA–CRUS-T Framework for Risk-Tiered, Runtime-Governed AI Systems in
> Financial Services*. Preprint, 2026.

CRUS-T scores an AI capability or execution trajectory on four dimensions —
**C**onsistency, **R**obustness, **U**ncertainty awareness, and **S**afety —
each built from three standard, auditable sub-metrics, aggregated with a
collapse-sensitive weighted geometric mean, and gated on a conservative
statistical bound rather than a point estimate. Safety additionally carries
a discontinuous hard gate: a single critical policy violation zeroes the
governed safety score regardless of the continuous score, so it can never
be averaged away.

This library implements the score itself (Section 5 of the paper) plus a
thin tier-gating layer on top of it (Section 5.6–5.7) that decides
continue/escalate/contain against a threshold vector you supply. It does
**not** implement the paper's separate five-dimension risk-tiering method
(Section 4 — Authority/Impact/Agency/Exposure/Recoverability → control
tier); that assigns a *design-time* control tier, whereas this library is
about the *runtime* reliability score that verifies a tier's assumptions
continue to hold in production.

## Status

Early (`0.1.0`). The math is unit-tested against the paper's equations and
against the worked example in Appendix C. The weights, tolerances, and
thresholds you configure are governance parameters, not statistical
defaults — see [Governance parameters](#governance-parameters) below
before using this for anything with real consequence.

## Install

```bash
pip install -e .          # from a local checkout
pip install -e ".[dev]"   # + pytest, for running the test suite
```

Requires Python 3.10+. No required runtime dependencies.

## How the code maps to the paper

| Paper section | Module |
| --- | --- |
| 5.1 Design principles and normalization primitives | `crus_t.primitives` |
| 5.2 Consistency (C1/C2/C3) | `crus_t.metrics.consistency` |
| 5.3 Robustness (R1/R2/R3) | `crus_t.metrics.robustness` |
| 5.4 Uncertainty awareness (U1/U2/U3) | `crus_t.metrics.uncertainty` |
| 5.5 Safety (S1/S2/S3) + critical-violation hard gate | `crus_t.metrics.safety` |
| Eqs. 8/12/16/20-21 — dimension aggregation | `crus_t.dimensions` |
| Eq. 22 — composite score | `crus_t.score` |
| Eq. 24 — bootstrap lower-confidence-bound gating | `crus_t.bootstrap` |
| 5.6-5.7 — tier gate, runtime zones (continue/escalate/contain) | `crus_t.gating` |
| Per-run scoring | `crus_t.run` |
| Rolling many runs up into a batch/portfolio view | `crus_t.aggregate` |
| Governance-owned weights/tolerances | `crus_t.config` |

Every public function's docstring cites the specific equation it
implements, so you can go from a number in the paper straight to the code
that produces it (and back).

## What this library does *not* do for you

Several CRUS-T sub-metrics depend on a learned component the paper
deliberately keeps out of the governed math: an entailment model for
groundedness (S1), an embedding or entailment model for semantic
similarity (C1/C2), a task-quality function `Q(.)` for Robustness. This
library takes the *outputs* of those components (an entailment boolean, a
similarity score, a quality score) as plain function arguments — it does
not call any model itself. Section 5.5 of the paper is explicit about why:
"a probabilistic verifier operating in the same semantic space as the
system it polices is architecturally blind to fluent-but-wrong outputs,"
so whatever model you plug in needs to be validated and governed
separately, on its own precision/recall for the violation class it's
detecting.

## Quickstart

```python
from crus_t import (
    compute_consistency, compute_robustness, compute_uncertainty,
    governed_safety, PolicyObservation, safety_to_dimension_score,
    ReliabilityVector, composite,
    ThresholdVector, evaluate_gate,
    make_run_score, aggregate_run_scores,
)

# 1. Compute each dimension from its three sub-metrics (already observed
#    from your evaluation harness / production telemetry).
c = compute_consistency(c1=0.94, c2=0.88, c3=0.97)
r = compute_robustness(r1=0.85, r2=0.96, r3=0.80)
u = compute_uncertainty(u1=0.89, u2=0.81, u3=0.85)

safety_result = governed_safety(
    entailed_claims=98, total_claims=100,
    policy_observations=[PolicyObservation("no_stale_citation", severity=1.0, violation_count=0)],
    n_trajectories=100,
    authorized_calls=10, total_external_calls=10,
    critical_violation_observed=False,
    tau_safety=0.10,
)
s = safety_to_dimension_score(safety_result)

vector = ReliabilityVector(c, r, u, s)
print(vector.as_dict())          # {"C": 0.93, "R": 0.87, "U": 0.85, "S": ...}
print(composite(vector))         # single reporting number, Eq. (22)

# 2. Package it as a run and gate it against a tier's thresholds. Threshold
#    values are always your own governance input -- this library has no
#    built-in per-tier table.
thresholds = ThresholdVector(consistency=0.80, robustness=0.75, uncertainty=0.78, safety=0.90)
gate = evaluate_gate(vector.as_dict(), thresholds)
run = make_run_score(run_id="run-001", capability_id="drafting", vector=vector, gate=gate)
print(gate.zone)                 # RuntimeZone.CONTINUE / ESCALATE / CONTAIN

# 3. Later, roll many runs up into a batch view.
agg = aggregate_run_scores([run, ...])
print(agg.escalation_rate, agg.containment_rate, agg.axis_summaries["safety"].mean)
```

For a regulatory-grade decision, gate on a bootstrap lower bound instead of
the point estimate (Eq. 24):

```python
from crus_t import bootstrap_lower_bound

# `statistic` recomputes a dimension score from a resampled item list --
# wire this to your own evaluation-set records.
result = bootstrap_lower_bound(evaluation_items, statistic=my_dimension_statistic, seed=42)
gate = evaluate_gate({"C": result.lower_bound, ...}, thresholds)
```

See `examples/worked_example.py` for a runnable version of Appendix C's
Drafting-capability example, and `examples/quickstart.py` for the snippet
above.

## Governance parameters

Every tolerance (`tau`), every sub-metric weight, every threshold vector,
and the bootstrap confidence level are, per Section 5.1 of the paper,
*governance parameters*: "set by the second line of defense from pooled
validation evidence, versioned, and subject to challenge — not free
parameters tuned to produce a desired outcome." `crus_t.config.GovernanceConfig`
gives them one explicit, serializable home; `ThresholdVector` in
`crus_t.gating` is a plain, explicit input with no built-in defaults. This
library will not stop you from tuning a threshold until a report looks
good — that governance discipline lives with your process, not the code.

## Development

```bash
pip install -e ".[dev]"
pytest                 # run the test suite
pytest --cov=crus_t    # with coverage
```

## License

MIT. See `LICENSE`.
