Metadata-Version: 2.4
Name: bendex-resilience
Version: 0.1.1
Summary: Tests whether an AI agent or stateful system recovers or cascades after a perturbation, using twin-trajectory divergence analysis. Works with minimal historical data and no model of the system's internals.
Author: Hannah Nine
License: MIT
Project-URL: Homepage, https://github.com/9hannahnine-jpg/bendex-resilience
Project-URL: Repository, https://github.com/9hannahnine-jpg/bendex-resilience
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.10
Provides-Extra: api
Requires-Dist: fastapi>=0.100; extra == "api"
Requires-Dist: uvicorn>=0.23; extra == "api"
Dynamic: license-file

# Bendex Resilience

Tests whether an AI agent (or any stateful system) **recovers or cascades**
after a perturbation — a corrupted memory entry, a failed tool call, a
config change — using twin-trajectory divergence analysis.

## What makes this different

Most AI agent testing tools inject a failure and check one thing: did the
task still complete? That's a pass/fail light switch — it can't tell the
difference between "acting weird but fine" and "acting weird and actually
breaking."

This tool instead:
1. Clones your system into two copies.
2. Applies a small, controlled perturbation to one copy.
3. Runs both copies through the same real workload.
4. Watches whether their behavior stays in sync (**recovers**) or drifts
   apart (**cascades**).

**No historical baseline required before you can test it.** No model of
your system's internals needed. It works by comparing the two live copies
to each other, not by comparing to weeks of past data.

## Validated results

Tested against a real language model (Qwen2.5-0.5B-Instruct), not just
synthetic data — see [`colab_shareable_demo.py`](https://github.com/9hannahnine-jpg/bendex-resilience/blob/main/colab_shareable_demo.py),
runnable yourself in one click:

| Metric | Result |
|---|---|
| Accuracy (40 trials: 20 healthy, 20 real injected failures) | **95.0%** (38/40) |
| Healthy systems correctly left alone | 18/20 (2/20 false alarms) |
| Real failures caught | 20/20 |

Also validated on systems where classical stability-analysis methods don't
apply at all — nonlinear systems (90% accuracy against an exact known
answer) and systems with memory/delay (100% accuracy against an exact known
answer). Also validated to work with far less historical data than standard
covariance-based drift detection methods require.

## Known limitations — stated plainly, not hidden

- **Detection quality depends on calibration data size.** Measured: 62%
  accuracy at 7 calibration observations, 83% at 25, 95% at 40+. Use at
  least ~40 observations where possible.
- **Does not yet automatically find your system's worst-case
  vulnerability.** You tell it what scenario to test; it doesn't discover
  unknown weaknesses on its own. This is open, unsolved work.
- **Does not localize which specific subsystem is failing** — tells you
  *that* something broke, not *which part* of a multi-component system.

## Quick start

```python
from resilience_v2 import ResilienceDetector

detector = ResilienceDetector()
detector.calibrate(nominal_trajectory)   # your system's normal behavior
result = detector.check(test_trajectory) # behavior after a perturbation
print(result)  # {'verdict': 'RECOVERS' or 'CASCADES', 'flagged_at_step': ..., ...}
```

Or run it as an API — see [`api.py`](https://github.com/9hannahnine-jpg/bendex-resilience/blob/main/api.py)
(FastAPI, single `/analyze` endpoint, stateless).

## Files

- `resilience_v2.py` — core detector
- `api.py` — REST API wrapper
- `colab_shareable_demo.py` — real-LLM validation demo, runnable in Colab
- `tests/` — stress tests (scale, edge cases, false-positive rate)

## License

MIT
