HomeProductsAgentic Chaos

Find the failure.
Prove the recovery.

Inject controlled faults into LLM and agent workflows. Test timeouts, rate limits, corrupted output, and recovery behavior before those failures reach your users.

pip install agentic-chaos
  • Open source
  • Controlled fault injection
  • Reproducible experiments
Interactive example

Watch an agent recover from a fault.

A simulated resilience experiment injects a tool timeout and shows retries, fallback, and recovery contract checks.

Resilience experiment · tool timeoutSimulated
0Injected faults
0Retry attempts
PendingContract checks
  • search_docsrunningbaseline
  • retry · search_docsstandby—
  • fallback · cached_docsstandby—
  • recovery contractpending≤ 3 attempts
RECENT ACTIVITY

Baseline started. Example scope: search_docs only; other tools remain outside the experiment.

Example data, simulated in your browser. No live telemetry or real actions.

Why Chaos Testing for Agents?

Agentic AI systems are brittle. Your agent works fine in the lab, but what happens when the LLM times out, a tool fails, or shared memory gets corrupted? Most teams don't know until production breaks.

🤐

Silent Failures Invisible to Monitoring

Your LLM returns garbage, but latency and tokens look normal. No alert fires. Chaos testing catches this.

⛓️

Cascade Failures in Multi-Agent Systems

One tool fails → Agent A fails → Agent B cascades. Without topology tracking, you miss the link.

🔄

No Recovery Testing

Your retry logic might fail under rate limits. Your turn limits might not prevent infinite loops. Prove it works.

📊

Resilience Unproven

Observability shows cost/latency. Chaos shows recovery. Together, they prove your agent is production-ready.

🚀

Framework-Agnostic

Works with any Python callable. LLM frameworks, agent frameworks, your custom code—no lock-in.

⚡

Zero Setup Overhead

pip install agentic-chaos. Wrap your calls. Run under chaos. No framework modifications needed.

How It Works

agentic-chaos operates in two layers: LLM calls and agent workflows.

Three-Layer Testing

🔌

LLM Layer (v0.1)

Test individual LLM calls — timeouts, rate limits, silent degradation. Works against any LLM, any provider.

🤖

Agent Layer (v0.2)

Test multi-agent workflows — tool failures, memory corruption, infinite loops. Tracks topology and cascades.

⚖️

Fidelity & Handoff Layer (v0.3)

Score whether corruption actually mattered and break the edge between two agents — LLM-as-judge fidelity scoring plus edge-scoped handoff corruption/drop/delay.

Architecture & Fault Injection Flow

🤖
Your Agent Code
chaos_call(fn)
➔
🌪️
Chaos Session
Active context check
➔
⚡
Fault Injection
Timeout / Storm / Degradation
➔
📋
Resilience Evidence
ChaosReport JSON

Zero-Dependency Design

Python
from agentic_chaos.chaos import chaos_call, chaos_session

# Outside a session, this is a transparent pass-through
result = llm.complete("What is AI?")

# Inside a session, faults inject
with chaos_session(["token_timeout", "silent_degradation"]):
    try:
        result = chaos_call(llm.complete, "What is AI?", faults=["token_timeout"])
    except TokenTimeoutError:
        # Handle timeout, then retry without the fault
        result = llm.complete("What is AI?")

# That's it. No instrumentation. No framework lock-in.

Composition with Observability

Optional AgenticLens integration: merge chaos events straight into workflow reports so observability and resilience testing are one.

Python
from agenticlens import profile, step
from agentic_chaos.chaos import chaos_call, chaos_session

with chaos_session(["token_timeout"]) as session:
    with profile("Support Agent") as workflow:
        with step("Retriever"):
            chunks = chaos_call(retriever.search, query, faults=["token_timeout"])

# Merge chaos events into workflow
attach_events(session, workflow)

# agenticlens analyze workflow.json
# → Shows cost/latency + chaos impact together

All Fault & Drift Types

8 resilience & drift primitives across LLM, agent, handoff, and drift layers. Extensible—add your own.

⏱️ TokenTimeout LLM

Simulates a hung/slow completion. Hangs for N seconds then times out (or succeeds late).

Modes
raise, delay
CLI
token_timeout
🌪️ RateLimitStorm LLM

Simulates a burst of 429 errors, then recovery. Tests retry logic and backoff behavior.

Stateful
Yes (counts firings)
CLI
rate_limit_storm
🤐 SilentDegradation LLM

Real latency, garbage output. Monitoring-blind failure—the hardest to detect.

Detection
Very hard
CLI
silent_degradation
🔧 ToolCallFailure Agent

Force a tool to fail—error, timeout, or return empty. Tests tool unavailability.

Modes
error, timeout, empty
CLI
tool_failure
💾 MemoryCorruption Agent

Corrupt shared agent state—truncate to 50%, inject garbage, or garble text.

Modes
truncate, inject, garble
CLI
memory_corruption
♻️ InfiniteLoop Agent

Force agent to loop past termination. Tests turn-limit safeguards.

Stateful
Yes (tracks turns)
CLI
infinite_loop
🔗 HandoffCorruption Agent

Targets the edge between two agents instead of a node — corrupt, drop, or delay a handoff payload in transit.

Modes
corrupt, drop, delay
CLI
handoff_corruption
🧭 DriftDetector v0.4.0

Snapshot and compare prompts, models, outputs & retrieval sets. Detects silent changes and fails CI (exit code 2).

Signals
prompt, model, output
CLI
drift compare

Quick Start (5 minutes)

Install, wrap your calls, run under chaos.

1. Install

bash
pip install agentic-chaos

2. Wrap Your LLM Calls

Python
from agentic_chaos.chaos import chaos_call, chaos_session, TokenTimeoutError

with chaos_session(["token_timeout"]):
    try:
        result = chaos_call(
            llm.complete,
            "What is the weather?",
            faults=["token_timeout"]
        )
    except TokenTimeoutError:
        print("LLM timed out—handled gracefully")

3. Run from CLI

bash
agentic-chaos chaos run my_app.py \
  --inject token_timeout,rate_limit_storm,silent_degradation \
  --save chaos_report.json

# Output:
# Chaos Events
# Step         Fault                 Outcome    Message
# ────────────────────────────────────────────────────────
# Retriever    token_timeout         errored    call hung for 2.0s then timed out
# LLMCall      silent_degradation    degraded   output silently degraded

# Saved chaos report to chaos_report.json

4. View Reports

The report is standalone JSON — load it, analyze it, integrate with your observability stack.

JSON
{
  "chaos_events": [
    {
      "fault_type": "token_timeout",
      "step_name": "Retriever",
      "outcome": "errored",
      "message": "call hung for 2.0s then timed out"
    }
  ]
}

Ready to Test Your Agent's Resilience?

Clone the repo, run a quickstart example, or check out the full documentation.

Roadmap

From v0.1 shipped to v1.0 (ChaosHub community registry).

v0.1
✓ Shipped
LLM Chaos Toolkit (3 faults, CLI, AgenticLens adapter)
v0.2
✓ Shipped (2026-07-13)
Agent Failure Injector (3 agent faults, topology tracking, framework-agnostic wrap_tool()/wrap_node() helpers)
v0.3
✓ Shipped
Fidelity Judges & Handoff Chaos (LLM-as-judge scoring, edge-scoped faults)
v0.4
✓ Shipped (2026-08-15)
Prompt/Model Drift Detector (snapshot, compare, detect silent changes)
v0.5
✓ Shipped (0.5.0–0.5.3)
Streaming Faults, Provider Patching, Chaos Profiles & Safety Rails
v0.6
✓ Shipped (0.6.0)
Pytest Plugin & Contracts (pytest --chaos, 7 pass/fail contracts, CI gating)
v0.7
✓ Shipped (0.7.0)
Fault Cascades, Adaptive Intensity & Response Poisoning
v0.8
✓ Shipped (0.8.0)
Chaos Workflows, Declarative Experiments & Explosion Radius (gated campaigns, YAML lint, Scope, topology fuzzer)
v0.9
✓ Shipped (0.9.0)
Resilience Probes & Resilience Score (pre/during/post probes, 0–100 score, trends with retention, --min-score CI gate)
v1.0
🚧 Planned
ChaosHub (Shared Experiment Registry)

Each version is a minor bump on the same package — incremental PyPI releases with sustained activity.
View full roadmap →

How agentic-chaos Compares

What makes agentic-chaos unique in the market.

Feature ChaosEater Fault Injection FW Observability Tools agentic-chaos
Zero Dependencies ✗ ✗ ✗ ✅
LLM Faults ✗ ✓ ✓ ✅ (3 types)
Agent Faults ✗ Limited Limited ✅ (3 types)
Silent Degradation Not mentioned Implicit Implicit ✅ Explicit
Topology Tracking ✗ ✗ Tracing only ✅ Graph + cascades
Framework Agnostic Kubernetes only Partially Framework-specific ✅ Any Python callable
Production Ready Research Emerging Mature ✅ Shipped v0.3

Put Agentic Chaos to work.

Start with the documentation, explore the examples, or contribute on GitHub.