HomeProductsAgentic Chaos
Agentic Chaos · Stress-test
Find the failure.
Prove the recovery.
Inject controlled faults into LLM and agent workflows. Test timeouts, rate limits, corrupted output, and recovery behavior before those failures reach your users.
pip install agentic-chaos
- Open source
- Controlled fault injection
- Reproducible experiments
Watch an agent recover from a fault.
A simulated resilience experiment injects a tool timeout and shows retries, fallback, and recovery contract checks.
- search_docsrunningbaseline
- retry · search_docsstandby—
- fallback · cached_docsstandby—
- recovery contractpending≤ 3 attempts
Baseline started. Example scope: search_docs only; other tools remain outside the experiment.
Example data, simulated in your browser. No live telemetry or real actions.
Why Chaos Testing for Agents?
Agentic AI systems are brittle. Your agent works fine in the lab, but what happens when the LLM times out, a tool fails, or shared memory gets corrupted? Most teams don't know until production breaks.
Silent Failures Invisible to Monitoring
Your LLM returns garbage, but latency and tokens look normal. No alert fires. Chaos testing catches this.
Cascade Failures in Multi-Agent Systems
One tool fails → Agent A fails → Agent B cascades. Without topology tracking, you miss the link.
No Recovery Testing
Your retry logic might fail under rate limits. Your turn limits might not prevent infinite loops. Prove it works.
Resilience Unproven
Observability shows cost/latency. Chaos shows recovery. Together, they prove your agent is production-ready.
Framework-Agnostic
Works with any Python callable. LLM frameworks, agent frameworks, your custom code—no lock-in.
Zero Setup Overhead
pip install agentic-chaos. Wrap your calls. Run under chaos. No framework modifications needed.
How It Works
agentic-chaos operates in two layers: LLM calls and agent workflows.
Three-Layer Testing
LLM Layer (v0.1)
Test individual LLM calls — timeouts, rate limits, silent degradation. Works against any LLM, any provider.
Agent Layer (v0.2)
Test multi-agent workflows — tool failures, memory corruption, infinite loops. Tracks topology and cascades.
Fidelity & Handoff Layer (v0.3)
Score whether corruption actually mattered and break the edge between two agents — LLM-as-judge fidelity scoring plus edge-scoped handoff corruption/drop/delay.
Architecture & Fault Injection Flow
chaos_call(fn)ChaosReport JSONZero-Dependency Design
from agentic_chaos.chaos import chaos_call, chaos_session
# Outside a session, this is a transparent pass-through
result = llm.complete("What is AI?")
# Inside a session, faults inject
with chaos_session(["token_timeout", "silent_degradation"]):
try:
result = chaos_call(llm.complete, "What is AI?", faults=["token_timeout"])
except TokenTimeoutError:
# Handle timeout, then retry without the fault
result = llm.complete("What is AI?")
# That's it. No instrumentation. No framework lock-in.
Composition with Observability
Optional AgenticLens integration: merge chaos events straight into workflow reports so observability and resilience testing are one.
from agenticlens import profile, step
from agentic_chaos.chaos import chaos_call, chaos_session
with chaos_session(["token_timeout"]) as session:
with profile("Support Agent") as workflow:
with step("Retriever"):
chunks = chaos_call(retriever.search, query, faults=["token_timeout"])
# Merge chaos events into workflow
attach_events(session, workflow)
# agenticlens analyze workflow.json
# → Shows cost/latency + chaos impact together
All Fault & Drift Types
8 resilience & drift primitives across LLM, agent, handoff, and drift layers. Extensible—add your own.
Simulates a hung/slow completion. Hangs for N seconds then times out (or succeeds late).
Simulates a burst of 429 errors, then recovery. Tests retry logic and backoff behavior.
Real latency, garbage output. Monitoring-blind failure—the hardest to detect.
Force a tool to fail—error, timeout, or return empty. Tests tool unavailability.
Corrupt shared agent state—truncate to 50%, inject garbage, or garble text.
Force agent to loop past termination. Tests turn-limit safeguards.
Targets the edge between two agents instead of a node — corrupt, drop, or delay a handoff payload in transit.
Snapshot and compare prompts, models, outputs & retrieval sets. Detects silent changes and fails CI (exit code 2).
Quick Start (5 minutes)
Install, wrap your calls, run under chaos.
1. Install
pip install agentic-chaos
2. Wrap Your LLM Calls
from agentic_chaos.chaos import chaos_call, chaos_session, TokenTimeoutError
with chaos_session(["token_timeout"]):
try:
result = chaos_call(
llm.complete,
"What is the weather?",
faults=["token_timeout"]
)
except TokenTimeoutError:
print("LLM timed out—handled gracefully")
3. Run from CLI
agentic-chaos chaos run my_app.py \ --inject token_timeout,rate_limit_storm,silent_degradation \ --save chaos_report.json # Output: # Chaos Events # Step Fault Outcome Message # ──────────────────────────────────────────────────────── # Retriever token_timeout errored call hung for 2.0s then timed out # LLMCall silent_degradation degraded output silently degraded # Saved chaos report to chaos_report.json
4. View Reports
The report is standalone JSON — load it, analyze it, integrate with your observability stack.
{
"chaos_events": [
{
"fault_type": "token_timeout",
"step_name": "Retriever",
"outcome": "errored",
"message": "call hung for 2.0s then timed out"
}
]
}
Ready to Test Your Agent's Resilience?
Clone the repo, run a quickstart example, or check out the full documentation.
Roadmap
From v0.1 shipped to v1.0 (ChaosHub community registry).
Each version is a minor bump on the same package — incremental PyPI releases with sustained activity.
View full roadmap →
How agentic-chaos Compares
What makes agentic-chaos unique in the market.
| Feature | ChaosEater | Fault Injection FW | Observability Tools | agentic-chaos |
|---|---|---|---|---|
| Zero Dependencies | ✗ | ✗ | ✗ | ✅ |
| LLM Faults | ✗ | ✓ | ✓ | ✅ (3 types) |
| Agent Faults | ✗ | Limited | Limited | ✅ (3 types) |
| Silent Degradation | Not mentioned | Implicit | Implicit | ✅ Explicit |
| Topology Tracking | ✗ | ✗ | Tracing only | ✅ Graph + cascades |
| Framework Agnostic | Kubernetes only | Partially | Framework-specific | ✅ Any Python callable |
| Production Ready | Research | Emerging | Mature | ✅ Shipped v0.3 |
Start building
Put Agentic Chaos to work.
Start with the documentation, explore the examples, or contribute on GitHub.