An agent whose job is to supervise other agents
Production LLM agents fail silently — they invent policies, drift after a model upgrade, paper over broken tools. Today a human catches that by staring at dashboards. Cassandra automates the whole loop: it watches another agent through its traces, catches the failure, turns it into an eval dataset, scores a fix, and proposes a patch — all on its own.
Two separate agents, joined only by telemetry
The Patient ("ShopBot") is a fragile customer-support agent. Cassandra never touches its code — it watches it entirely through Arize Phoenix traces, exactly as a real supervisor would. The only back-channel is a sandboxed session_id="test" path used for replay/eval, which the Watcher filters out so Cassandra can never supervise itself into a loop.
One incident, eight stages — press play
Every failing trace flows through the same pipeline. A single Incident object threads through it, picking up a verdict, a root cause, an eval dataset, pass-rates, a candidate prompt, and a replay result. Step through each stage to see its input, what it does, its output, and the exact Phoenix MCP tool it calls.
What Cassandra actually catches
The canonical demo: a customer asks about a region with no policy data. The fragile prompt tells ShopBot to "never say you don't know" — so it fabricates. Cassandra catches it, patches the prompt, and replays the exact same input to prove the fix.
It doesn't just use MCP — it publishes its own
Cassandra consumes the partner Phoenix MCP, then turns around and exposes its supervision as tools any agent or IDE (Claude Desktop, Cursor) can call. Pick a tool to watch the JSON request flow in, the reasoning happen, and the JSON response flow out.
Cassandra grades its own diagnoses
The Arize track rewards agents that use their own observability to improve. Cassandra runs a hand-labeled trap library through the live Patient and its own Diagnostician, then scores its verdicts against ground truth — a diagnostic-accuracy scorecard. The watcher, watching itself.
Why this is more than "another agent"
Recursive by design
Almost every entry is an agent. Cassandra is an agent about agents — it supervises the exact workflow Phoenix was built for, automated and continuous.
Deep MCP surface
It exercises spans, annotations, datasets, prompt versions and experiments on the partner MCP — and publishes five of its own tools on top.
Honest numbers
Evaluation runs live against the agent (Phoenix MCP has no run-experiment tool), and the self-eval scorecard is the real thing — 18% → 100% by fixing root causes, not labels.
Real impact
Every team running LLM agents has this exact pain and solves it with eyeballs. Cassandra closes the loop while the human sleeps.