Metadata-Version: 2.4
Name: swarmax-fleet
Version: 4.3.5
Summary: Evidence-based operations for AI agent fleets: sealed audit trails, SLA-alarmed triage and AI-Act-ready privacy.
Author: Swarmax Maintainers
License: Apache-2.0
Project-URL: Homepage, https://github.com/goun7/Swarmax
Project-URL: Documentation, https://github.com/goun7/Swarmax#readme
Project-URL: Repository, https://github.com/goun7/Swarmax
Project-URL: Issues, https://github.com/goun7/Swarmax/issues
Keywords: ai-agents,observability,evidence,audit,opentelemetry,llm-ops,agent-fleet,compliance,ai-act,gdpr,swarm,multi-agent,ai-audit,agent-monitoring
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: System :: Monitoring
Classifier: Topic :: System :: Logging
Classifier: Topic :: Security
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Dynamic: license-file

<div align="center">

<img src="assets/logo-mark-256.png" width="128" alt="Swarmax logo" title="Swarmax">

# Swarmax

**The audit and operations layer for AI agent fleets.**

<img src="assets/banner-v2.png" width="100%" alt="Swarmax — audit and operations for AI agent fleets">

</div>

[![CI](https://github.com/goun7/Swarmax/actions/workflows/ci.yml/badge.svg)](https://github.com/goun7/Swarmax/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/swarmax-fleet)](https://pypi.org/project/swarmax-fleet/)
[![Python](https://img.shields.io/pypi/pyversions/swarmax-fleet)](https://pypi.org/project/swarmax-fleet/)
[![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)

---

## What it is

**Swarmax is the monitoring and audit layer that sits beside your agent
framework and watches the fleet in production.** If Datadog is how you know
your services are healthy, Swarmax is how you know your agents are — and how
you prove it. It is not a framework and does not run your agents; it consumes
the events they already emit.

## The problem

Every team running autonomous agents gets asked the same three questions, and
frameworks do not answer them:

- *Is this fleet actually working?* Traces show you a single run. Nobody shows
  you that the fleet is drifting, looping, or quietly failing at a rate that
  would page a human.
- *What did it cost?* Token bills arrive as one aggregate number. Which agent,
  which task, which tool loop burned the budget is usually unknowable until
  someone complains.
- *Can you prove any of it?* When a customer, an auditor or a regulator asks
  *"what did your agents do in March?"*, a dashboard screenshot is not
  evidence. Logs are mutable, and you cannot demonstrate a negative ("this
  record has not been altered since it was written").

Swarmax answers all three, and it is built around the third one.

## What is different

Three things we have not found anywhere else:

- **Sealed evidence, not logs.** Every task event lands in an append-only
  ledger. The ledger is periodically sealed with Ed25519 signatures and a
  Merkle root — tampering breaks verification — and each seal can be
  counter-signed by an independent RFC 3161 timestamp authority. When an
  auditor asks what your agents did, you hand over a chain that verifies
  offline, with no need to trust your server.
- **Alarms with a deadline.** Detection thresholds feed an escalation map
  where every alarm class carries an SLA: a runaway tool loop is Emergency
  with a 30-minute clock; an unseen error signature is Critical with 24 hours.
  Open alarms count down in the console and self-escalate when the clock
  expires. Alert fatigue is a measured, budgeted quantity (false-positive
  budget ≤ 5 % of evaluation cycles).
- **Compliance as a command.** The EU AI Act asks providers for logging,
  traceability and deletion guarantees. Swarmax ships a working privacy
  toolkit (RFC 8439 crypto-shredding, an Article 17 `/forget` endpoint,
  S3-compatible signed cold archive) and a compliance drill an auditor can
  run themselves: `make compliance-drill`.

Everything is Python ≥ 3.10 stdlib — **zero runtime dependencies** — so it
fits on a small VPS next to your agents, or scales to tens of millions of
events on ClickHouse. Every crypto and ingest path is pinned to its RFC with
published test vectors.

## Architecture — where the events go

Swarmax never sits in your agents' request path. It consumes the telemetry
they already emit and turns it into status, alarms and proof:

```
your agents (LangChain / CrewAI / AutoGen / custom SDK)
   │ OTLP/HTTP+JSON (gen_ai.* / swx.*)   FleetSdk span / guard   Langfuse bridge
   ▼                                     ▼                        ▼
   otlp.py ingest — HMAC anti-replay; unsigned OTLP is rejected (fail-closed 503)
   ▼
   pipeline.py — events + guard calls → rolling windows → per-agent facts
      error rate · deny ratio · retries · ticket aging · JSD tool drift · n-gram loop
   │
   ├─► ch.py — ClickHouse ReplacingMergeTree mirror (10M+ events, per-agent < 200 ms)
   ▼
   metrics/apd.py — every fact onto the escalation map: signal → severity + SLA clock
      every alarm raise and resolution is also appended to the evidence ledger
   │
   ├───────────────────────────────────────┐
   ▼                                       ▼
 console.py / report.py                   evidence_ledger (append-only)
 fleet status, SLA queue,                 SHA-256 prev-hash chain; UPDATE and
 per-agent drill-down, weekly digest      DELETE refused by triggers
                                          │
                                 sealing.py — Merkle root over the chain
                                 + Ed25519 signature over (root ‖ covers),
                                 optionally counter-signed by an RFC 3161 TSA
                                          │
                                 coldstore.py — S3-compatible archive;
                                 manifest + chunks verify offline, with no
                                 private key and no database
```

Three properties fall out of that shape, and they are the reason the project
exists:

- **status is derived, not reported** — every counter is recomputed from the
  store on read, so the console and the report can never disagree with the
  ledger underneath them;
- **alarms are ledger entries** — an alarm is not a notification, it is a row
  in an append-only chain, so "the fleet drifted on Tuesday" is a verifiable
  statement rather than a dashboard recollection;
- **proof is self-contained** — a seal verifies against the ledger alone, and
  an archived ledger verifies against its manifest alone; neither needs
  Swarmax running, which is what an auditor actually wants.

## Is my fleet working? — the three questions, answered

The portfolio question this project exists for is *"filoton çalışıyor mu, neye
mal oldu, denetimde kanıtlayabilir misin?"* Each part maps onto one read-only
path over the store.

**1. Is it working?** — four counters and one triage queue:

```python
from swarmax.db import connect, init_db_with_migrations
from swarmax.console import FleetConsole

conn = connect("data/swarmax.db")
init_db_with_migrations(conn)

s = FleetConsole(conn).summary()
# s == {"open_alarms": …, "breached": …, "agents": …, "events_24h": …}
# `breached` counts open alarms whose SLA clock has already expired.

for a in FleetConsole(conn).open_alarms():        # severity first, then deadline
    print(a["severity"], a["signal"], "due", a["sla_deadline"])
```

In production that surface is the console at <http://127.0.0.1:8080>
(`make console`): the SLA queue counts down, self-escalates when its clock
expires, and drills down per agent.

**2. What did it cost?** — per-agent attribution instead of one aggregate bill:

```python
from swarmax.report import WeeklyReport

print(WeeklyReport(conn).render())
# §1 behavioural drift · §2 escalations & SLA · §3 cost ranking (top 5, 7d)
#   · §4 first-seen error classes · §5 evidence integrity
```

**3. Can you prove it?** — the part nothing else does. Every alarm raise and
resolution is anchored in the append-only ledger; the ledger is sealed with a
Merkle root under an Ed25519 signature, optionally counter-signed by an
RFC 3161 timestamp authority. Verification recomputes everything from the
store, so it is evidence rather than a claim:

```python
from swarmax.evidence import verify_chain
from swarmax.sealing import verify_seals

verify_chain(conn)     # (True, N) — N entries re-walked, no broken link
verify_seals(conn)     # {"seals": N, "all_ok": True, "results": [...]}
```

Or as drills an auditor can run themselves — both exit 0 on PASS:

```bash
make drill             # export → rebuild → verify the whole store
make compliance-drill  # logging / traceability / erasure walkthrough (re-runnable)
```

Tampering anywhere in the chain breaks verification: see
[`tests/test_sealing.py`](tests/test_sealing.py) and
[`tests/test_audit_trail_dallar.py`](tests/test_audit_trail_dallar.py), and
the fleet-status invariants in
[`tests/test_fleet_status_dallar.py`](tests/test_fleet_status_dallar.py).

## In 30 seconds

```bash
pip install swarmax-fleet                 # stdlib-only; no dependency resolution
swarmax-init                             # create or upgrade the evidence store

python -c "
from swarmax import SwarmaxClient
c = SwarmaxClient('http://127.0.0.1:4318', 'my-key', b'my-secret')
c.set_agent('support-bot', model='gpt-4o-mini')
with c.span():                            # timed block -> signed telemetry
    run_my_agent_step()
c.guard('web_search', fn, q)              # tool call + runaway-loop protection
c.flush()
"

make seal                                 # seal the ledger: tampering breaks it
```

Every event above lands in an append-only, Merkle-sealed ledger that verifies
offline — a dashboard screenshot is not evidence, this is.

## Quickstart

```bash
pip install swarmax-fleet     # stdlib-only; no dependencies to resolve
# or from source:
git clone https://github.com/goun7/Swarmax.git
cd Swarmax
pip install -e .

swarmax-init                # create or upgrade a store (schema v10)
python demo.py              # synthetic fleet: 8/8 alarm classes + FP budget
make seed && make console   # realistic demo fleet -> http://127.0.0.1:8080
```

Demo console logins (created by the seeder — rotate immediately in
production): `root` / `swarmax-demo-admin`, `viewer` / `swarmax-demo-viewer`.

### Integrate in five minutes

```python
from swarmax import SwarmaxClient

client = SwarmaxClient("http://127.0.0.1:4318", "my-key", b"my-secret")
client.set_agent("support-bot", model="gpt-4o-mini")
with client.span():                 # timed block -> real telemetry
    run_my_agent_step()
client.guard("web_search", fn, q)   # tool call + runaway-loop protection
client.flush()
```

Full walkthrough: [`examples/quickstart.py`](examples/quickstart.py).
Node/Bun: [`examples/js/swarmax.js`](examples/js/swarmax.js) (zero-dependency
mini SDK with the same three call styles).

Already tracing with Langfuse? Mirror your fleet without touching your app:

```bash
python -m swarmax.bridges.langfuse_pull --help
```

Any framework exporting OpenTelemetry (LangChain, CrewAI, AutoGen,
OpenLLMetry…): run the signing relay and point `OTEL_EXPORTER_OTLP_ENDPOINT`
at it — unsigned OTLP is rejected, so nothing reaches the ledger unsigned:

```bash
swarmax-otel-relay --help
```

### Development

```bash
python -m pytest tests -q      # 3426 tests (measured 2026-10-19): RFC vectors, alarm injection, sealing, audit trail, fleet status, perf, metrics classifier/loop_breaker, semconv registry/validate
python scripts/smoke_e2e.py    # end-to-end: ingest -> alarms -> seal -> console
make js-test                   # Node SDK contract tests
```

## What's inside

| Module | What it does |
|---|---|
| `pipeline.py` | ingest → rolling windows → per-agent facts → APD alarms |
| `otlp.py` | OTLP/HTTP+JSON ingest with HMAC anti-replay (`gen_ai.*` / `swx.*` semconv) |
| `metrics/` | EWMA control cards, robust MAD z-scores, JSD tool-mix drift, Page-Hinkley, n-gram loop breaker, MAST-aligned classifier, metamorphic end-state oracle |
| `metrics/apd.py` | Alarm & Protection Dispatcher — the escalation map |
| `sealing.py` + `ed25519.py` | Merkle sealing, key rotation, RFC 8032 signatures (pure Python, vectors-tested) |
| `tsa.py` | RFC 3161 timestamping client — counter-signs seals against a real TSA |
| `auth.py` / `sso.py` | console auth (scrypt, sessions, CSRF, lockout) + OIDC SSO with PKCE |
| `ch.py` | ClickHouse ReplacingMergeTree mirror — 10 M events, per-agent queries < 200 ms |
| `coldstore.py` | AWS SigV4 archive to any S3-compatible store + offline chain verify |
| `privacy/` | ChaCha20-Poly1305 crypto-shredding, HKDF, `/forget` (RFC 8439 / 5869) |
| `console.py` | multi-tenant web console: SLA queue, agent drill-down, PNG chart export, weekly report download/e-mail |
| `dogfood.py` | fleet SDK (`FleetSdk`) + `SwarmaxClient` facade (task / span / guard) |
| `bridges/langfuse_pull.py` | Langfuse → Swarmax pull bridge: maps GENERATION observations to signed `gen_ai.*` spans (idempotent) |
| `bridges/otel_relay.py` | signing relay: any framework's unsigned OTLP/HTTP → signed Swarmax ingest (fail-closed 503) |

## MCP server — read-only fleet query core for agents

Three tools over stdio JSON-RPC (protocol `2025-06-18`), pure stdlib, same
contract as the Tamga/Sester MCP servers: never raises, returns a structured
GREEN/RED JSON proof instead of a traceback.

| Tool | What it answers |
|---|---|
| `fleet_status` | open alarms, breached-SLA count, agent count, 24 h event count |
| `agent_health` | single-agent state page |
| `audit_trail` | evidence-ledger chain + seal verification (verifies, never seals) |

The server is **read-only by design**: it never writes, never seals, never
forwards. `otel_relay` (stateless forwarder) and `seal_ledger` (write path)
are deliberately absent from it.

```bash
python3 mcp/server.py                        # stdio JSON-RPC
SWARMAX_MCP_DB=/path/swarmax.db python3 mcp/server.py
```

DB resolution order: `db_path` argument → `SWARMAX_MCP_DB` env → repo-local
`data/swarmax.db` default. Since the default is the **real** store, an MCP
client should reach for it only for read-only purposes, not to mutate open
alarms. Point `SWARMAX_MCP_DB` at a copy when in doubt.

Client side:

```json
{ "command": "python", "args": ["<repo>/mcp/server.py"],
  "env": { "SWARMAX_MCP_DB": "/path/swarmax.db" } }
```

Pinned by `tests/test_mcp_server.py` (subprocess-driven, real stdio):
initialize handshake, tool list, all three tools GREEN on the real DB,
unknown-tool JSON-RPC `-32601`, missing-DB RED, and the G20 default-DB
resolution pin (env and argument both absent → repo-local default, not a
fail-closed RED).

**Packaging note (G21).** `mcp/server.py` ships in the sdist (`MANIFEST.in`)
but is **not** a Python package inside the wheel — `pip install swarmax-fleet`
gives you the library and the four console scripts, not a runnable MCP
entry point. Run the server from a source checkout, or copy `mcp/server.py`
out of the sdist. Making it a `py-modules` entry would collide with the
`mcp` namespace package used by the MCP SDK, so the single-file module is
deliberately kept out of `site-packages` naming.

## Acceptance gates (all enforced in CI or by drills)

| Gate | Where |
|---|---|
| 100K events, p99 ingest < 2 s (512-span batches) | `tests/test_performance.py` |
| 8/8 alarm classes detected, correct severity + SLA | `tests/test_scenarios.py` |
| False positives ≤ 5 % of hourly cycles | `test_scenarios.py::test_baseline_produces_zero_false_positives` |
| Tamper → seal verification fails | `tests/test_sealing.py` |
| Ed25519 against RFC 8032 vectors | `tests/test_ed25519.py` |
| ChaCha20-Poly1305 against RFC 8439 (cross-checked with OpenSSL) | `tests/test_privacy_crypto.py` |
| OTLP replay rejected | `tests/test_otlp.py` |
| 10M events on real ClickHouse, worst per-agent query 78 ms | `make scale-gate-ch` |
| Exit drill: export → rebuild → verify | `make drill` |
| TSA counter-signature verifies with `openssl ts -verify` | `make trust-drill` |

## Operations

```bash
make otlp             # ingest service :4318  (set SWX_INGEST_SECRET)
make console          # fleet console :8080 (set SWARMAX_ADMIN_PASSWORD in production)
make seal             # seal the evidence ledger now
make drill            # quarterly exit drill (exit 0 = PASS)
make compliance-drill # executable auditor walkthrough (re-runnable)
make ch-mirror        # one-shot ClickHouse mirror pass
make cold-archive / cold-verify
make trust-drill      # RFC 3161 counter-signed sealing drill (live TSA)
```

The alarm map (`metrics/apd.py`): loop ≥ 2 → Emergency/30m · deny > 20 % →
Critical/4h · daily cost > 2×EWMA ∧ Z > 2.5 → Critical/24h · unseen error
class → Critical/24h · error rate > 20 %/24h → Critical/4h · retries ≥ 3 →
High/4h · JSD > 0.40 → Medium/weekly · ticket age > 24h → Emergency/2h with
self-escalation.

## Scope and limits — read this before you adopt it

Swarmax observes, alarms, proves and deletes. It deliberately does **not**:

- **run your agents** — it has no LLM client, no prompt store, no model
  gateway; point it at events your framework already emits;
- **replay or roll back tasks** — the evidence is for proving what happened,
  not re-running it;
- **score answer quality** — vitals that need raw prompts or pixels (per-turn
  hallucination scoring, toxicity checks) stay with the framework that
  produced them; Swarmax consumes their events;
- **act as your LLM gateway** — it measures cost from the tokens reported to
  it, it does not sit in the request path.

Deployment limits that are true today, not on a roadmap slide:

- **single node per fleet** — the honest unit is one sealed store per fleet;
  multi-node active-active and a managed cloud are planned, not shipped;
- **single-tenant console per store** — the multi-tenant schema exists, but
  there is no organisation-level isolation layer above it yet;
- **the seal clock is wall time** — sealing is driven by a scheduler or
  `make seal`; there is no real-time sealing guarantee;
- **OpenTelemetry GenAI semconv is still Development-stage** upstream —
  Swarmax pins a specific revision and extends it with a `swx.*` namespace
  rather than chasing a moving target.

## Documents

- `SWARMAX.md` — the full design record (Turkish), single source of record
- `SWARMAX_EN.md` — English edition
- `AI_ACT_COMPLIANCE.md` — the compliance dossier the drills exercise
- `DOGFOOD_PLAN.md` — the MT-1…MT-8 measurement protocol
- `CHANGELOG.md` · `CONTRIBUTING.md` · `SECURITY.md`

## Landing - "ajanlar calisiyor mu?" (G5, statik filo-audit-demo)

Tek-sayfalik filo-audit-demo: filo-genel-bakis (saglikli/uyari/belirsiz),
SLA'li acik-alarm kuyrugu, audit-trail zaman-cizelgesi ve MCP
arac-listesi (fleet_status / agent_health / audit_trail). Tamamen
statiktir - gomulu demo-veri disinda canli-endpoint/hostname/DB-yolu/anahtar
ICERMEZ; demo-veri mcp/server.py'nin gercek-DB ciktilarindan alinmistir.

    python3 -m http.server landing/      # statik-dagitim, varsayilan 8000

Testleri: python -m pytest tests/test_landing.py -q (26 test; ASCII,
statiklik, MCP arac/arguman sozlesmesi, Turkce icerik).

## License

Apache-2.0. Free to self-host, fork and run — the operation of it is what we
sell.

## Academic & technical references

- **Observability as evidence, not telemetry** — Swarmax does not invent
  its own event format; it consumes the events services already emit and
  seals them. The distinction matters: telemetry you generate yourself is
  a claim; telemetry the service emitted before you asked is closer to
  evidence.
- **Tamper-evident monitoring** — the core property x402's own ecosystem
  converged on independently:
  [x402 #2332](https://github.com/x402-foundation/x402/issues/2332)
  (post-settlement accountability, 214+ comments) — the summary line is
  "logs are rewritable; a hash-chain anchor is not." Swarmax applies the
  same property to fleet health that Sester applies to payment receipts.
- **EU AI Act Article 12** (automated logging, in force 2 December 2027):
  high-risk systems must keep logs that resist modification. An audit
  layer whose logs are rewriteable cannot satisfy this; hence the seal.
- **Mesh connections** — Swarmax is the audit layer for the fleet;
  [63-Sester](../63-Sester) provides the same property for payment
  receipts, [TamgaProtocol](../../05_acik_kaynak/TamgaProtocol) the
  persistent anchor. Same primitive, different surface.

## Akademik Kaynaklar (2024-2026)

Bu calisma asagidaki arastirmaya dayanir:

- [1] **AgentMonitor: A Plug-and-Play Framework for Predictive and Secure
  Multi-Agent Systems** — Chan et al., arXiv 2024 (cs.CL).
  [arXiv:2408.14972](https://arxiv.org/abs/2408.14972) — plug-and-play
  run-time monitoring of multi-agent LLM systems; the closest published
  analogue to Swarmax's per-agent fact pipeline.
- [2] **SIEVE: Selective Integrity Verification and Escalation for Defending
  LLM Agents against Indirect Prompt Injection** — Liang et al., arXiv 2025
  (cs.AI). [arXiv:2512.06716](https://arxiv.org/abs/2512.06716) —
  integrity verification plus an escalation policy on detected violations;
  the same verify-then-escalate shape as the APD alarm map.
- [3] **ESAA: Event Sourcing for Autonomous Agents in LLM-Based Software
  Engineering** — Brito dos Santos Filho, arXiv 2026 (cs.AI).
  [arXiv:2602.23193](https://arxiv.org/abs/2602.23193) — event sourcing as
  the persistence model for autonomous agent runs; the academic basis for
  the append-only ledger behind the ClickHouse mirror.
- [4] **Observability and Fault Injection for LLM-Based Multi-Agent Systems
  in Software Engineering** — Seyedghorban et al., arXiv 2026 (cs.SE).
  [arXiv:2608.24271](https://arxiv.org/abs/2608.24271) — observability and
  fault-injection testing for LLM multi-agent systems; the empirical
  grounding for Swarmax's alarm-class injection drills.
- [5] **NovaFabric: Tamper-Evident, Replayable Evidence for Autonomous AI
  Agent Runs** — Seyedkazemi Ardebili, arXiv 2026 (cs.CR).
  [arXiv:2609.12582](https://arxiv.org/abs/2609.12582) — tamper-evident,
  replayable evidence records for agent runs; independently reaches the
  sealed-evidence property the Merkle/Ed25519 chain provides here.
- [6] **Agent Flight Recorder: Tamper-Evident Audit Trails with On-Chain
  Anchoring for Long-Horizon Tool-Using Agents** — Bindschaedler et al.,
  arXiv 2026 (cs.CR). [arXiv:2609.01931](https://arxiv.org/abs/2609.01931) —
  anchored audit trails for long-horizon tool-using agents; the academic
  counterpart to Swarmax's sealed ledger and cold-archive anchoring.
- [7] **Agent-Warden: eBPF-Based Kernel-Native Process-File Provenance
  Tracking for LLM Agents** — Cui et al., arXiv 2026 (cs.CR).
  [arXiv:2609.38245](https://arxiv.org/abs/2609.38245) — kernel-native
  provenance capture for agent behaviour; complementary provenance layer
  beneath the application-level events Swarmax consumes.
- [8] **LLM Agents Can Easily Tamper With Their Own Traces** — Qin et al.,
  arXiv 2026 (cs.CR). [arXiv:2609.30266](https://arxiv.org/abs/2609.30266) —
  demonstrates that agent-emitted traces are rewriteable by the agents
  themselves; the threat model that makes the seal and counter-signature
  load-bearing rather than decorative.

<div align="center">
<img src="assets/logo-mono.svg" width="56" alt="" style="vertical-align:middle">
&nbsp;<sub>Seven points, one chain — measure everything, seal what matters.</sub>
</div>
