---

What it is

Swarmax is the monitoring and audit layer that sits beside your agent framework and watches the fleet in production. If Datadog is how you know your services are healthy, Swarmax is how you know your agents are — and how you prove it. It is not a framework and does not run your agents; it consumes the events they already emit.

The problem

Every team running autonomous agents gets asked the same three questions, and frameworks do not answer them:

• Is this fleet actually working? Traces show you a single run. Nobody shows you that the fleet is drifting, looping, or quietly failing at a rate that would page a human.
• What did it cost? Token bills arrive as one aggregate number. Which agent, which task, which tool loop burned the budget is usually unknowable until someone complains.
• Can you prove any of it? When a customer, an auditor or a regulator asks "what did your agents do in March?", a dashboard screenshot is not evidence. Logs are mutable, and you cannot demonstrate a negative ("this record has not been altered since it was written").

Swarmax answers all three, and it is built around the third one.

What is different

Three things we have not found anywhere else:

• Sealed evidence, not logs. Every task event lands in an append-only ledger. The ledger is periodically sealed with Ed25519 signatures and a Merkle root — tampering breaks verification — and each seal can be counter-signed by an independent RFC 3161 timestamp authority. When an auditor asks what your agents did, you hand over a chain that verifies offline, with no need to trust your server.
• Alarms with a deadline. Detection thresholds feed an escalation map where every alarm class carries an SLA: a runaway tool loop is Emergency with a 30-minute clock; an unseen error signature is Critical with 24 hours. Open alarms count down in the console and self-escalate when the clock expires. Alert fatigue is a measured, budgeted quantity (false-positive budget ≤ 5 % of evaluation cycles).
• Compliance as a command. The EU AI Act asks providers for logging, traceability and deletion guarantees. Swarmax ships a working privacy toolkit (RFC 8439 crypto-shredding, an Article 17 /forget endpoint, S3-compatible signed cold archive) and a compliance drill an auditor can run themselves: make compliance-drill.

Everything is Python ≥ 3.10 stdlib — zero runtime dependencies — so it fits on a small VPS next to your agents, or scales to tens of millions of events on ClickHouse. Every crypto and ingest path is pinned to its RFC with published test vectors.

Architecture — where the events go

Swarmax never sits in your agents' request path. It consumes the telemetry they already emit and turns it into status, alarms and proof:

your agents (LangChain / CrewAI / AutoGen / custom SDK)
   │ OTLP/HTTP+JSON (gen_ai.* / swx.*)   FleetSdk span / guard   Langfuse bridge
   ▼                                     ▼                        ▼
   otlp.py ingest — HMAC anti-replay; unsigned OTLP is rejected (fail-closed 503)
   ▼
   pipeline.py — events + guard calls → rolling windows → per-agent facts
      error rate · deny ratio · retries · ticket aging · JSD tool drift · n-gram loop
   │
   ├─► ch.py — ClickHouse ReplacingMergeTree mirror (10M+ events, per-agent < 200 ms)
   ▼
   metrics/apd.py — every fact onto the escalation map: signal → severity + SLA clock
      every alarm raise and resolution is also appended to the evidence ledger
   │
   ├───────────────────────────────────────┐
   ▼                                       ▼
 console.py / report.py                   evidence_ledger (append-only)
 fleet status, SLA queue,                 SHA-256 prev-hash chain; UPDATE and
 per-agent drill-down, weekly digest      DELETE refused by triggers
                                          │
                                 sealing.py — Merkle root over the chain
                                 + Ed25519 signature over (root ‖ covers),
                                 optionally counter-signed by an RFC 3161 TSA
                                          │
                                 coldstore.py — S3-compatible archive;
                                 manifest + chunks verify offline, with no
                                 private key and no database

Three properties fall out of that shape, and they are the reason the project exists:

• status is derived, not reported — every counter is recomputed from the store on read, so the console and the report can never disagree with the ledger underneath them;
• alarms are ledger entries — an alarm is not a notification, it is a row in an append-only chain, so "the fleet drifted on Tuesday" is a verifiable statement rather than a dashboard recollection;
• proof is self-contained — a seal verifies against the ledger alone, and an archived ledger verifies against its manifest alone; neither needs Swarmax running, which is what an auditor actually wants.

Is my fleet working? — the three questions, answered

The portfolio question this project exists for is "filoton çalışıyor mu, neye mal oldu, denetimde kanıtlayabilir misin?" Each part maps onto one read-only path over the store.

1. Is it working? — four counters and one triage queue:

from swarmax.db import connect, init_db_with_migrations
from swarmax.console import FleetConsole

conn = connect("data/swarmax.db")
init_db_with_migrations(conn)

s = FleetConsole(conn).summary()
# s == {"open_alarms": …, "breached": …, "agents": …, "events_24h": …}
# `breached` counts open alarms whose SLA clock has already expired.

for a in FleetConsole(conn).open_alarms():        # severity first, then deadline
    print(a["severity"], a["signal"], "due", a["sla_deadline"])

In production that surface is the console at http://127.0.0.1:8080 (make console): the SLA queue counts down, self-escalates when its clock expires, and drills down per agent.

2. What did it cost? — per-agent attribution instead of one aggregate bill:

from swarmax.report import WeeklyReport

print(WeeklyReport(conn).render())
# §1 behavioural drift · §2 escalations & SLA · §3 cost ranking (top 5, 7d)
#   · §4 first-seen error classes · §5 evidence integrity

3. Can you prove it? — the part nothing else does. Every alarm raise and resolution is anchored in the append-only ledger; the ledger is sealed with a Merkle root under an Ed25519 signature, optionally counter-signed by an RFC 3161 timestamp authority. Verification recomputes everything from the store, so it is evidence rather than a claim:

from swarmax.evidence import verify_chain
from swarmax.sealing import verify_seals

verify_chain(conn)     # (True, N) — N entries re-walked, no broken link
verify_seals(conn)     # {"seals": N, "all_ok": True, "results": [...]}

Or as drills an auditor can run themselves — both exit 0 on PASS:

make drill             # export → rebuild → verify the whole store
make compliance-drill  # logging / traceability / erasure walkthrough (re-runnable)

Tampering anywhere in the chain breaks verification: see tests/test_sealing.py and tests/test_audit_trail_dallar.py, and the fleet-status invariants in tests/test_fleet_status_dallar.py.

In 30 seconds

pip install swarmax-fleet                 # stdlib-only; no dependency resolution
swarmax-init                             # create or upgrade the evidence store

python -c "
from swarmax import SwarmaxClient
c = SwarmaxClient('http://127.0.0.1:4318', 'my-key', b'my-secret')
c.set_agent('support-bot', model='gpt-4o-mini')
with c.span():                            # timed block -> signed telemetry
    run_my_agent_step()
c.guard('web_search', fn, q)              # tool call + runaway-loop protection
c.flush()
"

make seal                                 # seal the ledger: tampering breaks it

Every event above lands in an append-only, Merkle-sealed ledger that verifies offline — a dashboard screenshot is not evidence, this is.

Quickstart

pip install swarmax-fleet     # stdlib-only; no dependencies to resolve
# or from source:
git clone https://github.com/goun7/Swarmax.git
cd Swarmax
pip install -e .

swarmax-init                # create or upgrade a store (schema v10)
python demo.py              # synthetic fleet: 8/8 alarm classes + FP budget
make seed && make console   # realistic demo fleet -> http://127.0.0.1:8080

Demo console logins (created by the seeder — rotate immediately in production): root / swarmax-demo-admin, viewer / swarmax-demo-viewer.

Integrate in five minutes

from swarmax import SwarmaxClient

client = SwarmaxClient("http://127.0.0.1:4318", "my-key", b"my-secret")
client.set_agent("support-bot", model="gpt-4o-mini")
with client.span():                 # timed block -> real telemetry
    run_my_agent_step()
client.guard("web_search", fn, q)   # tool call + runaway-loop protection
client.flush()

Full walkthrough: examples/quickstart.py. Node/Bun: examples/js/swarmax.js (zero-dependency mini SDK with the same three call styles).

Already tracing with Langfuse? Mirror your fleet without touching your app:

python -m swarmax.bridges.langfuse_pull --help

Any framework exporting OpenTelemetry (LangChain, CrewAI, AutoGen, OpenLLMetry…): run the signing relay and point OTEL_EXPORTER_OTLP_ENDPOINT at it — unsigned OTLP is rejected, so nothing reaches the ledger unsigned:

swarmax-otel-relay --help

Development

python -m pytest tests -q      # 3426 tests (measured 2026-10-19): RFC vectors, alarm injection, sealing, audit trail, fleet status, perf, metrics classifier/loop_breaker, semconv registry/validate
python scripts/smoke_e2e.py    # end-to-end: ingest -> alarms -> seal -> console
make js-test                   # Node SDK contract tests

What's inside

Module	What it does
pipeline.py	ingest → rolling windows → per-agent facts → APD alarms
otlp.py	OTLP/HTTP+JSON ingest with HMAC anti-replay (gen_ai.* / swx.* semconv)
metrics/	EWMA control cards, robust MAD z-scores, JSD tool-mix drift, Page-Hinkley, n-gram loop breaker, MAST-aligned classifier, metamorphic end-state oracle
metrics/apd.py	Alarm & Protection Dispatcher — the escalation map
sealing.py + ed25519.py	Merkle sealing, key rotation, RFC 8032 signatures (pure Python, vectors-tested)
tsa.py	RFC 3161 timestamping client — counter-signs seals against a real TSA
auth.py / sso.py	console auth (scrypt, sessions, CSRF, lockout) + OIDC SSO with PKCE
ch.py	ClickHouse ReplacingMergeTree mirror — 10 M events, per-agent queries < 200 ms
coldstore.py	AWS SigV4 archive to any S3-compatible store + offline chain verify
privacy/	ChaCha20-Poly1305 crypto-shredding, HKDF, /forget (RFC 8439 / 5869)
console.py	multi-tenant web console: SLA queue, agent drill-down, PNG chart export, weekly report download/e-mail
dogfood.py	fleet SDK (FleetSdk) + SwarmaxClient facade (task / span / guard)
bridges/langfuse_pull.py	Langfuse → Swarmax pull bridge: maps GENERATION observations to signed gen_ai.* spans (idempotent)
bridges/otel_relay.py	signing relay: any framework's unsigned OTLP/HTTP → signed Swarmax ingest (fail-closed 503)

MCP server — read-only fleet query core for agents

Three tools over stdio JSON-RPC (protocol 2025-06-18), pure stdlib, same contract as the Tamga/Sester MCP servers: never raises, returns a structured GREEN/RED JSON proof instead of a traceback.

Tool	What it answers
fleet_status	open alarms, breached-SLA count, agent count, 24 h event count
agent_health	single-agent state page
audit_trail	evidence-ledger chain + seal verification (verifies, never seals)

The server is read-only by design: it never writes, never seals, never forwards. otel_relay (stateless forwarder) and seal_ledger (write path) are deliberately absent from it.

python3 mcp/server.py                        # stdio JSON-RPC
SWARMAX_MCP_DB=/path/swarmax.db python3 mcp/server.py

DB resolution order: db_path argument → SWARMAX_MCP_DB env → repo-local data/swarmax.db default. Since the default is the real store, an MCP client should reach for it only for read-only purposes, not to mutate open alarms. Point SWARMAX_MCP_DB at a copy when in doubt.

Client side:

{ "command": "python", "args": ["<repo>/mcp/server.py"],
  "env": { "SWARMAX_MCP_DB": "/path/swarmax.db" } }

Pinned by tests/test_mcp_server.py (subprocess-driven, real stdio): initialize handshake, tool list, all three tools GREEN on the real DB, unknown-tool JSON-RPC -32601, missing-DB RED, and the G20 default-DB resolution pin (env and argument both absent → repo-local default, not a fail-closed RED).

Packaging note (G21). mcp/server.py ships in the sdist (MANIFEST.in) but is not a Python package inside the wheel — pip install swarmax-fleet gives you the library and the four console scripts, not a runnable MCP entry point. Run the server from a source checkout, or copy mcp/server.py out of the sdist. Making it a py-modules entry would collide with the mcp namespace package used by the MCP SDK, so the single-file module is deliberately kept out of site-packages naming.

Acceptance gates (all enforced in CI or by drills)

Gate	Where
100K events, p99 ingest < 2 s (512-span batches)	tests/test_performance.py
8/8 alarm classes detected, correct severity + SLA	tests/test_scenarios.py
False positives ≤ 5 % of hourly cycles	test_scenarios.py::test_baseline_produces_zero_false_positives
Tamper → seal verification fails	tests/test_sealing.py
Ed25519 against RFC 8032 vectors	tests/test_ed25519.py
ChaCha20-Poly1305 against RFC 8439 (cross-checked with OpenSSL)	tests/test_privacy_crypto.py
OTLP replay rejected	tests/test_otlp.py
10M events on real ClickHouse, worst per-agent query 78 ms	make scale-gate-ch
Exit drill: export → rebuild → verify	make drill
TSA counter-signature verifies with openssl ts -verify	make trust-drill

Operations

make otlp             # ingest service :4318  (set SWX_INGEST_SECRET)
make console          # fleet console :8080 (set SWARMAX_ADMIN_PASSWORD in production)
make seal             # seal the evidence ledger now
make drill            # quarterly exit drill (exit 0 = PASS)
make compliance-drill # executable auditor walkthrough (re-runnable)
make ch-mirror        # one-shot ClickHouse mirror pass
make cold-archive / cold-verify
make trust-drill      # RFC 3161 counter-signed sealing drill (live TSA)

The alarm map (metrics/apd.py): loop ≥ 2 → Emergency/30m · deny > 20 % → Critical/4h · daily cost > 2×EWMA ∧ Z > 2.5 → Critical/24h · unseen error class → Critical/24h · error rate > 20 %/24h → Critical/4h · retries ≥ 3 → High/4h · JSD > 0.40 → Medium/weekly · ticket age > 24h → Emergency/2h with self-escalation.

Scope and limits — read this before you adopt it

Swarmax observes, alarms, proves and deletes. It deliberately does not:

• run your agents — it has no LLM client, no prompt store, no model gateway; point it at events your framework already emits;
• replay or roll back tasks — the evidence is for proving what happened, not re-running it;
• score answer quality — vitals that need raw prompts or pixels (per-turn hallucination scoring, toxicity checks) stay with the framework that produced them; Swarmax consumes their events;
• act as your LLM gateway — it measures cost from the tokens reported to it, it does not sit in the request path.

Deployment limits that are true today, not on a roadmap slide:

• single node per fleet — the honest unit is one sealed store per fleet; multi-node active-active and a managed cloud are planned, not shipped;
• single-tenant console per store — the multi-tenant schema exists, but there is no organisation-level isolation layer above it yet;
• the seal clock is wall time — sealing is driven by a scheduler or make seal; there is no real-time sealing guarantee;
• OpenTelemetry GenAI semconv is still Development-stage upstream — Swarmax pins a specific revision and extends it with a swx.* namespace rather than chasing a moving target.

Documents

• SWARMAX.md — the full design record (Turkish), single source of record
• SWARMAX_EN.md — English edition
• AI_ACT_COMPLIANCE.md — the compliance dossier the drills exercise
• DOGFOOD_PLAN.md — the MT-1…MT-8 measurement protocol
• CHANGELOG.md · CONTRIBUTING.md · SECURITY.md

Landing - "ajanlar calisiyor mu?" (G5, statik filo-audit-demo)

Tek-sayfalik filo-audit-demo: filo-genel-bakis (saglikli/uyari/belirsiz), SLA'li acik-alarm kuyrugu, audit-trail zaman-cizelgesi ve MCP arac-listesi (fleet_status / agent_health / audit_trail). Tamamen statiktir - gomulu demo-veri disinda canli-endpoint/hostname/DB-yolu/anahtar ICERMEZ; demo-veri mcp/server.py'nin gercek-DB ciktilarindan alinmistir.

python3 -m http.server landing/      # statik-dagitim, varsayilan 8000

Testleri: python -m pytest tests/test_landing.py -q (26 test; ASCII, statiklik, MCP arac/arguman sozlesmesi, Turkce icerik).

License

Apache-2.0. Free to self-host, fork and run — the operation of it is what we sell.

Academic & technical references

• Observability as evidence, not telemetry — Swarmax does not invent its own event format; it consumes the events services already emit and seals them. The distinction matters: telemetry you generate yourself is a claim; telemetry the service emitted before you asked is closer to evidence.
• Tamper-evident monitoring — the core property x402's own ecosystem converged on independently: x402 #2332 (post-settlement accountability, 214+ comments) — the summary line is "logs are rewritable; a hash-chain anchor is not." Swarmax applies the same property to fleet health that Sester applies to payment receipts.
• EU AI Act Article 12 (automated logging, in force 2 December 2027): high-risk systems must keep logs that resist modification. An audit layer whose logs are rewriteable cannot satisfy this; hence the seal.
• Mesh connections — Swarmax is the audit layer for the fleet; 63-Sester provides the same property for payment receipts, TamgaProtocol the persistent anchor. Same primitive, different surface.

Akademik Kaynaklar (2024-2026)

Bu calisma asagidaki arastirmaya dayanir:

• [1] AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems — Chan et al., arXiv 2024 (cs.CL). arXiv:2408.14972 — plug-and-play run-time monitoring of multi-agent LLM systems; the closest published analogue to Swarmax's per-agent fact pipeline.
• [2] SIEVE: Selective Integrity Verification and Escalation for Defending LLM Agents against Indirect Prompt Injection — Liang et al., arXiv 2025 (cs.AI). arXiv:2512.06716 — integrity verification plus an escalation policy on detected violations; the same verify-then-escalate shape as the APD alarm map.
• [3] ESAA: Event Sourcing for Autonomous Agents in LLM-Based Software Engineering — Brito dos Santos Filho, arXiv 2026 (cs.AI). arXiv:2602.23193 — event sourcing as the persistence model for autonomous agent runs; the academic basis for the append-only ledger behind the ClickHouse mirror.
• [4] Observability and Fault Injection for LLM-Based Multi-Agent Systems in Software Engineering — Seyedghorban et al., arXiv 2026 (cs.SE). arXiv:2608.24271 — observability and fault-injection testing for LLM multi-agent systems; the empirical grounding for Swarmax's alarm-class injection drills.
• [5] NovaFabric: Tamper-Evident, Replayable Evidence for Autonomous AI Agent Runs — Seyedkazemi Ardebili, arXiv 2026 (cs.CR). arXiv:2609.12582 — tamper-evident, replayable evidence records for agent runs; independently reaches the sealed-evidence property the Merkle/Ed25519 chain provides here.
• [6] Agent Flight Recorder: Tamper-Evident Audit Trails with On-Chain Anchoring for Long-Horizon Tool-Using Agents — Bindschaedler et al., arXiv 2026 (cs.CR). arXiv:2609.01931 — anchored audit trails for long-horizon tool-using agents; the academic counterpart to Swarmax's sealed ledger and cold-archive anchoring.
• [7] Agent-Warden: eBPF-Based Kernel-Native Process-File Provenance Tracking for LLM Agents — Cui et al., arXiv 2026 (cs.CR). arXiv:2609.38245 — kernel-native provenance capture for agent behaviour; complementary provenance layer beneath the application-level events Swarmax consumes.
• [8] LLM Agents Can Easily Tamper With Their Own Traces — Qin et al., arXiv 2026 (cs.CR). arXiv:2609.30266 — demonstrates that agent-emitted traces are rewriteable by the agents themselves; the threat model that makes the seal and counter-signature load-bearing rather than decorative.