Kelpie

Kelpie keeps an AI agent farm moving. Open-source SRE for fleets of coding agents on Hermes, Claude Code and OpenClaw: it stops runaway agent loops before they burn a night of tokens, unsticks blocked, forgotten and orphaned tasks, catches CI outages and wasted builds, and tells you each day what the chaos cost and what fixed it. Deterministic rules, no LLM in the loop, and every rule starts observe-only until you let it act.

pip install kelpie-sre && kelpie demo

pip install kelpie-sre installs the latest alpha, 0.1.0a3: read its release notes before you run it on a farm. For the latest main: pip install git+https://github.com/ch8er/kelpie.

kelpie demo runs a simulated farm on your machine and shows Kelpie finding and fixing its problems. No farm, accounts or keys needed; Python 3.12 or newer. It replays a working day of real farm incidents on a simulated Hermes board, offline, then prints that day's report (--web also opens it as a web page, --keep keeps the farm and the full report):

Alpha (0.1.0a3). Config and storage formats may still change. Kelpie starts safe: global dry-run on and every rule in observe mode, so it changes nothing until you let it.

What it found on a real farm

Kelpie's rules come from incidents on a working agent farm and from field reports of other farms. Almost none of them are about model quality; they are about work that silently stops.

Kelpie also measures whether a fix worked. After one manual fix on that farm, review time dropped from 48 to 27 minutes and review failures from 9 of 29 to 1 of 11; the daily report now computes that before/after for every fix it knows about.

On a live farm

Kelpie runs on that farm, mostly in observe mode. Measured there so far:

• A stop on macOS: kelpie selftest stop on the farm's Mac stopped a looping test run after 11 identical calls; its worker ended 0.0 s after SIGTERM and no model call followed (#231). Before the macOS backend, a stopped worker ran on for about 3 minutes.
• A ~12 h toolchain outage (incident 20): to Kelpie it looked like seven independent human waits; the report showed 221.5 h of "needs you" and 83.4 h of stale blocks. Kelpie now reports a block storm, CI jobs that never start and correlated tool failures as one outage; the report side is in progress (#229).
• A token baseline: 210.4M tokens in 24 h, 92% cache reads, 8.1M per completed task, and 97 sleep calls inside sessions, which wait_condition now flags (#197).
• Before/after of three fixes made within 72 minutes, direction only (one farm, small samples): tasks closed per active hour 2.0 → 4.1–4.8, returns per closed task 0.50 → 0.26–0.32, sessions ending failed or without a status 17–21% → 1–9% (#193).

Every incident and what it cost: the incident table.

Quickstart

Every quickstart starts read-only (dry-run, observe). kelpie doctor checks the setup and prints a fix under each failure; kelpie why <task> explains why one task is not moving; kelpie ci-config shows, as a diff, the workflow changes rule 9 would propose. On a new machine doctor fails disk watched until node_exporter and Prometheus watch the agents' disks (Host resources); the paths in its fix: lines (docs/..., src/...) are in the repository.

Hermes

On the Hermes host:

pip install kelpie-sre
mkdir -p ~/.hermes/plugins/kelpie            # the hooks plugin: tool calls -> Kelpie
cp "$(python -c 'import kelpie.adapters.hermes.plugin as m; print(m.__file__)')" \
   ~/.hermes/plugins/kelpie/__init__.py

Add its plugin.yaml and enable it in Hermes as in step 3 of the Hermes quickstart, then:

# kelpie.toml
[adapters.hermes]
db_path = "~/.hermes/kanban.db"
spool = "~/.hermes/kelpie/spool.jsonl"

# Hermes heartbeats can be up to an hour old (Kelpie's default is 30 minutes)
[engine.rules.exit_contract]
session_stale_after = "60m"
[engine.rules.orphan_status]
session_stale_after = "60m"
[engine.rules.wait_condition]
session_stale_after = "60m"

kelpie serve      # control loop; /metrics and /healthz on 127.0.0.1:9464
kelpie doctor     # in a second shell
kelpie report     # yesterday's cost-of-chaos report

OTLP traces, Prometheus and Grafana: the Hermes quickstart.

Claude Code

On the host that runs Claude Code, with every session of the farm sharing one task list (export CLAUDE_CODE_TASK_LIST_ID=farm):

pip install kelpie-sre
mkdir -p -m 700 ~/.claude/kelpie
cp "$(python -c 'import kelpie.adapters.claude_code.hook as m; print(m.__file__)')" \
   ~/.claude/kelpie/hook.py

Register the hook in ~/.claude/settings.json as in step 4 of the Claude Code quickstart, then:

# kelpie.toml
[adapters.claude_code]
task_lists = ["farm"]
spool = "~/.claude/kelpie/spool.jsonl"

kelpie serve
kelpie doctor

Claude Code's own OpenTelemetry traces: the Claude Code quickstart.

Any agent that sends OpenTelemetry traces

kelpie serve accepts OTLP traces (OpenTelemetry GenAI conventions) on POST /v1/traces. With traces alone, the loop and budget breaker, budgets and the token sections of the daily report work for any platform; rules about the task board need a platform adapter. See Traces.

OpenClaw

On the Gateway host, as the Gateway's user (the Workboard plugin gives Kelpie tasks; without it Kelpie watches sessions, which is enough to stop runaway loops):

pip install kelpie-sre
mkdir -p ~/.openclaw/kelpie/plugin && chmod 700 ~/.openclaw/kelpie
(cd "$(python -c 'import kelpie.adapters.openclaw.plugin as m, os; print(os.path.dirname(m.__file__))')" &&
  install -m 0644 index.js kelpie_hook.py openclaw.plugin.json package.json ~/.openclaw/kelpie/plugin/)
openclaw plugins install --link ~/.openclaw/kelpie/plugin --force --accept-capabilities
openclaw plugins enable kelpie
openclaw config set plugins.entries.kelpie.hooks.allowConversationAccess true
openclaw config set plugins.entries.kelpie.config.python "$(command -v python3)"
openclaw gateway restart

# kelpie.toml
[adapters.openclaw]
state_dir = "~/.openclaw"
cli = "/usr/local/bin/openclaw"   # the output of `command -v openclaw`, absolute

kelpie serve
kelpie doctor

Stopping runaway loops (OpenClaw has no loop guard by default): the OpenClaw quickstart.

GitHub Issues (Copilot coding agent, Codex cloud, the Claude Code Action, OpenHands)

For agents that take an issue and return a pull request, the issues are the task queue:

# kelpie.toml
[adapters.github_issues]
repos = ["your-org/your-repo"]
token_env = "KELPIE_ISSUES_TOKEN"   # Issues, Pull requests: read; Issues: write only to act
label = "agent"

[adapters.github_issues.agents.copilot]
assignee = "Copilot"

kelpie serve
kelpie doctor

Step by step: the GitHub Issues quickstart. Linear works the same way ([adapters.linear], teams = ["ENG"], an API key in $LINEAR_API_KEY; agents are the app users issues are delegated to): configuration.

Other platforms: CONTRIBUTING explains how to add an adapter.

Docker

From a clone of the repository:

git clone https://github.com/ch8er/kelpie.git && cd kelpie
cp .env.example .env   # set GRAFANA_ADMIN_PASSWORD
docker compose up      # Kelpie, Prometheus and Grafana with Kelpie's dashboard

Dashboard: every metric on one Grafana dashboard, plus Prometheus alerts about Kelpie itself; with or without compose (Homebrew on macOS): dashboard.

How it works

• A snapshot, then rules. Every 30 seconds Kelpie builds one immutable snapshot of the farm (tasks, owners, dependencies, conditions, sessions, spans, resources) and runs every rule over it. Rules are pure functions: the same snapshot gives the same actions.
• Actions go through the platform. Requeue, unblock, wake, comment, stop, cancel a build, escalate: each goes to the adapter or connector that can carry it out, through a ledger with cooldowns and rate limits, so Kelpie never loops on its own output. What a platform cannot do becomes an alert.
• Observe, suggest, act. Each rule has a mode, and a global dry-run overrides them all. The act checklist is the gate before letting a rule act.
• Wait for conditions, not in sessions. An agent registers what it waits for (CI green, PR merged, quota reset, a human's task) and exits; Kelpie wakes the task when it holds.
• A daily report. Hours and tokens lost to chaos, the biggest wait by cause with a suggestion, what Kelpie fixed and would have fixed, what needs a human, farm throughput per role, the top 3 recommendations, and the before/after effect of recent fixes. An LLM root-cause section is optional and off by default.
• Use what exists. Host metrics come from node_exporter and Prometheus, dashboards from Grafana, and traces can be forwarded to Langfuse, Phoenix or Datadog. Kelpie exposes /metrics.

The rules:

The incidents behind each rule, the design and the daily report in full: How Kelpie works.

Documentation

• How Kelpie works: rules, daily report, alerts, traces, budgets
• Install and run: uv, Docker, kelpie doctor, ports, tokens, data
• Several machines: kelpie agent on each, a central kelpie serve (laptop + cloud VM, two VMs)
• Dashboard and alerts: Grafana and Prometheus with or without docker compose; the runbook
• Configuration reference: every section, rule and threshold
• Quickstarts: Hermes, Claude Code, OpenClaw, GitHub Issues; the observe-to-act checklist
• Privacy: what Kelpie stores from traces and hooks, and how to keep less
• Extending Kelpie: rules, adapters, connectors, report sections
• Changelog · Roadmap · Plan · Releasing

Contributing

Incidents from your farm are the most useful contribution: each one becomes a synthetic fixture and a rule test. CONTRIBUTING.md covers incidents, rules and adapters. Security issues: SECURITY.md.

License

Apache License 2.0, see LICENSE. PyPI: kelpie-sre · CLI: kelpie.