autoOps

An autonomous SRE agent. Point it at a Kubernetes namespace and it models the estate, names the service that is broken, shows the evidence it used, and re-reads the cluster to check its own answer — or hands a human a briefing saying why it could not.

Prime objective: an agent that operates real estates unaided through one core runtime loop. The measure of done is transfer — the loop working on an estate it was not built for (ADR-0038 D6) — never a benchmark score.

What works today. The nine-stage loop is built and has run against real clusters read-only. Perception is deterministic and comes first: 35 senses read the cluster's own declared state and nominate what looks broken, while metrics, traces and logs supply the evidence that either confirms a candidate or refuses it. A sense never votes on its own nomination. Diagnosis then reaches a confident answer or escalates, and says which.

What it deliberately will not do. It does not act. Stage 8 emits an incident note or a remediation proposal, never a change. The one typed mutation verb that exists, k8s_scale_workload, has an executor that raises unconditionally and is reachable from no live path — the boundary is code, not policy. It stays locked until the reasoning is trustworthy enough to earn it.

Install

uv tool install autoops-sre                      # or: pip install autoops-sre
docker pull ghcr.io/epappas/autoops:latest       # entrypoint is `autoops`

The distribution is autoops-sre; the command it installs is autoops. The bare name autoops on PyPI belongs to an unrelated project — do not install it.

autoops version && autoops --help

Three paths — a Python tool, a container image, or a source checkout — with the exact commands, what the image does and does not contain, and how to verify its provenance: docs/operations/install.md.

The Loop

recall -> understand -> [detect -> map topology -> map services -> localize] -> rca -> mitigate -> remember

The bracket is the investigation. rca matches over everything inside it plus what was recalled; remember closes the loop so the next run starts from a model rather than from nothing. Perception is deterministic and precedes action — the estate model generates the agent's moves, not the other way round (ADR-0038, ADR-0039).

Try It Without A Cluster

A real model drives the real agent stack over committed recordings — no cluster, and no credentials beyond an LLM key. The run is graded against the fixture's known answer, so you see both what it concluded and whether it was right.

uv sync
export OPENROUTER_API_KEY=...            # or put both in .env (never committed)
export OPENROUTER_MODEL=anthropic/claude-opus-4.8

uv run autoops offline --fixtures tests/fixtures/observability --results-dir ./out

It writes the trajectory, results.tsv and — only if it was wrong — a miss record, all under ./out. Four problems are available: --problem localization (default), detection, rca, cascade-localization.

Running It Against A Cluster

Read docs/operations/operator-runbook.md — the operator's document: prerequisites, exit codes and what to do about each, the exact shape of a confident answer and of an escalation, troubleshooting, and the safety rails.

The short version, on a loopback (kind/k3s) cluster:

uv run autoops diagnose --namespace <ns> --results-dir ./out

It runs a three-vote panel and decides fail-closed: exit 0 is a confident answer, exit 3 is an escalation briefing for a human, exit 4 means every sense came back without a read and no answer above it can be trusted.

Once an estate is written into a config file — ./autoops.yaml, or --config <path>, or $AUTOOPS_CONFIG — a routine diagnosis is autoops diagnose --namespace <ns> and nothing else. Every flag overrides the config, the config overrides the environment, and with no config file every command resolves exactly as it did before (ADR-0206).

autoops watch is the continuous mode — scheduled panels with a cadence floor, a rolling token budget, and escalations delivered to a directory or a webhook.

Any cluster that is not loopback is refused before a single read until it is named in an allowlist the owner has signed out of band. That refusal is the authorization boundary and it lives in the loader, not in the model.

The Command Surface

Seven verbs, plus version (ADR-0206, ADR-0208):

Every pre-ADR-0206 command name still dispatches, hidden and with a deprecation notice naming its replacement, because committed evidence manifests and estate crossing scripts record those invocations verbatim.

Safety

autoOps is built to run against real infrastructure, so the constraints come first:

• The model is never the authorization boundary. Every tool call passes a deny-by-default policy gate before execution, and gets a structured reason it can replan on rather than a bare refusal.
• Read-only by construction on the shipped path. The Cedar read-only profile plus a destructive re-check; a diagnosis cannot mutate.
• Approvals are short-lived, target-scoped and re-checked before every target contact — deleting an allowlist entry aborts a run already in flight.
• Secrets never enter the model context or the logs.

Documentation

docs/ is the map — every document, what it answers, and who it is for. The four an operator needs:

Design

• Decisions: docs/architecture/adr/ — ADR-0038 and ADR-0039 define the loop; ADR-0113 records why a sense is a node in the estate graph rather than a vote for a culprit.
• Architecture: system-architecture.md (structure) · incident-loop.md (behaviour) · data-plane.md
• Estates: deploy/estates/ — eighteen deployable fault estates, each with its own contract and offline validators.
• Research: docs/research/ · beliefs that constrain the design live in _meta/beliefs.md.

Contributing

make help        # every target
make check       # the merge gate: lint, imports, types, dead code, tests + coverage, security, estates

See CLAUDE.md (= AGENTS.md) and CONTRIBUTING.md. The hard rule: no detection, localization, RCA or mitigation capability is "done" without an end-to-end run that closes the telemetry loop — re-reading real cluster state to verify the outcome, not the agent's self-report.

License

MIT — see LICENSE.