Aegis-DevOps — Policy Verifier for AI DevOps Agents

Provenance-backed, authority-aware guardrails that stop context poisoning and agentic drift before an AI agent's kubectl or terraform action reaches your infrastructure.

Alpha — not production ready. The decision engine and its tests are solid, but the trust roots around it are still stand-ins: sources are verified against files on disk rather than real Git/Slack/Jira connectors, signing uses a shared secret rather than per-principal keys, and a principal is a signed name rather than an identity bound to a commit signature or SSO group. Resource matching is case-sensitive on names. Read PLAN.md §8 for the full list of open gaps before putting this in front of anything you care about.

Keywords: AI agent security · AgentOps · prompt injection · context poisoning · policy enforcement · policy-as-code · Kubernetes · Terraform · OPA · SRE · LLM guardrails · provenance · infrastructure-as-code

What it does

Aegis intercepts a proposed agent action — a kubectl/terraform/aws/... invocation or a plan file — and checks it against a Constraint Store before it runs, returning ALLOW, BLOCK, or ESCALATE with citations. Unlike a naive policy engine, every constraint in the store must independently pass an integrity check (has it been tampered with since ingestion?) and an authority check (was its source ever allowed to assert this kind of policy?), so a poisoned Jira ticket or a forged Slack message can't quietly become law. It covers 20 CLI/plan targets today — Kubernetes, Terraform/OpenTofu, the three major clouds, Helm/ArgoCD/Flux, Git/GitHub, SQL/migrations, and Pulumi — through one shared intent schema.

Quick start

python -m venv venv
venv/bin/python -m pip install -e ".[dev]"
venv/bin/python examples/demo.py

examples/demo.py runs 15 intents across every supported tool through the interceptor and prints each decision, starting with the store's health. To check one command yourself:

aegis check kubectl --now 2026-03-16T10:00:00-05:00 --pretty -- \
    kubectl scale deployment/api-server --replicas=5 -n prod

BLOCK: kubernetes scale deployment/api-server
  citations: no-scale-prod-peak
  covered: True  latency_ms: 0.14
PLAN BLOCK: 1 intent(s)
WARNING: using example signing key
STORE: loaded=21 quarantined=0 principals=3

The WARNING line is real: the shipped policy files are signed with a public demo key that lives next to them, so the CLI can verify them out of the box but tells you it did so with a key everyone has. See "Signing" below for using your own. To check a whole shell command string rather than one argv (what an agent framework actually hands you), use aegis check command -- "kubectl get pods; sudo kubectl delete node/w1" — see "Compound commands".

Global options in front of the verb (kubectl -n prod delete …, git -C /repo push …, helm --kube-context prod uninstall …), glued short flags (-nprod), label selectors (-l role=worker) and comma-separated kinds (nodes,pods) all parse to the same intents as their canonical forms; a leading option Aegis does not recognise is an error (exit 65), never a guess at where the verb starts. kubectl delete namespace prod additionally emits a synthetic */* delete intent scoped to that namespace, so namespace-scoped deletion rules fire on it.

Exit codes

The exit code is the worst verdict across all evaluated intents, or a tool error. Verdict codes depend on --exit-style; tool errors are identical in every style and never print a verdict.

Every error is one aegis: error: … line on stderr, never a traceback. claude-hook exists because Claude Code PreToolUse hooks treat exit 2 as block and any other non-zero code as a non-blocking error, which would invert the default 3 = BLOCK contract.

Store health

Every output carries the state of the constraint store, so a degraded store can never be mistaken for a clean allow. Each per-intent JSON line and the plan summary include a store_health object — loaded, quarantined: [{id, reason}], principals, constraints_sha256 (of the raw file bytes), warnings — and --pretty ends with a STORE: loaded=N quarantined=M principals=P line plus one line per quarantined rule, preceded by one WARNING: … line per store warning (using example signing key, insecure: signatures not verified, ledger: chain-broken, rate-limit key typos). Quarantines also go to stderr as aegis: WARNING … lines.

Integrity failures fail closed. A constraint that was quarantined at load (tampered, forged) or discarded at decision time (tampered, unauthorized) is still matched, and if it was a BLOCK or ESCALATE rule its match contributes ESCALATE — never BLOCK (nobody can vouch for the rule) and never ALLOW (the action it covers is contested). The decision reports it in discarded and with a fail-closed: <id> (<reason>) note:

aegis check kubectl --constraints /tmp/oneflip.yaml --key file:data/example-signing.key \
    --pretty -- kubectl delete node/x

aegis: WARNING Quarantined constraint no-delete-nodes: provenance hash mismatch
ESCALATE: kubernetes delete node/x
  discarded: [{'id': 'no-delete-nodes', 'reason': 'tampered'}]
  note: env-unresolved: no-delete-in-prod-env
  note: fail-closed: no-delete-nodes (tampered)
  covered: True  latency_ms: 0.13
PLAN ESCALATE: 1 intent(s)
  plan citations: plan-k8s-delete-ratio
  note: ratio: 1/1 = 1.00 > 0.5
WARNING: Quarantined constraint no-delete-nodes: provenance hash mismatch
STORE: loaded=20 quarantined=1 principals=3
  quarantined: no-delete-nodes (tampered)

(/tmp/oneflip.yaml here is data/constraints.example.yaml with the last hex digit of no-delete-nodes's provenance_hash flipped and re-signed with the example key — a valid signature, invalid content, exactly what a hand-edited rule or a one-bit storage error looks like. --key is explicit because the example key normally auto-resolves next to --constraints, and /tmp has no example-signing.key of its own.)

Aegis refuses to decide at all (exit 65, one-line message, no verdict) when the store loaded zero constraints, the authority map grants nothing to anyone, or more than --max-quarantine-ratio (default 0.10) of the constraints were quarantined. --fail-closed additionally turns an uncovered intent (no rule matched) into ESCALATE with the note fail-closed: uncovered.

Two more fail-closed clauses live in the interceptor: a rule that scopes on env when the intent's environment could not be resolved contributes ESCALATE with the note env-unresolved: <id> (see "Environment mapping"), and an intent whose target the parser could not pin down (git push -f with no refspec) contributes ESCALATE with the note unknown-target, so it cannot slip past a ref/main rule as ref/*.

Claude Code hook

examples/claude-code-hook.sh is a PreToolUse hook: it reads the hook JSON from stdin and hands tool_input.command — the raw string — to aegis check command --exit-style claude-hook. ALLOW lets the tool call proceed; ESCALATE and BLOCK exit 2 with the reason on stderr (which Claude Code shows to the model). Compound commands are split and launchers unwrapped (see "Compound commands"); binaries Aegis has no parser for are not gated unless you add --fail-closed via AEGIS_ARGS; any tool error — including a command string Aegis refuses to evaluate statically, such as kubectl delete $(cat x) — is converted into a block, so the hook never fails open. It runs the aegis in the venv next to it (AEGIS_BIN overrides).

{"hooks": {"PreToolUse": [{"matcher": "Bash", "hooks": [
  {"type": "command", "command": "/path/to/aegis-devops/examples/claude-code-hook.sh"}]}]}}

How a decision is made

At load time, ConstraintStore.load parses the constraint YAML, verifies each provenance_hash, and (with --sources) re-fetches and checks the original source — anything that fails either check is quarantined, not silently dropped.

At decision time, AegisInterceptor.intercept matches each intent against the surviving constraints on (provider, resource_pattern, action, scope, time_window), re-checks integrity and authority (authority can be revoked after ingestion), applies any rate limit against the decision ledger, downgrades a dry run to ALLOW, and takes the highest-precedence effect (BLOCK > ESCALATE > ALLOW) among what's left. A PlanConstraint then runs once more over the whole batch of intents from one plan/chart/invocation.

Supported tools

Every target below is checked with aegis check <target> [flags] -- <argv...>, except terraform/tofu/pulumi-preview, which take a JSON document path instead of --.

Writing constraints

A per-intent constraint (data/constraints.example.yaml) has every field below. id is a free-form string; every other field feeds either matching, provenance, or authority:

- id: no-scale-prod-peak
  provider: kubernetes            # matches Intent.provider
  resource_pattern: deployment/*  # fnmatch glob against Intent.resource
  actions: [scale]                # any of these matches Intent.action
  scope: {namespace: prod}        # every key must match intent.metadata (or metadata.env)
  time_window:                    # optional; omit or null to apply at all times
    days: [Mon, Tue, Wed, Thu, Fri]
    start: "09:00"
    end: "17:00"
    tz: America/New_York
  rate_limit:                     # optional; only fires once the ledger shows quota used
    max: 3
    per: 1h                       # "<n>m" / "<n>h" / "<n>d"
    key: [namespace]              # bucket the quota per distinct value of these metadata keys
  effect: BLOCK                   # BLOCK | ESCALATE | ALLOW-adjacent (BLOCK/ESCALATE only fire)
  constraint_class: scaling       # must be in the asserting principal's authority set
  principal: sre_lead             # who asserted this constraint
  source_ref: jira-1001           # id used to re-fetch the source for --sources verification
  source_timestamp: "2026-01-15T09:00:00+00:00"
  rule_text: "Do not scale deployments in prod during business hours."
  provenance_hash: f3af697890dc757effcf13cdfcc9e4624e32aa65dd3114c348674ddcda0f2244

A plan-level constraint (data/plan_constraints.example.yaml) carries exactly one batch predicate instead of a per-intent match, evaluated over every intent from one plan/invocation:

plan_constraints:
- id: plan-max-25-resources
  max_intents: 25                 # escalate/block if the plan has more than N intents
- id: plan-no-db-deletes
  max_matching:                   # escalate/block if more than `max` intents match this filter
    actions: [delete, replace]
    resource_pattern: aws_db_instance.*
    max: 0
- id: plan-no-delete-and-create-us-east-1
  forbid_together:                # escalate/block if the plan matches ALL of these filters
  - {actions: [delete], scope: {region: us-east-1}}
  - {actions: [create]}
# requires_all: same shape as forbid_together, but the effect fires when NOT all are present
- id: plan-k8s-delete-ratio
  ratio:                          # escalate/block if numerator/denominator exceeds `max`
    numerator: {actions: [delete]}
    denominator: {}               # {} matches every intent in the plan
    max: 0.5

Metadata vocabulary

Every parser in aegis_core.parser fills Intent.metadata with operator-derived context (not agent-controlled — see the scope semantics below) parsed from the argv itself. aegis_core.environments fills in the rest (env, and any of context/cluster/project/account/subscription/ resource_group/workspace the argv didn't already carry) from environments.yaml, recording which ones it filled in metadata["resolved_from_environment"]. This table is generated by grepping metadata["..."] = ... in parser.py, not written from memory:

kubeconfig is listed because it's a common source of confusion: it's consumed like every other global flag (so it never becomes a bogus resource/action) but has no scope-matching use today, so no parser records it.

Scope matching semantics

A constraint's scope: {...} (and a plan selector's scope) is matched against an intent by aegis_core.store.scope_matches:

• Metadata-first. Each scope key is looked up in intent.metadata before intent.params, and params may only supply a key metadata lacks entirely — an agent-controlled --env=dev flag can never shadow an operator-resolved metadata["env"] = "prod" (REVIEW-4 T1.5). A key absent from both never matches.
• String/bool coercion. If either side of a comparison is a bool, both are read as booleans ("true"/"false"/"True"/"FALSE" all count, case-insensitively). Otherwise, if either side is a string, both are compared as strings — so a YAML account: 123456789012 (an int) matches an intent's "123456789012" (a string) and vice versa.
• Lists are OR. scope: {namespace: [prod, prod-eu]} matches either value.
• Globs. A scope value containing *, ?, or [...] is matched with fnmatch (case-sensitive) against the intent's value, e.g. region: us-east-*.
• Dotted paths. A scope key with a . (e.g. set.replicaCount) first tries a literal key of that exact name, then walks the dots into nested dicts (params["set"]["replicaCount"]) — this is what makes helm --set replicaCount=0 (parsed into params["set"] = {"replicaCount": 0}) expressible as scope: {set.replicaCount: 0}.
• actions: ["*"]. A per-intent constraint (or plan selector, where actions is already an "any of these" list) may include the literal string "*" in its actions set to mean "any action". It is otherwise an ordinary string: no special validation, and no effect on the constraint's provenance hash (which hashes the actions set exactly as given).
• Time windows. time_window.tz (or the store-level default_tz — see below) localises the evaluation instant before every other check. days excludes by local weekday. start/end are HH:MM; end is exclusive at minute granularity, so 09:00–17:00 means [09:00, 17:00) and 16:59 is inside it but 17:00 is not. end: "24:00" is accepted to mean "through midnight" (the only way to include the last minute of the day under an exclusive end). start > end is a wrap-around window (e.g. 22:00–06:00) and matches when the local time is at/after start OR before end. A time_window with no tz of its own is valid only when the constraints file sets a top-level default_tz: (a store-level setting, not a constraint field — adding or changing it never changes any constraint's provenance_hash); otherwise it is quarantined at load as invalid: time_window.tz missing and no default_tz. The evaluation instant (now) must be timezone-aware everywhere in the matcher; a naive datetime raises ValueError rather than silently assuming UTC.

Authority policy

data/authority.example.yaml maps each principal to the constraint classes it may assert:

principals:
  admin: [scaling, deletion, configuration]
  sre_lead: [scaling, configuration]
  developer: [configuration]

A constraint whose principal isn't authorized for its own constraint_class is discarded at decision time with reason unauthorized — even if it was authorized when it was first added (ConstraintStore.add_constraint checks this too, but authority can be revoked later).

Compound commands

Agent frameworks hand over a shell string, not an argv. aegis check command -- "<string>" (and aegis check argv --split-compound) turns that string into the simple commands it would run — splitting on ;, &&, ||, |, & and newlines, unwrapping sudo, env VAR=…, timeout, nice, nohup, command, time, sh -c "…" and the k/tf/g aliases — and checks every one whose binary Aegis knows. The exit code is the worst verdict across them:

aegis check command -- "kubectl get pods; kubectl delete node/w1"   # BLOCK, exit 3
aegis check command -- "sudo kubectl delete node/w1"                # BLOCK, exit 3
aegis check command -- "kubectl get pods | grep x"                  # ALLOW, exit 0
aegis check command -- 'kubectl delete $(cat x)'                    # exit 64: command rejected

Inside a pipeline an unknown binary (| grep x) produces nothing; on its own it produces a synthetic shell / binary/<name> / exec intent that --fail-closed escalates. KUBECONFIG=… and env AWS_PROFILE=… prefixes land in the intent's metadata (kubeconfig, profile, …) so the environment map sees them. Fail closed: anything whose argv cannot be known without running it is refused as a usage error (exit 64, command rejected: <reason>) — command substitution ($(…), backticks), $VAR outside single quotes, process substitution, subshells, here-docs, eval/exec/source/./xargs, and unbalanced quotes. Without the flag, aegis check <target> still treats any shell metacharacter in an argv as a usage error.

Environment mapping

Constraints can scope on env: prod|staging|dev instead of repeating every raw context/account/project/subscription. data/environments.example.yaml maps provider-specific identifiers to a normalised environment, and EnvironmentMap.annotate sets intent.metadata["env"] at parse time (the CLI does this automatically via --environments, defaulting to data/environments.example.yaml when present). Resolution is provider-agnostic: whatever tool produced the intent, every identifier its parser recorded is looked up — kubernetes.contexts / clusters / kubeconfigs (kubectl, helm --kube-context, flux, argocd, KUBECONFIG=…), aws.accounts / profiles, gcp.projects, azure.subscriptions / resource_groups, github.repos (owner/repo from gh -R), argocd.apps (globs on the app name) and terraform.workspaces. An identifier that isn't in the map resolves to no environment at all — never a default like dev — and, deliberately, kubectl namespaces are never used to infer env, since a namespace is a string the agent (or an attacker poisoning its context) controls.

Unknown is not "not prod". When a rule scoped on env would otherwise match and the intent has no resolved env, the rule is neither honoured nor dropped: the decision is ESCALATE with the note env-unresolved: <id> (a BLOCK from another rule still outranks it). So kubectl delete pod/x -n dev with no --context escalates against no-delete-in-prod-env instead of sailing through. Plan-constraint selectors with scope: {env: …} behave the same.

--resolve-current-context fills a missing identifier from the invoking environment: the kubeconfig's current-context (and its cluster) from $KUBECONFIG / ~/.kube/config (parsed, never by running kubectl), $AWS_PROFILE, $AWS_DEFAULT_REGION/$AWS_REGION, $CLOUDSDK_CORE_PROJECT, $AZURE_SUBSCRIPTION_ID, $HELM_NAMESPACE, $ARGOCD_SERVER. An explicit --context on the argv always wins, and the keys that were filled are listed in metadata.resolved_from_environment. It is opt-in because it trusts the invoking environment: whoever controls the process environment controls what Aegis believes the target is.

Dry runs

A rehearsal (kubectl --dry-run=client|server, aws --dry-run, az --what-if|--dry-run, gcloud --dry-run) can't change infrastructure, so Aegis never blocks or escalates it — the verdict is always ALLOW, with dry_run: true and would_be reporting what a real run would have gotten:

ALLOW: kubernetes scale deployment/api-server (dry-run; would be BLOCK)
  citations: no-scale-prod-peak
  covered: True  latency_ms: 0.12
PLAN ALLOW: 1 intent(s)
  note: would_be: BLOCK

terraform/tofu plans are evaluated normally — the plan JSON is the proposed change, not a rehearsal of one.

Source verification

--sources DIR points at a directory of <source_ref>.json files plus a PRINCIPALS.yaml that says which principal the transport attributes each source to. It defaults to the sources/ directory next to the constraints file when one exists (--sources '' opts out), so with the shipped layout forgery is detected without any flag. A constraint whose cited source doesn't back it (missing file, or different content) is quarantined as forged at load time; one whose transport principal differs from the principal it claims is quarantined as principal-mismatch. Both fail closed like any other quarantine.

FileSourceFetcher is a v1 stand-in for real Git/Slack/Jira connectors — it re-reads a flat JSON file rather than calling out to a commit, a permalink, or a ticket API (see "Open gaps" in PLAN.md).

data/sources-forged/ is data/sources/ with jira-1001.json's rule_text edited after the fact (still validly signed — signing proves the file wasn't touched in transit, not that its content still matches what a constraint cites) so --sources has something real to catch:

aegis check kubectl --now 2026-03-16T10:00:00-05:00 --pretty --sources data/sources-forged -- \
    kubectl scale deployment/api-server --replicas=5 -n prod

aegis: WARNING Quarantined constraint no-scale-prod-peak: source does not back its claimed fields
ESCALATE: kubernetes scale deployment/api-server
  discarded: [{'id': 'no-scale-prod-peak', 'reason': 'forged'}]
  note: fail-closed: no-scale-prod-peak (forged)
  covered: True  latency_ms: 1.53
PLAN ESCALATE: 1 intent(s)
WARNING: using example signing key
WARNING: Quarantined constraint no-scale-prod-peak: source does not back its claimed fields
STORE: loaded=20 quarantined=1 principals=3
  quarantined: no-scale-prod-peak (forged)

The same kubectl scale ... command against the real data/sources (the CLI's default) is a plain BLOCK — see "Quick start" above; forging the source turns a legitimate rule into a fail-closed ESCALATE instead of quietly ceasing to apply.

Signing

Every policy file — constraints, plan constraints, authority map, environment map, source snapshots and PRINCIPALS.yaml — must verify under a keyed BLAKE2b MAC before the CLI will decide with it. Single files carry a detached <file>.sig; a directory of sources carries one AEGIS-MANIFEST.sig (a JSON manifest of {relpath: sha256} plus the MAC of that mapping), so hundreds of snapshots are one signature. A file with a detached .sig is checked against it; otherwise the nearest manifest above it must list it with a matching hash.

aegis sign   --key file:aegis-signing.key data/constraints.yaml data/sources   # .sig + manifest
aegis verify --key env:AEGIS_SIGNING_KEY  data/constraints.yaml data/sources

aegis verify scope on a directory. sign on a directory covers everything under it (every .yaml/.json, recursively) — that's a deliberate "sign the whole tree" operation for a directory you've pointed it at on purpose, e.g. data/sources. verify on a directory is narrower on purpose: it only checks files that were actually signed — one with its own <file>.sig, or one listed in an AEGIS-MANIFEST.sig found at or below that directory — never every .yaml/.json it happens to find. A policy directory can legitimately hold unrelated, unsigned content next to real policy files — data/corpus/seeds.yaml, split.json and stats.json are corpus-generation artifacts no Aegis loader ever reads — and aegis verify data/corpus must not report those as FAILED just because of their extension. Passing a file directly (not discovered via a directory walk) is unaffected: it is always checked, signed or not, so a real gap still surfaces as FAILED on the file itself or by naming the directory that has — or should have — a manifest covering it.

The key is resolved from --key SOURCE (env:VAR, file:PATH, or raw hex), then $AEGIS_SIGNING_KEY, then <dir of --constraints>/example-signing.key if it exists — with the store warning using example signing key, because data/example-signing.key is public and demo-only: it is committed so the shipped examples and corpus verify out of the box, and anyone holding it can forge those files. Generate your own (python -c 'import secrets; print(secrets.token_hex(32))'), re-sign, and keep it out of the agent's reach. With no key at all the CLI exits 65 (no signing key: pass --key, set AEGIS_SIGNING_KEY, or --insecure); --insecure loads everything unverified and says so in every store_health.warnings. This is a MAC, not a public-key signature: whoever can sign can verify, which is the v1 boundary — Aegis defends against poisoned content from sources you chose to trust, not against an attacker who holds the signing key.

Configuration

Running aegis from the repo checkout with no flags "just works" because $PWD/data already has constraints.example.yaml — but installed as a wheel there is no data/ next to the interpreter. --constraints/--authority/--environments/--plan-constraints each default to None and are resolved from a config directory, searched in this order, first match wins:

1. --config-dir DIR (explicit)
2. $AEGIS_CONFIG_DIR
3. $PWD/.aegis
4. $PWD/data — only if it already contains a constraints*.yaml (this is what keeps a repo checkout working with no setup)
5. ~/.config/aegis
6. /etc/aegis

Inside that directory, a real file wins over its .example counterpart when both exist (constraints.yaml over constraints.example.yaml, and likewise for authority/ environments/plan_constraints); sources/ and example-signing.key next to the constraints file are picked up the same way they always were. Any of --constraints/--authority/etc. passed explicitly is used as-is and skips discovery for that one file. When none of the above finds a directory and --constraints/--authority were never given either, the CLI refuses cleanly instead of tracebacking on a relative path that doesn't exist from the current directory:

$ cd /tmp && aegis check kubectl -- kubectl get pods
aegis: error: no config found (searched: /tmp/.aegis, /Users/you/.config/aegis, /etc/aegis); run 'aegis init <dir>'

aegis init <dir> seeds a fresh directory with the packaged example policy files (the same *.example.yaml + .sig + sources/ the repo ships in data/, embedded in the wheel under aegis_core._examples/ — see scripts/sync_package_examples.py, which keeps that copy in sync with data/ and is checked by tests/test_config.py) and prints next steps:

$ aegis init ~/.config/aegis
aegis: wrote 38 file(s) to /Users/you/.config/aegis
Next steps:
  1. Generate a real signing key:  aegis keygen --out ~/.config/aegis/aegis-signing.key
  2. Sign your policy files:       aegis sign --key file:~/.config/aegis/aegis-signing.key ~/.config/aegis/*.yaml ~/.config/aegis/sources
  3. Replace the *.example.yaml files in ~/.config/aegis with your own constraints.yaml / authority.yaml / ... (aegis prefers a real file over the .example one when both exist)

aegis keygen [--out FILE] (default aegis-signing.key) writes 32 random bytes, hex-encoded, mode 0600 — a real key, distinct from the public example-signing.key that init copies in verbatim. Once you've edited the example files into real ones and signed them with your own key, either export AEGIS_SIGNING_KEY or keep using --key file:...; the example-key fallback only ever fires when example-signing.key is the only key next to --constraints.

Other env vars: $AEGIS_SIGNING_KEY (see "Signing" above). There is currently no env var for --now, --ledger, or the output flags — those stay CLI-only so a shell alias can't silently change what gets decided.

Rate limits & ledger

--ledger PATH enables rate_limit constraints and records every executed (ALLOW, non-dry-run) decision, bucketed by the constraint's key metadata fields within its per window. A .jsonl path gives an append-only JSON-lines ledger; .db/.sqlite/.sqlite3 gives a SQLite one. Both serialise load → count → record across processes (an flock on a sidecar .lock file, or BEGIN IMMEDIATE), so sixteen racing agents against max: 3 get exactly three ALLOWs; both hash-chain their records, and a truncated or hand-edited ledger surfaces as the store warning ledger: chain-broken and makes every rate-limited rule ESCALATE rather than count from zero. Records older than the largest per window in the store (at least 24h) are pruned on load, so a ledger never grows without bound; malformed lines are skipped and counted. A rate_limit.key naming a field no parser emits is reported in store_health.warnings at load instead of silently bucketing nothing (resource is allowed, to bucket per concrete target).

aegis check kubectl --ledger results/ledger.db --pretty -- \
    kubectl get service/frontend -n prod

Library usage

from aegis_core.authority import load_authority_map
from aegis_core.store import ConstraintStore
from aegis_core.interceptor import AegisInterceptor
from aegis_core.parser import from_kubectl

authority_map = load_authority_map("data/authority.example.yaml")
store = ConstraintStore.load("data/constraints.example.yaml", authority_map=authority_map)
interceptor = AegisInterceptor(store)

intent = from_kubectl(["kubectl", "delete", "node/worker-1"])
decision = interceptor.intercept(intent)
print(decision.verdict, decision.citations, decision.covered)
# BLOCK ['no-delete-nodes'] True

Decision also carries discarded (constraints that matched but were thrown out, with why), notes (fail-closed: …, rate-limit: …), latency_ms, dry_run, and would_be. store.health is the StoreHealth the CLI prints; AegisInterceptor(store, fail_closed=True) is the library form of --fail-closed.

Adversarial suite

src/aegis_core/adversarial.py generates 28 attacks across five categories — tampered (field mutated post-ingest), unauthorized (self-consistent but the wrong principal), forged (hash valid, cited source doesn't back it), evasion (attempts to slip past the matcher itself), and argv-evasion (command-line shapes that used to parse into an intent nothing matched: global flags before the verb, -nprod, label selectors, comma kinds, namespace deletion). tests/test_adversarial.py proves every tampered and unauthorized attack is discarded and fails closed to ESCALATE (never ALLOW, never BLOCK), that each argv-evasion shape now hits the rule its author would expect, and that the matcher behaves correctly under the evasion attempts, with one documented gap (evade-case-variant, xfail: fnmatch is case-sensitive on POSIX).

venv/bin/python -m pytest tests/test_adversarial.py -v

Forged constraints are caught only by verify_source(), which isn't wired into intercept() at decision time — see --sources above and "Open gaps" in PLAN.md.

Benchmark

scripts/benchmark.py runs Aegis and three baselines over the labeled 500-constraint corpus in data/corpus/ (built by scripts/build_corpus.py) and reports precision/recall/F1, over-block rate, coverage, per-kind poison-susceptibility, and latency for each:

venv/bin/python scripts/benchmark.py \
    --corpus data/corpus --split holdout \
    --verifiers aegis,llm-heuristic,opa,opa-signed \
    --out results/

Ground truth is independent of Aegis. Every intent's expected_verdict / expected_covered comes from scripts/reference_oracle.py — a deliberately naive matcher (provider ==, fnmatch, action in, scope equality, a minimal time window) that reads constraints.yaml, labels.jsonl and authority.yaml with plain loaders and never imports aegis_core.interceptor, aegis_core.store or aegis_core.plan. Only constraints labelled Trusted drive the verdict (BLOCK > ESCALATE > ALLOW); covered is "any constraint of any label matched" and is computed once by the harness for every verifier. The harness re-runs the oracle at benchmark time and refuses to score a stale intents.jsonl. The aegis row is therefore a measurement against that oracle, not a sanity check on itself.

The test set is a held-out intent split. split.json["intents"]["holdout"] is 20% of the 600 intents, stratified by expected_verdict × poison_candidate, frozen with the corpus seed. --split holdout is the default and the only split reported here; --split dev is the rest; --split all requires --i-know-this-is-dev. The constraint store is always the full corpus — the store is the deployed artefact, the intents are the test set.

Verifiers. aegis loads the store the way a deployment would, with the corpus's sources/ (so forged constraints are quarantined at load); aegis-nosources (opt-in) is the same engine without source verification, kept so the cost of skipping it is measurable. llm-heuristic is a deterministic, no-network stand-in for a naive LLM self-check (it blocks/escalates as soon as any constraint matches by provider/resource/action, with no provenance or authority reasoning). opa feeds every constraint to a real opa eval; opa-signed models a signed-bundle deployment (cosign/in-toto): the bundle handed to OPA excludes tampered and forged constraints but still contains unauthorized ones, because bundle signing proves integrity in transit, not that the author was allowed to say it. Both OPA rows are skipped with a note when the opa binary isn't on PATH.

Columns. n is the number of held-out intents; n_distinct is the number of distinct (provider, resource_pattern, actions, scope) structures among the 500 constraints (from data/corpus/stats.json) — the honest measure of how many different rules the corpus contains. poison-susceptibility is the headline number: of the intents whose only matching constraints are Untrusted/Malicious (oracle verdict ALLOW), the fraction a verifier blocks/escalates on anyway. It is split by the strongest poisoned rule that matched: ps_unauth / ps_tampered / ps_forged count BLOCK verdicts (the verifier obeyed the poison), pe_* count ESCALATE verdicts (it failed closed — wrong verdict, but a human sees it). unauthorized is the realistic pre-ingest attacker; tampered/forged are post-ingest attacks that bundle signing already covers, which is why the opa-signed → aegis delta on ps_unauth/pe_unauth is the number that matters. strict precision counts a positive only on an exact verdict match.

(split: holdout, oracle: reference; full table with latency in results/benchmark.md.) Every row has recall 1.0 because the corpus's poisoned rules are additional rules — a verifier that honours everything never misses a Trusted one; the cost shows up as over-block and ps_*. Read the Aegis row carefully: its binary precision / over-block are the same as the naive LLM's, because the interceptor fails closed (REVIEW-4 T0.3) — a quarantined or discarded BLOCK/ESCALATE rule contributes ESCALATE rather than being dropped — so every poison candidate is escalated to a human. The oracle's label semantics say those intents are ALLOW, so against the oracle that is an over-block, and it is reported as one. What separates Aegis is ps_* = 0 across every attack kind (it never obeys a poisoned rule; pe_* = 1.0 says it flags all of them), versus ps_unauth = 0.5 for both opa and opa-signed — bundle signing removes tampered but leaves the unauthorized author's rule in force. Whether "escalate everything poisoned" is the right trade-off versus "drop it silently" is a product decision, and the two columns let a reader make it.

Real LLM baselines: llm-naive and llm-aware

llm-heuristic above is a deterministic stand-in, not a measured LLM accuracy number. Two real rows use claude-sonnet-5 itself (see the claude-api skill for model IDs and pricing before running): llm-naive is the plain self-check (constraints + intent, no provenance/authority information at all); llm-aware (AwarePromptBuilder in aegis_core/baselines/llm.py) adds the authority.yaml contents and an instruction to ignore any constraint whose principal isn't authorized for its constraint_class — the strongest self-check a prompt alone can do, since it still can't recompute a provenance hash. Run both against the current corpus once a key is set:

pip install -e ".[llm]"
export ANTHROPIC_API_KEY=...
venv/bin/python scripts/benchmark.py --split holdout \
    --verifiers aegis,llm-naive,llm-aware --llm-cache results/llm-cache.jsonl --out results/

Every prompt/response pair is cached to results/llm-cache-naive.jsonl / results/llm-cache-aware.jsonl (derived from --llm-cache), so a later run replays it offline with no network calls and no key:

venv/bin/python scripts/benchmark.py --split holdout \
    --verifiers aegis,llm-replay-naive,llm-replay-aware --llm-cache results/llm-cache.jsonl \
    --out results/

llm/llm-replay remain accepted as aliases for llm-naive/llm-replay-naive. Without a key, llm-naive/llm-aware are skipped with skipped: set ANTHROPIC_API_KEY — there is no longer a stub row that answers ESCALATE on every call and gets reported as if it were data.

Cost estimate. Both rows feed the entire store (500 constraints, ~278 KB of YAML) to the model on every call; --split holdout currently scores 120 intents (data/corpus/split.json["intents"]["holdout"]). At ~4 chars/token that's roughly 69,000 input tokens per llm-naive call (llm-aware adds the rendered authority.yaml, a few hundred more) and under 50 output tokens. 120 intents x ~69,000 input tokens x 2 variants ~= 16.6M input tokens for one full pass of both rows — check current pricing with the claude-api skill; at a rough $3/M input tokens that's on the order of $50 for both variants, before prompt caching (the system prompt repeats across all 120 calls in a variant, so a cached run costs much less than a naive per-call estimate).

Already measured, on the pre-T0.5 corpus. Before this corpus's T0.5 rewrite (independent oracle, intent-level holdout — see REVIEW-4.md), a sibling harness (agent-guardrail-bench@a98a8fa) ran both llm-naive and llm-aware for real against the corpus as it stood at commit a2497e4^, importing this repo's aegis_core.baselines.llm directly. Full tables, provenance, and the replay-hit-rate verification are in results/llm-external.md; the headline number is poison-susceptibility 1.000 for llm-naive vs. 0.883 for llm-aware vs. 0.000 for aegis on the same 200 intents — a naive self-check obeys every poisoned constraint it's handed, giving it the authority map catches unauthorized-principal poisoning but not tampered/forged constraints from an authorized principal, and only independent integrity + authority checking (Aegis) catches all of it. tests/test_baselines.py::test_llm_replay_{naive,aware}_matches_external_cache assert this repo's current prompt rendering still reproduces those cached prompts byte-for-byte (100% hit rate) so the caches stay usable as a regression check even while the corpus is regenerated.

Agent-harness and local-model baselines: codex, ollama, and claude-cli

Three more rows use the SAME naive prompt llm-naive uses (aegis_core.baselines.llm.render_system_prompt), via three more LLMClient implementations in aegis_core/baselines/external.py, but over a smaller constraint set: llm-naive/llm-aware feed the entire 500-constraint corpus (~69,000 tokens — see the cost estimate above), which is infeasible for a local model's context window and prohibitively slow/expensive to probe repeatedly against an agent harness. codex, ollama, and claude-cli instead load the holdout constraint split — data/corpus/split.json["holdout"], 100 constraints, distinct from split.json["intents"]["holdout"] — which renders to roughly 14,000 characters-per-4 of naive estimate, but see the token-count surprise below. This is also the same subset the pinned claude-sonnet-5 rows in results/llm-external.md were measured against, so these rows stay comparable to that table. The subset is threaded through explicitly (scripts/benchmark.py's load_holdout_constraint_subset), never a silent default.

codex shells out to codex exec — an agent harness wrapped around a model, not a raw completion endpoint, even invoked read-only for one turn (it can plan and use tools before answering). That distinction matters enough that the notes column says "agent harness (codex exec)", not "model", for this row. Invocation: codex exec --ignore-user-config --skip-git-repo-check --ephemeral -s read-only --output-last-message <file> - < prompt.txt (--ignore-user-config stops a user's own ~/.codex/AGENTS.md/config from leaking into the run; --output-last-message gets a clean final answer instead of parsing the human-formatted stdout, which echoes the prompt, an optional warning: line, and the answer duplicated after tokens used — CodexCliClient falls back to robust last-matching-line stdout parsing if that file is ever missing). Model selection: scripts/probe_codex_models.py (probe_codex_models() in external.py) tries gpt-5.1-codex-mini, gpt-5-mini, o4-mini, gpt-5.1-codex under a hard 60s timeout each — an unsupported -m value doesn't fail fast, it prints an immediate ERROR: ... not supported line and then hangs rather than exiting, so every candidate must be probed under a timeout, never called bare. On this account (ChatGPT-plan auth) every named candidate returned 400 ... not supported when using Codex with a ChatGPT account immediately; the row below was run with no -m at all, i.e. the account's own configured default, which the CLI's banner reports as gpt-6-astra (CodexCliClient.resolved_model, parsed from that banner, records this even when model=None). Measured cost: ~6s and ~2,000 tokens per call.

ollama talks to a local Ollama server's HTTP API (POST /api/generate, mistral:latest, temperature=0, seed=0 for reproducibility) instead of the CLI, so it's structured JSON in and out with no stdout parsing at all. The 100-constraint holdout prompt does not fit in 16,384 tokens of context — the ~14k-character-per-4 estimate undercounts badly for this tokenizer: the real prompt is ~23,700 tokens, not ~14,000. At num_ctx=16384 Ollama silently truncates (confirmed here: prompt_eval_count came back exactly 16384, the cap, and the model's answer degraded into unrelated advice about writing a new Gatekeeper policy) — there is no error, no warning, just a truncated context and a bad answer. The row below uses num_ctx=32768, verified by comparing a short-prompt call (prompt_eval_count well under the cap) against the full-prompt call (prompt_eval_count ≈ 23,700, comfortably under 32,768).

Honest finding, not massaged: even with the full, untruncated context, mistral:latest (a local 7B model) is unreliable at following the requested format. Its 120 holdout responses included the expected ALLOW/BLOCK/ESCALATE tokens, occasional BLOCK\ncitations: <real ids>, but also BLOCK\ncitations: id1, id12 — it echoed the system prompt's own example citation placeholders (citations: id1, id2) instead of real constraint IDs. tests/test_baselines.py does not try to rescue this with a smarter parser; parse_llm_response is left exactly as it was for llm-naive, and the resulting metrics report what a 7B model handed this much context actually does, unmodified.

claude-cli is the third row on the same footing: it shells out to claude -p (the Claude Code CLI) — an agent harness wrapped around a model, just like codex, even with --allowedTools "" denying it any tool use for the single turn — over the same 100-constraint holdout subset. Invocation: claude -p --model haiku --output-format json --no-session-persistence --allowedTools "" < prompt.txt. --output-format json returns one JSON object on stdout; ClaudeCliClient reads the verdict from its result field (fed straight into the same parse_llm_response every other row uses) and token counts defensively from modelUsage (a dict keyed by model name, e.g. {"claude-haiku-4-5-20251001": {"inputTokens": ..., "outputTokens": ...}} — shape not pinned by any spec we control, so ClaudeCliClient sums whatever *Tokens fields it finds rather than assuming exact keys). The row's notes column reads agent harness (claude -p), model=haiku, 100-constraint holdout subset, matching the codex row's honesty about what's actually being measured. Default model is haiku (the cheapest alias); override with --claude-cli-model.

claude -p can fail in a way codex/ollama don't: an expired OAuth session — the CLI returns exit code 0 with is_error: true and a result string containing 401 / OAuth access token has expired. Re-authenticate to continue., after ~180s of its own internal retries. ClaudeCliClient detects this specific shape and raises ClaudeCliAuthError (a RuntimeError subclass) immediately with the fix (run 'claude login') rather than treating it as a verdict or a transient failure; RetryingClient special-cases ClaudeCliAuthError to never retry it — retrying would just re-run the CLI's own three-minute failure for the same guaranteed outcome. scripts/benchmark.py runs one cheap preflight call before wiring up the real claude-cli row specifically to catch this case up front and skip cleanly with skipped: claude CLI is not authenticated: run 'claude login', instead of failing 100 times (once per holdout intent) over the full 23.7k-token prompt.

Reproduce (real calls; requires codex on PATH, ollama serve running with mistral:latest pulled, and claude on PATH and logged in via claude login):

venv/bin/python scripts/benchmark.py --split holdout \
    --verifiers aegis,codex,ollama,claude-cli \
    --codex-cache results/codex-cache.jsonl --ollama-cache results/ollama-cache.jsonl \
    --claude-cli-cache results/claude-cli-cache.jsonl \
    --out results/

Replay from the cache recorded by the run above, fully offline:

venv/bin/python scripts/benchmark.py --split holdout \
    --verifiers aegis,codex-replay,ollama-replay,claude-cli-replay \
    --codex-cache results/codex-cache.jsonl --ollama-cache results/ollama-cache.jsonl \
    --claude-cli-cache results/claude-cli-cache.jsonl \
    --out results/

All three rows are skipped cleanly (skipped: codex not on PATH / skipped: ollama server not reachable / skipped: claude not on PATH or skipped: claude CLI is not authenticated: run 'claude login'), exactly like the opa row, when the binary/server isn't available or not authenticated; the -replay variants are skipped with a note when their cache file doesn't exist yet.

The opa binary is installed separately — it isn't a Python dependency.

Corpus

scripts/build_corpus.py expands hand-derived seeds (drawn from the Kubernetes failure-stories index and the OPA Gatekeeper policy library, not invented from scratch) into 500 labeled constraints with real, computed provenance hashes: Trusted (~50%, valid provenance and authorized principal), Untrusted (~25%, valid provenance but unauthorized principal), and Malicious (~25%, split between tampered and forged). Re-running with the same --seed reproduces every file byte-for-byte.

It also emits 600 intents (300 aimed at Trusted rules, 150 aimed at poisoned rules that no Trusted rule shadows, 150 that match nothing), labelled by scripts/reference_oracle.py — never by the interceptor — and data/corpus/stats.json, which records the diversity numbers the benchmark header reports alongside n = 500: n_distinct_structural = 196, n_distinct_patterns = 71, n_distinct_rule_text = 66. split.json carries two 20% holdouts: a constraint split (per label, for store-level experiments) and the intent split (stratified by expected verdict × poison candidate) that scripts/benchmark.py scores by default. To relabel intents after editing labels or constraints by hand: venv/bin/python scripts/reference_oracle.py --corpus data/corpus.

Project status / roadmap

The engine (constraint store, interceptor, environment mapping, dry-run handling, rate limits, plan-level constraints, and parsers for every tool in "Supported tools") is complete. See PLAN.md for the full roadmap, backlog, and §8 for an honest list of open gaps (the real-LLM baseline hasn't been run, there are no real Git/Slack/Jira connectors, the signing key is a shared secret rather than a public-key signature, and a few others).

Why not OPA/Gatekeeper?

OPA/Gatekeeper evaluates structured API objects against hand-authored rules. Aegis derives unstructured human constraints (from Slack, Jira, Git) and applies authority-driven validation to the agent's intent before it reaches the infrastructure.

Development

venv/bin/python -m pytest -q      # 1198 passed, 1 xfailed (last full green run; re-run to confirm)
venv/bin/python -m ruff check src tests scripts examples
vhs docs/demo.tape                # regenerate docs/demo.gif

Adding a new parser or tool: see docs/CONTRIBUTING.md.

License

Apache-2.0 — see LICENSE.