ansible-heal-agent

An agent that watches an Ansible pipeline, diagnoses the failures it produces, patches the offending inventory / playbooks / vars, commits the fix, and re-runs — for a small set of well-understood failure classes.

Real, unedited output from make demo, captured with asciinema via a scripted driver: three seeded failures, three conventional commits, pipeline green. The SHAs in the git log are the ones that run produced.

What it does, precisely

Four failure classes, against real ansible-playbook or against a bundled simulator:

class	detected via	fix	verified against real Ansible
host pattern matches nothing	callback plugin — real Ansible logs a warning and exits 0	rename the closest inventory entry	heals to green
undefined variable	callback plugin (name extracted)	define it in group_vars with an inferred default	heals to green
unresolvable / removed module	text scan — parse errors abort before callbacks fire (exit 4)	swap the module for its modern equivalent	rewrite verified; see below
module from an uninstalled collection	same text scan; ansible-galaxy collection list confirms the collection is absent	declare it in collections/requirements.yml — never install	declaration verified; install is the operator's

The first two are asserted end to end in tests/test_real_ansible.py: real ansible-playbook fails, the agent patches, and the binary is run again and exits 0.

The module class is honest about where it stops. The agent rewrites the module name and the run no longer fails on the old one — but the replacement for docker / docker_container lives in the community.docker collection, so the play only reaches green once that collection is installed. So the agent takes the one step that is its to take: it asks ansible-galaxy whether community.docker is installed and, when it is not, declares it in the repo's collections requirements file (an existing one if the repo has one, otherwise the first candidate inside the write surface), as its own fix(deps) commit. That is what makes the dependency reviewable, and it is what AWX / Automation Controller install from on their own. Installing it on this control node is the operator's call, not the agent's: the next iteration reports declared in <file> but not installed — run ansible-galaxy collection install -r <file> and stops. It declines, rather than guesses, when the collection is installed but lacks the module (a typo or version mismatch), when ansible-galaxy cannot answer, when the existing file is a legacy roles list, or when the name is not a valid collection name. Argument carry-over is per-module: the docker mappings pass the task's arguments through unchanged, while apt_key → get_url deliberately rewrites them, because a keyring fetch does not take the same arguments as a key import.

Anything else is reported, not guessed at. See Where it declines.

Quick start

make install    # deps, plus the `ansible-heal` CLI on your PATH
make demo

make demo seeds a broken baseline into a disposable .demo-workspace/, heals it, and prints the commits it landed. It does not write to this checkout. On a machine without the community.docker collection installed, the run ends honestly red over the module class — the agent declares the collection and stops (see What it does, precisely); install it with ansible-galaxy collection install community.docker and re-run to see the pipeline reach green.

Against your own repository:

# See what it would do. Writes nothing into ~/infra and commits nothing;
# the run log and transcript go to a scratch dir outside it, path printed.
ansible-heal run --repo ~/infra --dry-run

# Open a PR and stop. Your base branch is not touched.
ansible-heal run --repo ~/infra --require-human-approval

# Apply directly (what the demo does).
ansible-heal run --repo ~/infra

The run defaults to the playbook at ansible/playbooks/site.yml; pass --playbook when your entrypoint lives elsewhere (ansible-heal run --help lists every knob).

Safety model

These are implemented and tested, not planned.

ansible-core is the authority, and it is required. apply and --require-human-approval refuse to run without ansible, ansible-inventory and ansible-doc on PATH. Every check that stops the agent damaging a working repository — does this host exist, does this module resolve — is answered by ansible-core, and without it they all return "unknown". It is a hard runtime dependency for that reason, not a test one.

The host set comes from ansible-inventory --list, so it covers every source the repo configures: a second inventory file, a plugin config, a dynamic script. Nine rounds of audit taught this the expensive way — the agent used to decide which files were inventories and which sources it could read by inspecting filenames, suffixes and exec bits, and every such predicate was defeated by an ordinary spelling it had not enumerated. Enumerating spellings does not converge. Asking ansible-core does.

When ansible-core itself refuses the repository's ansible.cfg — it ignores one kept in a world-writable directory, which is the state of every checkout under /tmp in a CI sandbox — the probe hands that cfg's own static sources to the binary explicitly rather than decline on a config it just watched being ignored; a source that would execute repository code is never passed.

In apply and PR modes the repository's own inventory plugins run, because the operator has pointed the agent at their repository and asked it to commit there — that is no more than ansible-playbook would do. In --dry-run, the mode for a repository you have not decided to trust, they stay disabled, and a dynamic source simply makes the answer unavailable — which refuses the rename rather than authorising it.

The same principle covers the rest: a module is migrated only if ansible-doc says ansible-core cannot resolve it (apt_key still resolves, so rewriting a playbook that uses it is a modernisation the operator chooses, not a repair); a variable is added only when it is defined nowhere Ansible would look, including the inventory's own vars: and any vars_files:; and a file with uncommitted changes is not committed at all, because git commit -- <path> takes the working tree and would fold the operator's work into the agent's commit.

The destructive step asks ansible-core, not the simulator. Before any inventory rename — from the deterministic diagnoser or from the LLM — the agent runs ansible <pattern> --list-hosts and lets ansible-core resolve its own inventory, exactly as it would for a real run: the repo's ansible.cfg, a comma-separated inventory list, ANSIBLE_CONFIG, ANSIBLE_INVENTORY. If ansible-core resolves the pattern, no rename is proposed, whatever the simulator concluded.

Two things this gets wrong easily, both of which it got wrong before:

• Handing ansible the agent's own guess with -i overrides the very thing being asked about. On a repo listing two inventory files the probe saw one of them, answered "no such host" truthfully, and a live host in the other was renamed away.
• A probe that cannot answer returns "unknown", never "no". Collapsing those would let any environment problem re-open every hole the gate exists to close.

The probe runs with only ansible's builtin file parsers enabled and no plugin path from the target repo, because --dry-run is the mode you point at a repository you have not decided to trust, and a third-party inventory plugin is executable code that repository supplies.

This is the single most important line of defence in the project, and it exists because of a shape that repeated across six audit rounds: the simulator failed to match some ordinary Ansible construct — IPv6 literals, ? globs, whitespace separators, nested groups, exclusion, intersection — and the diagnoser read "no match" as "the inventory is wrong" and renamed a live host. Each of those was fixed individually and each fix left a neighbouring case broken. Patching a reimplementation of Ansible's pattern language toward correctness does not converge; asking the real thing does. Where ansible-core is not installed the guards below still apply, but the pipeline result is a simulation and the README says so.

What the simulator is not. pipeline/runner.py is a simulator, and the gap between it and ansible-core is where nearly every destructive bug in this repo has come from. It now implements union, ! exclusion, & intersection, ~regex, */? globs, groups nested or flat-by-children, IPv6 literals, and comma/semicolon/whitespace separation, and it refuses to evaluate host ranges (web-0[1:3]) rather than guessing. Where it still cannot decide, it says so and the agent declines. PIPELINE_RUNNER=real hands the whole question to ansible-playbook.

One deliberate non-feature: a group mapping placed directly under all: is not treated as a group, because ansible-core skips it ("Skipping unexpected key (webservers) in group (all)") and resolves it to zero hosts. An earlier round added support for that shape and a test asserting it; both were wrong, and a false green is worse than a false failure.

Only a checked host rename may touch an inventory. An edit_file fix targeting one — the shape the LLM prompt actually asks for — is refused outright. A free-text search-and-replace across an inventory is not something this agent can validate, and a replace span once took out a host that no failure had ever mentioned.

One writer per repository. Git's index is a single shared file. Two concurrent runs interleaved add and commit badly enough that one run's staged edit landed inside the other's commit, so apply and PR modes take an exclusive lock on the target repo for the duration.

Writes are atomic. A patch goes to a temp file in the same directory and is renamed over the target. write_text truncates first, so a write that failed part-way — a full disk, a quota — left a file cut off mid-key that still parsed as YAML, which meant nothing downstream noticed.

Write allowlist. The patcher refuses any target outside ANSIBLE_HEAL_ALLOWED_PATHS (default ansible/**), checked before the file is touched at all. Since the target path can come from an LLM, this is what stops a diagnosis proposing agent/core.py from being applied.

The check runs twice: once on the path the diagnosis asked for, and again on the path it resolves to. Only checking the declared name made the allowlist bypassable by a symlink — ansible/group_vars/all.yml pointing at shared/secrets.yml is inside the write surface by name and outside it in fact, and the write would not even show up in git, because the committer stages the declared path whose own bytes never changed.

$ ansible-heal run --repo ~/infra --allowed-paths 'infra/**' --dry-run
repo:  /home/you/infra
mode:  dry-run
write surface: ['infra/**']
artefacts:     /tmp/ansible-heal-dryrun-z4bsk0d0  (nothing is written to the repo)

3 proposal(s); nothing written.

BLOCKED: refusing to write ansible/playbooks/webservers.yml: outside the allowed
write surface ['infra/**']. Set ANSIBLE_HEAL_ALLOWED_PATHS to widen it.

BLOCKED: refusing to write ansible/group_vars/all.yml: outside the allowed write
surface ['infra/**']. Set ANSIBLE_HEAL_ALLOWED_PATHS to widen it.

BLOCKED: refusing to write ansible/inventory.yml: outside the allowed write
surface ['infra/**']. Set ANSIBLE_HEAL_ALLOWED_PATHS to widen it.

transcript: /tmp/ansible-heal-dryrun-z4bsk0d0/transcripts/run-20260822-2103.md

Success: False  Iterations: 0  Final exit: 2

One line per refused write — three proposals, three refusals. Line wrapping above is this document's; the real output does not wrap.

Three modes.

mode	writes	commits	re-runs pipeline	touches base branch
--dry-run	no — not even a run log; artefacts go to a scratch dir outside the repo	no	no	no
--require-human-approval	yes	heal/<run-id>	no	no
default	yes	current branch	yes	yes

Approval mode commits to a new branch, pushes, opens a PR via gh if present, checks your branch back out, and stops. The pipeline is deliberately not re-run: the point is to hand a human a reviewable change, not to self-certify.

Every YAML patch is parsed before it is written. If the result is not parseable YAML, nothing is written and the agent falls back or reports. The check is keyed on the file suffix, so it covers every file the agent currently knows how to edit — inventory, playbooks, group_vars — and would not cover a non-YAML target such as a .j2 template if a future fix class introduced one.

Every CLI run leaves a transcript (unless --no-transcript) — failures, diagnoses, diffs, commit SHAs, final status. Example.

Where it declines

A diagnoser that always produces a patch is indistinguishable from one that produces a wrong patch. These cases report and stop — on the console, not only in the transcript:

$ ansible-heal run --repo ~/infra
NO FIX: no inventory host resembles 'web-server-01' (inventory has
['db-primary']); renaming an unrelated host would be a guess, so no fix is
proposed

A hosts: pattern that is not one stale hostname. This is where most of the damage lived, and every case below was a real repository the agent broke while reporting success:

• a group name — an empty group (a tier scaled to zero, a dynamic-inventory placeholder) makes Ansible skip the play and exit 0. Renaming the nearest host to match deletes a live host and creates a group/host collision
• localhost, which Ansible always provides implicitly. Renaming a real host to it redirects every later localhost play at that machine
• a multi-host pattern — web-01 db-01, web-01,db-01, a YAML list. These resolve properly now; when one cannot, it is not a hostname to rename to
• all or * matching nothing, which means the inventory is empty
• any pattern containing ! & [ ] * ? ~ — or a : that is not part of an IPv6 address. Those are pattern syntax, not names. The check runs on the pattern as the operator wrote it, not on a fragment a runner split out of it: fd00::21 reached the guard as the token 21, looked like an ordinary hostname, and a real address was renamed to fd00

The repository's own state.

• ansible-vault encrypted files are never read or rewritten. The ciphertext is a valid YAML scalar, so a validity check alone passes it and a string-replace edit writes plaintext over the operator's secrets
• a hardlinked target, whose other names the write surface cannot bound
• a merge, rebase, cherry-pick, revert or bisect in progress — the agent will not commit into a half-finished operation and conclude it for you
• a detached HEAD, where commits would be unreachable from any branch
• a commit git itself refuses: a failing hook, a submodule, an ignored path
• a file missing, unreadable, not a regular file, or not the YAML shape its caller needs

The ordinary cases.

• no inventory host close enough to the one the play targets
• an inventory host another play still targets, where renaming it would break that play
• a module with no known replacement in MODULE_REPLACEMENTS
• a variable already defined in group_vars/all*, on the play, or on the task
• a variable defined only in group_vars/<group> or host_vars/ — it names the file and stops. Adding a global default would override the operator's per-host value for every other host, and merging every vars file into one namespace to call the run green hides a play that genuinely fails on a host the variable was never defined for. Both answers are wrong; saying which file defines it is not
• any failure class it has no rule for

A refusal is a result. heal() returns them on HealResult.declined, and the CLI prints each one.

A run that patches nothing stops immediately rather than burning its retry budget re-running a pipeline that cannot change.

Configuration

variable	default	meaning
ANSIBLE_HEAL_REPO_ROOT	this checkout	repository to operate on (--repo)
ANSIBLE_HEAL_ALLOWED_PATHS	ansible/**	comma-separated globs the patcher may write
PIPELINE_RUNNER	mock	real runs ansible-playbook; errors if it is not installed
ANSIBLE_HEAL_LLM_PROVIDER	auto	anthropic | openrouter | z-ai
ANSIBLE_HEAL_LLM_MODEL	per provider	model override

The LLM part

agent/diagnoser.py asks a model for a structured fix and validates it against the same gates as everything else. The request uses each provider's structured-output mode — on Anthropic a single forced tool whose input_schema is the diagnosis contract (DIAGNOSIS_SCHEMA), with the system prompt marked for prompt caching; on OpenRouter a strict json_schema response format — and the reply is re-validated locally against the same schema whichever provider answered, so an off-contract answer is rejected before it reaches a gate. Client errors (400/401/403) fail immediately; only 408/429/5xx are retried. As before, if the model is unavailable, returns malformed JSON, or proposes a patch that does not apply, the agent falls back to the deterministic rules and says so in the transcript.

Providers are auto-detected: ANTHROPIC_API_KEY, then OPENROUTER_API_KEY, then a z-ai CLI on PATH. HTTP is urllib from the standard library — no provider SDK is required. API keys are redacted from any error the bridge raises, because transcripts get committed and uploaded.

With no provider configured the agent is fully deterministic, and the demo says so rather than implying a model was involved.

Tracing

Every component emits an OpenTelemetry span — ansible_heal.run, ansible_heal.pipeline.run, ansible_heal.diagnose, ansible_heal.commit, and one chat <model> span per LLM call carrying the GenAI semantic conventions (gen_ai.provider.name, gen_ai.request.model, gen_ai.usage.input_tokens / output_tokens / cache_read.input_tokens), so GenAI-aware backends show model calls, token spend and prompt-cache hits without custom mapping.

pip install 'ansible-heal-agent[otel]'
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 ansible-heal run --repo ~/infra --dry-run

It is optional: without OpenTelemetry installed the instrumentation is a no-op, and ANSIBLE_HEAL_OTEL=0 switches it off when it is. Spans carry names, counts, exit codes and paths — never prompts, completions, file contents or credentials; those stay in the transcript.

The two runners

PIPELINE_RUNNER=real shells out to ansible-playbook and collects failures from two sources, because neither sees everything: a callback plugin for runtime events, and a text scan for parse-time errors, which abort before any callback fires. The two overlap — a no-hosts warning and an undefined variable are both visible to each — so records are de-duplicated on whatever that failure class's fix is keyed on: the variable name for an undefined variable, the module for a removed one, the pattern for a host. Keying them all on (type, host, …) looked right and silently failed for variables, because the text scan's record carries no host; the operator got two identical proposals and a spurious "already defined" patch failure.

Exit codes are interpreted by what the pipeline means: a play skipped because its host pattern matched nothing exits 0 in real Ansible, and is reported here as a failure. A deployment that silently configured zero hosts is not healthy.

The default mock runner (pipeline/runner.py) parses the playbooks and inventory itself and emits Ansible-shaped logs. It exists so the demo and the bulk of the suite run in under a second with no Ansible installed. It is a simulator and is labelled as one.

Code scanning (SARIF + GitHub Action)

--dry-run --sarif out.sarif writes every finding as SARIF 2.1.0: one rule per failure class (AHA001 host pattern, AHA002 variable, AHA003 module, AHA004 missing collection, AHA900 unclassified), located at the playbook line that triggered it, with the exact diff the apply mode would commit in the alert body. A validated fix is a warning, a refusal a note, a write the surface would block an error. Fingerprints are stable, so re-scans update alerts rather than duplicate them.

The repository is also a composite action. It is read-only against the scanned repo:

permissions:
  contents: read
  security-events: write
steps:
  - uses: actions/checkout@v4
  - uses: adventurewave-labs/ansible-heal-agent@v0
    with:
      playbook: ansible/playbooks/site.yml
      runner: real            # or mock
      fail-on-findings: false

Findings land in pull-request annotations and Security → Code scanning. Inputs reach the shell through env:, never interpolated into the script.

MCP server

ansible-heal mcp serves the agent over the Model Context Protocol (stdio, JSON-RPC 2.0, stdlib only), so another agent can ask what is broken and what this one would change:

claude mcp add ansible-heal -- ansible-heal mcp

tool	returns
diagnose	dry-run findings for a repo path: failure, diagnosis, proposed diff, or the decline / block reason
explain_decline	for one failure, whether the agent would fix it and why — or why not
list_failure_classes	classes and their SARIF rule ids

It is read-only by construction: there is no apply tool, every tool is annotated readOnlyHint, runs in dry-run mode with the target repo's own inventory plugins disabled, and writes its artefacts outside the repository. Committing to someone's infrastructure stays a decision made at a terminal, not one delegated to whatever model is on the other end of the pipe.

Measuring it

$ make eval          # or: ansible-heal eval [--runner real] [--llm]
ok   healed    host/web-01->web-server-01  (1.4s)
...
ok   declined  decline/unrelated-host  (1.5s)
ok   declined  decline/empty-group  (1.5s)

heal rate: 100.0%   false-fix rate: 0.0%   decline precision: 100.0%

agent/evaluation.py generates a deterministic corpus — 16 cases the agent should heal (host renames, undefined variables, both at once) and cases it must decline (an unrelated host, an empty group, a module with no known replacement, a variable defined only in host_vars) — runs the real apply-mode loop against each in a throwaway git repo, and reports:

metric	meaning
heal rate	heal cases that ended green with a commit
false-fix rate	decline cases where the agent committed anything — the number that matters
decline precision	of the refusals it made, how many were right

A green pipeline with no commit does not count as a heal. Results go to eval-report/eval-report.{json,md}; CI runs it gated at ≥90% heal rate and zero false fixes and posts the table to the job summary. --llm scores the model path through the same gates, so the two are directly comparable.

Tests

make test          # 320 tests
make lint

• tests/test_perturbation.py — 37 tests, of which 25 vary a failure the agent must generalise across names — variable names, host names and the module — so the agent is not tuned to one scenario. They check dynamically whether the seeded module's replacement itself resolves via ansible-doc, and assert a fully green pipeline only when it does — in an environment without the community.docker collection, the host and variable classes still converge on every name, and the module swap still lands as a genuine commit, but the pipeline honestly stays red over it rather than claim a green it did not earn. The other 12 are the counterweight: 8 pin the inferred defaults and 4 assert the agent refuses rather than guessing.
• tests/test_real_ansible.py drives a real ansible-playbook process and asserts the end state by running the binary again afterwards — for the host and variable classes, that means exit 0 from ansible itself, not from our own bookkeeping.
• tests/test_safety.py asserts the allowlist and PR-mode invariants directly: the file is byte-unchanged, the base branch is at the same SHA.
• Every test that touches a repository runs against a scratch repo under tmp_path; the rest are pure-function tests over config and the LLM bridge. A guard fixture fails any test that dirties this checkout, and CI re-checks with git status --porcelain afterwards.

Roadmap

Honest list of what is not here:

• more failure classes (this handles four)
• MODULE_REPLACEMENTS covers apt_key, docker, docker_container — and note that on ansible-core 2.19 apt_key still resolves, so that mapping is a modernisation rather than a fix for a broken play
• converging the module class against real Ansible, which needs the replacement collection present
• a heal-rate figure on a corpus of real broken repositories — make eval measures a generated corpus, which is necessary but not the same thing
• an LLM-path eval number: ansible-heal eval --llm exists, but no published figure yet (it needs a provider key in CI)
• remediation for ssh_conn_refused (FR-11) and Slack notification (FR-12)

See PRD.md for requirement-by-requirement status and PLAN.md for how it was built.