anvil

README

Explore the docs
Report Bug | Request Feature

Introduction

Anvil is a declarative provider-aware execution engine for running Python tasks across cloud and service target fleets. Describe the work in YAML, keep task logic in plain Python modules, and let the engine handle authentication, target resolution, dependency ordering, bounded concurrency, and structured results. The current runtime supports AWS, Azure, Cloudflare, Datadog, GCP, GitHub, GitLab, and PagerDuty through the same provider-neutral contracts.

For complete Anvil documentation, see the Anvil documentation hub.

Key references:

• Configuration
• Providers and provider profiles
• Built-in tasks and processors
• Task contract and task workflows
• CLI and results
• Extension best practices
• Complete examples

Why Anvil?

Anvil is built for teams that need repeatable cloud workflows, such as inventory, validation, enforcement, cleanup, and reporting, to run consistently across provider targets and regions.

• Declarative orchestration
  • Define execution in reusable YAML instead of one-off scripts.
  • Configure provider targets, regions, tasks, task dependencies, dry runs, fail-fast behavior, and concurrency in one place.
• Multi-target by default
  • AWS can discover active organization accounts and enabled regions, with include/exclude filtering.
  • Azure subscriptions and GCP projects can run from explicit IDs or provider discovery.
  • Cloudflare preserves account and zone boundaries, GitLab preserves group and project boundaries, and Datadog and PagerDuty execute at organization or account scope without forcing cloud-specific hierarchy concepts.
• Parallel execution and caching
  • Control concurrency at the target, account, and region levels. See Caching and reuse.
• Shared discovery and session reuse
  • Validate targets, discover supported provider metadata, and reuse session/runtime state before execution.
• Task isolation
  • Write tasks as simple Python files with a run(...) function.
• Built-in tasks
  • Use provider-package tasks for common AWS operations and universal tasks where they apply.
  • Provider-owned task package entry points can add universal tasks or tasks for any discovered provider.
• Structured output and safer operations
  • Record structured results at task, account/target, target group, and engine levels.

Usage

[!TIP] It is recommended to use the foundry-anvil-template.

The template exposes project-local processors without forking Anvil.

If you do not need/want the full Anvil framework and only want a simple starting point for small AWS Organization tasks, see: templates/aws_multi_account_template.py

1. Install Anvil with the provider SDKs you need:
   1. Installed package users can choose provider extras with pip. Base installs include AWS support and the default CLI behavior: uv pip install anvil
   2. All other providers require users to install via extras: uv pip install "anvil[xxxx]", so like uv pip install "anvil[azure]"
   3. Install every optional provider dependency with: uv pip install "anvil[all]"
   4. Source checkout users should sync the matching uv extra instead: uv sync --extra <provider>, or use uv sync --extra all for every provider
2. When using the uv tool, there are several ways to run and install dependencies. Here are only a couple examples:
3. uv sync:
   1. Sync the project's dependencies with the environment: uv sync
   2. Activate the virtual environment: .venv\Scripts\activate
4. uv run:
   1. Run a command in the project environment.: uv run example.py <args>
      1. uv run anvil run --config-file ./yaml/orgs.yaml
   2. Note that if you use uv run in a project, i.e. a directory with a pyproject.toml, it will install the current project before running the script.

There are multiple global commands:

anvil --version  # Print the installed Anvil version
anvil list      # List available tasks, processors, and providers
anvil validate  # Inspect environment health or run focused validation checks
anvil run       # Execute YAML-defined workflows
anvil results   # Query JSONL results and rerun failures

Container image

The official batteries-included OCI image is published publicly to GitHub Container Registry with every first-party provider dependency installed. It uses the existing Anvil CLI directly and does not include cloud CLIs, Terraform, kubectl, or other unrelated operator tooling.

Run a specific release locally with Docker or Podman:

docker run --rm ghcr.io/jschronicles/anvil:<version> --version

Mount the working directory to make configuration available and retain results:

docker run --rm \
  --volume "$PWD:/workspace" \
  ghcr.io/jschronicles/anvil:<version> \
  run --config-file /workspace/anvil.yaml

Anvil writes run output beneath /workspace/results. The mounted directory must be writable by the image's non-root user (UID and GID 10001). Supply cloud and service credentials at runtime through environment variables, workload identity, managed identity, or read-only credential mounts; never add credentials to an image. Provider profiles can be mounted at /home/anvil/.anvil/config.toml or selected with ANVIL_CONFIG.

Stable releases publish exact and minor tags plus latest. Starting with Anvil 1.0, releases also publish a major tag. Prereleases publish only their exact tag. For reproducible deployments, pin the image digest reported by GHCR. The publication workflow verifies anonymous access; repository administrators must set the GHCR package visibility to public if it does not inherit the public repository visibility on its first publication.

Example AWS task configuration:

This executes the configured targets and tasks, then writes structured results under ./results.

anvil run --config-file ./yaml/orgs.yaml

# orgs.yaml example
schema_version: 2

targets:
  - name: smoke
    provider:
      name: aws
      mode: organization
      options:
        profile: root
    tasks:
      - name: noop

Example Azure task configuration:

schema_version: 2

targets:
  - name: azure-subscriptions
    provider:
      name: azure
      mode: subscriptions
      options: {}
    include:
      - 00000000-0000-0000-0000-000000000000
    regions:
      - eastus
    tasks:
      - name: count_resource_groups

See more provider configurations for simple, multi-target, include/exclude, and advanced YAML files for every provider.

Provider profiles

Anvil provider profiles are optional. Cloudflare, Datadog, GitHub, GitLab, and PagerDuty can load reusable settings from ~/.anvil/config.toml; set ANVIL_CONFIG to use a different file. AWS, Azure, and GCP continue to use their provider-native credential configuration instead.

When neither a named profile nor inline profile fields are configured, Anvil first applies providers.<provider>.default when that table exists. If it does not exist, the provider uses its normal environment variables, SDK credential chain, workload identity, or other native fallback where supported. Profiles are namespaced by provider and profile name:

[providers.cloudflare.security]
api_token_env = "CLOUDFLARE_SECURITY_TOKEN"

[providers.github.work]
token_env = "GITHUB_WORK_TOKEN"
api_url = "https://api.github.com"

[providers.gitlab.default]
token_env = "GITLAB_TOKEN"
url = "https://gitlab.example.com"

These fields identify environment variables or provider-native credential locations; they do not contain the credentials themselves. Set the referenced variables through your shell, CI secret store, or runtime environment.

Select a named profile with provider.options.profile. A profile named default is selected automatically when the target does not provide inline authentication or connection options. The following target fragments omit unrelated fields with # ....

Cloudflare uses the named security profile while keeping its resource selector inline:

provider:
  name: cloudflare
  mode: zones
  options:
    profile: security
    account_id: '11111111111111111111111111111111'
# ...

GitHub references the named work profile:

provider:
  name: github
  mode: repositories
  options:
    profile: work
# ...

GitLab uses providers.gitlab.default because no profile or inline connection fields are configured:

provider:
  name: gitlab
  mode: projects
  options: {}
# ...

A named profile cannot be combined with inline profile fields such as token_env, api_url, or url. Resource selectors that do not belong to the profile remain inline, as shown by Cloudflare's account_id. Each provider continues to validate its own supported profile and target options.

See Provider profiles for default-profile behavior, supported fields, and multi-account or multi-endpoint patterns.

Provider task packages

[!NOTE] Duplicate task names across all packages and plugins applicable to the selected provider are rejected as ambiguous.

Task compatibility is determined by package location.

See the built-in component catalog for the tasks shipped by each provider. For invocation IDs, task-to-task result sharing, always_run, partial recovery results, and scope-aware dependencies, see Task workflows.

• anvil.providers.tasks.<task> is universal and can run for any provider.
• anvil.providers.<provider>.tasks.<task> runs only for the provider named by that package segment.

Use target dry_run: true to review planned removals before execution.

Extension package discovery

Tasks, processors, and providers are discovered from package folders. Adding a public module or provider folder to an already registered package does not require another entry-point declaration. Discovery records names and sources without importing child implementations; normal execution imports only the selected components. Duplicate names are rejected as ambiguous and report every conflicting source.

See Extension best practices for package layouts, current entry-point groups, provider factories, and implementation guidance. See the Task contract for task compatibility, lazy discovery, and ambiguity behavior.

The Provider reference documents all stock provider modes, authentication options, target selectors, locations, and validation behavior and includes a focused configuration example for every provider.

See Selectors and regions for exact include, exclude, all, glob, management, and payer rules.

For delegated-administrator patterns, keep the base session on the delegated-admin profile. Anvil uses that base session directly for the delegated-admin account if it appears in Organizations discovery, and assumes role_name in every other selected account, including the management/payer account. AWS organization targets accept management and payer as case-insensitive aliases for that account in include and exclude filters.

schema_version: 2

targets:
  - name: security
    provider:
      name: aws
      mode: organization
      options:
        profile: delegated-admin-security
        role_name: SecurityAuditRole
    regions:
      - us-east-1
    include:
      - management
    tasks:
      - name: noop

Results

anvil results queries completed run output without rerunning cloud work. Use it to filter historical JSONL results by target, account, region/location, task, or status, emit JSON/JSONL for automation, rerun failed work, or run a processor against a completed results directory. When a run has failures, Anvil prints ready-to-use anvil results commands that point at the affected run's results.jsonl file so you can inspect or rerun the failed execution targets.

See more at Common result queries and Rerun failures.

Validation

Use anvil validate before a run to inspect the local environment or perform one or more focused checks without running tasks:

anvil validate

With no switches, anvil validate prints offline diagnostics for the current Anvil environment, including Python and Anvil versions, optional provider dependency availability, provider/task/processor discovery, local auth source hints, and result path state. It does not call cloud APIs, validate live access, or run tasks.

Validate a YAML config file offline:

anvil validate --config-file ./yaml/orgs.yaml

This parses the config, validates schema and target shape, and checks CLI override semantics without checking credentials or calling provider APIs.

Run focused validation categories:

anvil validate --tasks --processors --auth --config-file ./yaml/orgs.yaml

--tasks and --processors validate discovery, keyword-only callable signatures, and operator-facing detail documentation. Validation rejects additional required parameters that Anvil cannot supply at runtime. --providers validates the provider contract. --auth validates cloud access for the configured targets after loading and validating the config file.

See more at Task validation.

Processors

Processors run after a target finishes and turn Anvil results into reports or integration artifacts. Use them for formats that should stay outside task logic, such as HTML, SARIF, Markdown, JSON summaries, tickets, or notification payloads. Processor modules expose a documented keyword-only run(*, context, output, metadata) callable. context.target_results is the canonical result collection; target-level runs additionally set context.target_name, from which target_result and target_result_path are derived. Treat context data and processor metadata as invocation snapshots. Target post_run processor output is written under the run's reports directory, so output: smoke.html becomes <run_dir>/reports/smoke.html.

Use html_report when you want a self-contained, human-readable report for a completed target:

schema_version: 2

targets:
  - name: smoke
    provider:
      name: aws
      mode: organization
      options:
        profile: root
    regions:
      - us-east-1
    tasks:
      - name: noop
    post_run:
      - processor: html_report
        output: smoke.html
        run_on_failure: true

Use sarif_report when detect_ tasks return sarif_findings and you want a SARIF 2.1.0 report for code-scanning or security tooling:

schema_version: 2

targets:
  - name: lambda-runtime-audit
    provider:
      name: aws
      mode: organization
      options:
        profile: root
    regions:
      - us-*
    tasks:
      - name: detect_deprecated_lambda_runtimes
    metadata:
      runtimes:
        - python3.8
        - nodejs16.x
    post_run:
      - processor: sarif_report
        output: lambda-runtimes.sarif
        run_on_failure: true

See more at HTML result reports, including examples for separating target-level reports or combining a completed run into one HTML report.

To build a custom processor, see Extension best practices.

Example Benchmarks

To measure concurrency behavior, the engine was tested across 3 organizations with a combined 260 accounts using the count_vpc task. The comparison below shows the same kind of work moving from sequential execution to organization-level parallelism and then to account-level parallelism.

The fastest measured run in this benchmark completed 260 accounts in about 1m 35s for 1 region, compared with a 3h 15m manual sequential estimate at 45 seconds per account. With 2 regions, the parallel account run completed in about 2m 48s.