fastly-sync 0.3.0

Synchronise Fastly CDN, rate limiter and WAF IP blocklist configuration on demand.

fastly-sync

fastly-sync synchronises Fastly configuration on demand from declarative sources:

• CDN cache and rate limiters are derived from an OpenAPI (openapi.json) document (local file or remote http(s) URL). Each path becomes a CDN cache setting whose policy is derived from its methods (GET/HEAD-only paths are cached, paths with mutating methods pass) and can be tuned with an x-fastly-cache extension; paths carrying an x-fastly-ratelimit extension become Fastly rate limiters.
• WAF IP blacklisting is driven by a text blocklist (local or remote), reconciled onto a Fastly Edge ACL.

Documentation

The project site is published from main to GitHub Pages: https://goabonga.github.io/fastly-sync/.

Requirements

• Python 3.12+
• uv for environment management
• A Fastly API token and the target service id

Install

pip install fastly-sync      # or: uvx fastly-sync --help

Usage

export FASTLY_API_TOKEN=...        # or pass --token
export FASTLY_SERVICE_ID=...       # or pass --service-id

# Apply EVERYTHING: CDN cache + rate limiters (one clone/activate) AND the WAF
# blocklist. Prints a plan, asks for confirmation, then applies.
fastly-sync sync all --openapi ./openapi.json --blocklist ./blocklist.txt
fastly-sync sync all --openapi ./openapi.json --blocklist ./blocklist.txt --dry-run
fastly-sync sync all --openapi ./openapi.json --blocklist ./blocklist.txt --no-confirm

# Apply a single target:
fastly-sync sync cdn          --openapi ./openapi.json
fastly-sync sync rate-limiter --openapi ./openapi.json
fastly-sync sync waf          --blocklist ./blocklist.txt

# WAF IP blacklisting: reconcile an Edge ACL from a text blocklist.
fastly-sync sync waf --blocklist ./blocklist.txt --bootstrap   # one-time ACL + VCL
fastly-sync sync waf --blocklist https://feeds.example.com/bad-ips.txt
fastly-sync sync waf --blocklist ./blocklist.txt --dry-run

# Show the live config applied on Fastly, per target (or `all`).
# Every show target accepts --output FILE and --format text|terraform|csv.
fastly-sync show all
fastly-sync show cdn --output cdn.txt           # same text, to a file
fastly-sync show waf --output blocklist.txt     # export the blocklist (round-trips)
fastly-sync show all --format csv               # CSV table
fastly-sync show all --format terraform -o fastly.tf   # Fastly Terraform resources

# Generate Terraform/CSV from the OpenAPI spec itself, WITHOUT touching Fastly
# (offline, no credentials needed — sync renders the desired config):
fastly-sync sync cdn --openapi ./openapi.json --format terraform -o cdn.tf
fastly-sync sync all --openapi ./openapi.json --blocklist ./blocklist.txt --format csv

# Shell completion (Typer):
fastly-sync --install-completion

WAF IP blacklisting

fastly-sync sync waf reconciles a Fastly Edge ACL from a text blocklist (one IP or CIDR per line, # for comments), local or remote. Entries are validated with Python's ipaddress, so IPv4, IPv6 and subnets all work:

# blocklist.txt
203.0.113.0/24      # botnet C2
198.51.100.7
2001:db8::/32
!10.0.0.0/8         # negated: an allowlist entry (do NOT match)

A leading ! marks a negated (allowlist) entry — it round-trips through show waf --output.

The reconciliation is a diff: missing IPs are added and stale ones removed (--dry-run reports without applying). Because ACL entries are mutable outside service versioning, ongoing syncs are fast and need no version bump.

Run once with --bootstrap to create the ACL and an enforcing vcl_recv snippet (if (client.ip ~ waf_blocklist) { error 403 "Forbidden"; }) on a new activated version; subsequent runs only touch the ACL entries. Use --acl-name to target a differently named ACL (default waf_blocklist).

fastly-sync show waf prints the live ACL entries; with --output FILE it writes them in the blocklist text format, so you can snapshot, diff or version the current blocklist. The output round-trips back through sync waf.

OpenAPI extensions (and custom keys)

CDN and rate limiter behaviour is read from per-path OpenAPI extensions: x-fastly-cache and x-fastly-ratelimit by default. Use custom keys in your document by passing --cache-key / --ratelimit-key to the sync commands, e.g. sync cdn --openapi spec.json --cache-key x-cdn.

OpenAPI CDN cache extension

By default the cache policy is derived from a path's methods: a GET/HEAD-only path is cached (TTL 3600s), any path exposing a mutating method (POST, PUT, …) is set to pass. Override per path with the x-fastly-cache extension (or your --cache-key):

{
  "paths": {
    "/widgets": {
      "get": {},
      "x-fastly-cache": {
        "action": "cache",
        "ttl": 3600,
        "stale_while_revalidate": 60,
        "stale_if_error": 300,
        "description": "Public widgets API"
      }
    }
  }
}

action is cache or pass (defaults to the method-derived value); ttl defaults to 3600s when caching; stale_if_error maps to Fastly's serve-stale window (stale_ttl). description is an optional note — the Fastly cache_settings object has no comment field, so it is stored on the endpoint's condition comment and shown by fastly-sync show cdn. (Rate limiters have no comment field in the Fastly API, so they cannot carry a description; ACL entries do — use the blocklist's # comment.)

Each cache setting is scoped to its path by a Fastly request condition built as a strict, anchored regex: each {param} becomes a single non-slash segment and both ends are anchored, e.g. /widgets/{id} → req.url ~ "^/widgets/[^/]+(?:\?|$)". Sibling paths such as /widget and /widgets therefore never overlap. When stale_while_revalidate (or stale_if_error) is set on a cached path, a Surrogate-Control response header carrying those directives is emitted and scoped to the same condition.

OpenAPI rate limiter extension

A path opts into a rate limiter with the x-fastly-ratelimit extension:

{
  "paths": {
    "/login": {
      "post": {},
      "x-fastly-ratelimit": {
        "name": "login",
        "limit": 100,
        "window": 60,
        "http_methods": ["POST"],
        "action": "response",
        "penalty_box_duration": 1,
        "client_key": "req.http.Fastly-Client-IP",
        "logger_type": "",
        "response_object_name": "",
        "uri_dictionary_name": "",
        "feature_revision": 1
      }
    }
  }
}

limit is required (requests per window, mapped to rps_limit / window_size); window defaults to 60s and name to a slug of the path. http_methods defaults to the path's HTTP methods (a list or a comma string); the remaining fields mirror the writable Fastly rate limiter API / Terraform provider arguments and use sensible defaults.

Reconciliation, pruning and confirmation

sync is declarative: the OpenAPI document is the source of truth. On each run it not only creates/updates the managed objects but also prunes the ones it previously created that are no longer in the spec (a removed path's cache setting / condition / serve-stale header, a removed rate limiter). Pass --no-prune to keep additive behaviour.

Pruning is scoped to objects fastly-sync owns — managed rate limiters are named with an fsync- prefix and cache settings are matched through their cache-… condition — so configuration created by hand is never deleted. A targeted run (sync cdn / sync rate-limiter) only ever touches that target.

Before applying, every sync target prints a plan and asks for confirmation. Use --dry-run to print the plan and stop, or --no-confirm to apply without prompting (CI). show <target> prints the live CDN, rate limiter and WAF configuration currently applied on Fastly.

Output formats (text / terraform / csv)

Both show and sync accept --format text|terraform|csv (with --output FILE / -o):

• show --format … renders the live config read from Fastly.
• sync --format … renders the desired config derived from the OpenAPI spec (and --blocklist) — offline, no credentials, nothing applied. Handy to generate IaC or a CSV from your spec in CI.

Both formats carry the full writable field set of each Fastly resource (rate limiter http_methods/action/penalty_box_duration/client_key/…, ACL negated, …).

terraform emits Fastly provider resources: a fastly_service_vcl with the cache_setting, condition, header (serve-stale) and rate_limiter and acl blocks, plus a fastly_service_acl_entries resource for the WAF IPs. It is a scaffold, not a turn-key .tf: fastly_service_vcl also requires name, domain and backend (which fastly-sync does not manage), so those are TODO placeholders; the fastly_service_acl_entries resource is complete.

csv emits one table per target (show cdn/rate-limiter/waf), or a generic kind,name,detail table for all.

Getting started (development)

git clone https://github.com/goabonga/fastly-sync.git
cd fastly-sync
uv sync
uv run pre-commit install
uv run ruff check src tests
uv run pytest

Versioning and release

Versions are bumped from Conventional Commits by multicz. On every push to main, CI computes the bump, writes the changelog, tags, and publishes to PyPI. Maintainers do not bump versions or edit the changelog by hand.

Stability and deprecation policy

fastly-sync follows Semantic Versioning and the standard Python n + 2 deprecation cadence (announce + warn in one release, remove in the release after the next). Full policy: docs/stability.md.

Contributing

See CONTRIBUTING.md for the workflow, the commit-message convention, and the test/lint expectations. By participating you agree to the Code of Conduct.

Security issues: please follow the disclosure process in SECURITY.md.

License

Distributed under the MIT License.