greasewood 0.1.2

Minimal self-hosted WireGuard mesh overlay — direct-or-fail, IPv6-only, CA-gated

Greasewood

A minimal, self-hosted, greasy WireGuard mesh network.

Its one priority is being easy to reason about. It was built by someone who lovingly maintained a fleet of hand-written WireGuard/networkd text files far past the point of practicality, and wanted the simplest possible upgrade.

• Private. Membership is gated by a certificate authority; revoke a node by not renewing it.
• Direct-or-fail. No routing, no relays. A link comes up directly or it honestly fails.
• IPv6 only overlay. The overlay is IPv6-only; the underlay may be IPv4 or IPv6.
• Linux-only. Leans heavily on systemd, nftables.
• Greasy. Uses the stock wg/ip tools over subprocess.
• Named. Every node gets a <host>.<mesh>.internal name and matching TLS certs from the same CA.
• Policy-derived topology. Roles + an allow-only grant table control who talks to whom.
• Self-certifying addresses. A node's IPv6 address is a hash of its identity key.
• Service TLS. The same CA issues auto-renewing x509 certs for your services (Postgres, nginx, …), with profiles that place them where each wants them.
• Offline-tolerant. The anchor can be down for a credential lifetime, nodes run from cache.
• Hands-off. Never automatically configures your main firewall. Port access control for the overlay lives on a dedicated table
• Auditable. Pure Python, one dependency. Fanatical logging.
• Self-contained. The coordination anchor is just a normal functioning node. Any node can become the coordination anchor.

Prior art

The nearest full-featured projects are Tailscale, Nebula, and innernet. Next to greasewood they're all bigger systems that do more: routing, NAT traversal, multi-platform, etc. Greasewood aims to be a minimal alternative.

Membership

Membership is a CA-signed credential with an expiry

Two keys per node, deliberately split:

• id_priv / id_pub (Ed25519) — durable identity. It derives the node's overlay address and authorizes credential renewal.
• wg_priv / wg_pub (X25519) — your normal wireguard keys.

Two signed objects:

• Credential — signed by the CA. Binds id_pub, wg_pub, overlay address, hostname, capabilities, and an expiry. Slow-moving (default 24h TTL).
• NodeRecord — signed by the node's own id_priv. Carries the credential plus fast-moving facts (endpoints, a sequence number) Its hostname is read from the credential. This is what gets published to other nodes through a directory.

Revocation is expiry-based (no CRL). gw revoke <id> on the anchor takes effect there immediately — the anchor re-reads its revoke list each reconcile cycle, dropping the node from its own interface within seconds, and refuses the node's renewals from then on. It does not reach the rest of the fleet instantly: other nodes keep trusting the node's credential until it expires. But since the node can no longer renew, that credential lapses within at most one credential_ttl, at which point every node rejects it. Shorten credential_ttl for a tighter bound.

The anchor

The anchor is just a normal mesh node that additionally holds the CA key and runs a small HTTP control plane: GET /directory, POST /publish, POST /renew, GET /health — bound to its overlay address (reachable only through the mesh, never the underlay). There is no separate coordination service, no relays, no SaaS, nothing always-on in the data path. Nodes poll /directory, merge records by highest sequence number, and cache them locally.

Because trust is anchored to the CA key (not a machine), any node can become the anchor — restore the key onto a replacement, or stand up a new CA and re-point the fleet. See Moving the anchor.

Direct-or-fail

There is no routing, no multi-hop, no relays, and no NAT traversal. Two nodes either form a direct WireGuard tunnel or the link fails. It never silently falls back to relaying through a third party, so there's no hidden path to reason about.

The only logic that modifies a normal wireguard configuration is the reconcile loop: every few seconds each node walks the directory and, per peer, runs seven checks: verify the CA signature, check expiry, verify the self-signature, verify the address derives from id_pub, check the revoke list, check the authorization policy — then installs or removes that peer with wg set. Membership changes, revocations, key rotations, and access policy all reduce to "add or remove a peer," computed locally with no coordinator. A link forms as long as at least one side is reachable (see Reachability) two unreachable nodes can't pair.

IPv6 overlay

The overlay is IPv6-only. Every node's mesh address is a hash of its identity key under a ULA prefix, and all in-tunnel traffic is IPv6.

The underlay (the real network each WireGuard endpoint lives on) can be IPv4 or IPv6. A node advertises whatever public endpoint(s) it has, and each node dials a peer over a family they share.

A dual-stack peer advertises both its v6 and v4 endpoints (v6 first). If the preferred one produces no handshake for ~20s, the reconcile loop rotates to the next advertised endpoint and keeps round-robining until one connects. So a peer reachable on v4 but with a broken v6 path still links, with no manual intervention. This is still direct-or-fail: only endpoints the peer advertised are ever tried, never a relay.

Self-certifying addresses

A node's overlay address is a hash of its identity public key:

fd8d:e5c1:db1a:7 : truncate64(blake2s(id_pub))
                   └── the last 64 bits ARE the key's fingerprint ──┘

That function is public and deterministic, so anyone holding a node's id_pub can recompute its address and check it matches. The address certifies itself against the key — no allocator, and no authority needs to vouch that "this address belongs to this node."

Offline tolerance

Every node caches the directory on disk and keeps its tunnels running from that cache, so the anchor can be down for up to one credential lifetime and existing node↔node links are unaffected. The anchor is never in the data path. Only new enrollments and credential renewals need a reachable anchor. Restore or replace the anchor within that window (see Moving the anchor and the RUNBOOK) and nothing ever drops.

Linux-only

greasewood is built on Linux-specific kernel interfaces — the in-kernel WireGuard module and the kernel's own networking — and is best run as a systemd service (the recommended way to keep the daemon up across reboots and crashes; a bare gw run works for dev). It relies on the kernel's WireGuard and on systemd rather than shipping its own userspace transport (the way a Go implementation such as wireguard-go does) or its own supervisor. It reaches those kernel interfaces via the stock wg/ip tools (see Auditable). Other platforms would need a different data-plane backend and a different supervisor, and are out of scope: greasewood is, deliberately, a Linux tool.

Greasy

greasewood drives the data plane by shelling out to the stock wg, ip, and nft command-line tools over subprocess, rather than talking to the kernel directly through netlink (via a binding like pyroute2, or the userspace transport a Go implementation such as wireguard-go ships). The clean way would be netlink; this is the greasy way. It's a deliberate trade, and the single biggest one in the codebase.

Why it's the right trade here. greasewood's top priority is being easy to reason about, and its scale is small — tens of nodes, a reconcile every few seconds, not thousands of updates a second. At that scale the readability win dominates, and the surface it depends on (wg set, ip route, nft) is small and stable, so the fragility is bounded.

Pros:

• It is the audit trail. Every data-plane change is a real command, so greasewood can log the exact wg/ip/nft argv it ran, with the exit code and the reason — see Auditable. You can copy any line, run it by hand, and compare against wg show.
• One dependency. The tools are already on the host, so the whole thing stays pure Python with one dependency (cryptography). A netlink binding would add a dependency (or a pile of fragile low-level socket code).
• Reproducible by hand. Debugging is "what command did it run, and what happens when I run it myself" — not decoding a binary protocol.
• Matches the operator's mental model. greasewood is for people who already manage WireGuard by hand; the code reads the way they'd type.

Cons:

• Slower. A process spawn per command costs more than a netlink round-trip. Fine at this scale; it would not suit a control plane pushing thousands of rapid updates.
• Text parsing is more brittle than structured netlink replies. greasewood is tied to the tools' CLI output, which can shift across versions.
• Depends on the tools being present and current. wg, ip, and nft must be on PATH at compatible versions.
• No transactional batching. netlink can apply a set of changes atomically; subprocess means more round-trips and per-command error handling (exit codes and stderr rather than typed errors).

Auditable

The entire thing is pure Python (3.11+), one dependency (cryptography), one binary (gw), and it manages the data plane by shelling out to wg/ip via subprocess (the greasy trade): you can read the exact wg set peer … commands, run them by hand, and compare them against what wg show reports.

That pays off in the command trail. Because every data-plane change is a subprocess, greasewood records every ip/wg command it issues — always, with the exit code, how long it took, and why it ran — to a durable, rotating <data_dir>/audit.log and to the journal. One greppable logfmt line per command:

ts=2026-07-02T10:15:03Z INFO greasewood.audit: cmd rc=0 t=12ms \
  ctx="reconcile: +peer db01 [fd8d:e5c1:db1a:7::a1] seg=prod" \
  argv="wg set gw-myfleet peer <pub> allowed-ips fd8d:e5c1:db1a:7::a1/128 endpoint [203.0.113.7]:51900 ..."

In addition to the per-command lines, the same file carries a domain-event trail one line per mesh state transition grep event= audit.log:

ts=2026-07-02T22:12:03Z INFO greasewood.audit: event=policy prev=4 seq=5 grants=3
ts=2026-07-02T22:12:08Z INFO greasewood.audit: event=topology added=2 removed=1 peers=7

So the narrative reads at a glance: policy v4→v5 was adopted, and the next reconcile settled the topology (+2/−1, 7 peers) with the per-peer +peer/ -peer commands right below carrying the who and why. A topology line appears only when membership actually changes (a re-verified endpoint isn't a transition), so steady state stays silent.

And you don't have to read raw logs: gw narrate translates the trail into plain English — grouping the commands of each operation and explaining what each did and why:

$ gw narrate --since 2h
● 2026-07-02 22:12:03Z  A new node enrolled through the door and was installed
                        as a peer (db01 [fd8d:e5c1:db1a:7::a1] from fd52:ba5e::9).
    ✓ Set up the WireGuard tunnel to peer qa7IAQabcd=: accept and route its
      overlay address fd8d:e5c1:db1a:7::a1 (a /128 host route — one address per
      peer, derived from its identity key); dial it at 203.0.113.7:51900; send a
      keepalive every 25s to hold the path open.                          (14ms)
    ✓ Route traffic for fd8d:e5c1:db1a:7::a1 over gw-myfleet — wg configures the
      peer but not the kernel route, so greasewood adds it explicitly.     (3ms)

Filter it (--peer db01, --failures, --grep, --stats), point it at any log file or - for stdin, or --raw to see the argv alongside. This is the core of the auditability claim: you can reconstruct and read as a narrative every change greasewood ever made to your kernel's network state.

Install

Requires Python 3.11+, the WireGuard userspace tools (wireguard-tools/wg), and iproute2 (ip). The kernel WireGuard module is built into Linux 5.6+ and autoloads on first use.

Two ways to install on a host that will run the daemon; both set up the managed systemd service.

With pipx (recommended on Linux) — the standard way to install a Python application in its own isolated environment, straight from PyPI:

sudo apt install pipx wireguard-tools    # Debian/Ubuntu; use your distro's pkg mgr
sudo pipx install --global greasewood

--global puts gw on root's PATH so sudo gw … resolves, and the daemon service launches as <interpreter> -m greasewood, so it stays valid wherever pipx put the package. pipx manages only the Python side — install the WireGuard tools separately with your distro's package manager (shown above). Upgrade with sudo pipx upgrade greasewood.

Prefer uv? sudo UV_TOOL_BIN_DIR=/usr/local/bin uv tool install greasewood is the faster equivalent (uv tool is uv's app installer). uv has no --global flag, so point its bin dir at a spot already on root's PATH yourself, as shown.

To run an unreleased commit instead of the latest release, point pipx at the git URL — it builds the default branch directly, no clone needed:

sudo pipx install --global "git+https://github.com/cschlick/greasewood.git"

(For a git install, pull newer commits with sudo pipx reinstall greasewood — a clean re-pull; plain pipx upgrade can skip a git install when the version string hasn't moved.)

With the bundled installer — a self-contained alternative that also installs the WireGuard deps and pins a fixed venv at /opt/greasewood. Re-run any time (after a git pull) to upgrade in place:

git clone https://github.com/cschlick/greasewood.git
cd greasewood
sudo ./install.sh

Distro packages — .deb and .rpm are attached to each GitHub release; they bundle their own Python, so they need nothing but glibc (and pull in wireguard-tools, iproute2, nftables):

sudo apt install ./greasewood_<ver>_amd64.deb     # Debian/Ubuntu
sudo dnf install ./greasewood-<ver>.x86_64.rpm    # Fedora/RHEL

On Arch it's on the AUR: yay -S greasewood. See packaging/ for how these are built.

Either way you get the gw command (and man gw), and gw create/join install + enable the systemd service. Most subcommands need sudo/root (they create WireGuard interfaces and edit routing); gw watch does not. For plain library/dev use, pip install greasewood (or pip install '.[test]' from a checkout for pytest).

After install the workflow is just setup/join → the daemon runs as a managed systemd service that gw create/gw join set up for you — see Running as a service. The Quickstart below runs it by hand with gw run to show the moving parts.

First-run pitfalls

Every one of these comes from a real fleet coming up; each has a one-command fix.

• sudo: gw: command not found after a plain pipx install. A per-user install lands in ~/.local/bin, which sudo's secure_path never searches. And "fixing" it with sudo pipx install (no --global) is the same trap one home directory over: it lands in /root/.local/bin, on nobody's PATH. The --global flag isn't cosmetic — the daemon is a root service, so its venv must live somewhere root-owned and stable, not inside a login account's home. Uninstall the stray one (pipx uninstall greasewood, with sudo if it went to root's home) and reinstall with --global.

• pipx: error: unrecognized arguments: --global. The distro's pipx predates 1.5 (Ubuntu 24.04 ships 1.4.3). Set by hand exactly what --global sets:

  sudo PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install greasewood
  

  Upgrades on that host need the same env vars (a plain sudo pipx upgrade looks in /root/.local and finds nothing). Or skip pipx and use the bundled install.sh, which pins its own venv at /opt/greasewood.

• FileNotFoundError: ... 'wg' at join. pipx installs only the Python side; the WireGuard tools come from your distro (sudo apt install wireguard-tools). Don't be fooled by the join getting partway: the kernel module being present (ip link add … type wireguard succeeds) says nothing about the wg binary being installed. The distro .deb/.rpm packages pull it in for you; pipx cannot.

• credential verification failed: no trusted CA signature found at join — even with a fresh token. If the anchor was re-created (gw create --force) after its daemon started, the daemon keeps signing with the old CA it loaded at startup while every new invite embeds the new on-disk CA: the join's door tunnel comes up (the door key survives a re-create) and then the credential is rightly refused. Restart the anchor daemon so it re-reads the key, then mint a fresh invite:

  sudo systemctl restart greasewood@<mesh>   # on the anchor
  sudo gw invite                             # token minted AFTER the restart
  

Quickstart

1. Bootstrap the anchor

On the machine that will hold the CA and serve enrollment:

sudo gw create mymesh          # names live under *.mymesh.internal

create generates the CA, the persistent door key, the policy routing for the enrollment door, and the anchor's own credential, then writes /etc/greasewood_mymesh.toml. By default create also starts the daemon: it brings up the gw-mymesh WireGuard interface, serves the control plane, and watches for door windows.

The anchor takes this machine's hostname like any other node. You tell which node is the anchor from role: anchor in gw watch, not from its name. (Pass --hostname <name> to override the default.)

2. Enroll a node

Enrollment uses a transient WireGuard "door". This provides a mechanism to allow new nodes onto the mesh that is much lower trust than, for example, relying on ssh. Also since wireguard doesn't respond to connections without a recognized credential, it is a much cleaner external profile than, for example, running an http server for configuration on the underlay. To add a new node to the mesh, create an invite token on the anchor:

sudo gw invite # prints token

Deliver that token to the new machine (any channel) and redeem it:

sudo gw join <token>

join derives a throwaway guest key from the token, stands up a temporary gw-door tunnel to the anchor, receives a CA-signed credential over it, tears the door down, and writes the node's config, then the anchor brings the node into the mesh.

Why a door — why can't the token just contain everything to peer over your mesh interface? Because WireGuard peering is mutual: to bring up a tunnel to the anchor over any interface, the anchor must already have your public key in its peer list. At invite time your real keys don't exist yet (they're generated locally at join, and private keys never travel) so the anchor cannot pre-authorize your real identity key.

What the token can do is carry a 32-byte seed that both sides expand (HKDF) into the same throwaway door keypair + PSK. The anchor derives that throwaway pubkey from the seed it minted and pre-installs it as a peer; you derive the matching private key — so now a tunnel can actually form. But it forms under a disposable, credential-less key, not your identity. That's why the door is a separate interface:

• it runs on its own dedicated door subnet (fd8d:…:d::/64) — not a throwaway address squatting on the real overlay (which would break the self-certifying address = hash(identity))
• it reaches only the enroll daemon (not the directory/control plane, not other peers) A token-holder can do exactly one thing: request a credential;
• it's torn down the moment your credential is issued.

So the door bootstraps joining the mesh with your real identity. You bring up the mesh interface, with your real key and its self-certifying address, only once you hold that credential. Running the throwaway peering over the mesh interface instead would drop a credential-less stranger onto the live mesh with a fake address and expose the whole control plane. The door is a temporary quarantine.

3. Check it

sudo gw watch              # a live ( or --snapshot) view of the mesh peers
sudo wg show gw-mymesh     # wg show works normally, showing each tunnel

gw diagnose is the tool to reach for when a peer won't connect. It's pairwise: it lays up to two named nodes plus the anchor side by side and explains, per pair, whether a tunnel can form — policy (roles/grants), reachability, and the firewall/routing directionality that's usually the real question. (gw watch is the fleet-wide link overview; diagnose is the focused deep-dive.)

sudo gw diagnose            # this host ↔ the anchor
sudo gw diagnose db01       # this host ↔ db01   (+ anchor as reference)
sudo gw diagnose db01 web1  # db01 ↔ web1        (+ anchor as reference)

The comparison table shows each node's addresses, reachability, roles, credential, and firewall for the mesh UDP port.

Running as a service

On a systemd host the daemon is managed for you — create and join install the service and start it, no extra command. A single template unit serves every mesh as its own instance greasewood@<name> (survives reboots, restarts on failure, logs to the journal), so the whole workflow is just:

sudo gw create mymesh                 # anchor  → greasewood@myfleet installed + running
sudo gw join "$TOKEN" --hostname n01  # node → greasewood@<mesh> installed + running
journalctl -u greasewood@mymesh  -f   # watch a mesh's daemon
systemctl status 'greasewood@*'       # all of them

There is no separate install/uninstall step: the service lifecycle rides on the mesh lifecycle. gw purge removes a mesh's instance (and the shared template when it's the last mesh) providing a from-scratch reset in one command.

• Not on systemd, or want to run it yourself? Pass --no-service to create/join; they print the sudo gw -c <config> run line instead and touch nothing under /etc/systemd. (A non-systemd host auto-falls-back to this even without the flag.)
• Instances run gw run as root (they manage WireGuard interfaces and routing). Don't also run gw run by hand while an instance is up, both would fight over the interface.
• A config-changing re-join (new anchor, etc) isn't auto-detected — the daemon reads its config at startup, so run sudo systemctl restart greasewood@<name> afterward.

Provisioning many nodes

Enrollment tokens are initiated by the anchor, never by nodes. A node cannot request admission; you (or an orchestrator acting on the anchor) decide to admit a machine, run gw invite, and deliver the token out of band. The node only redeems what it was handed. The door is single-slot by construction: each invite opens one enrollment window, and the anchor closes it the instant that node finishes joining. To persist a door over multiple joins (reuse a token, useful for provisioning many instances at once), use the --standing flag:

On the anchor:

sudo gw invite               # prints a one-time token
sudo gw invite --standing    # prints a multi-use token

Joining is the same either way

sudo gw join <token>

Firewall

greasewood never modfies your external (underlay) firewall Its control plane (51902/tcp) and enrollment RPC (51903/tcp) bind only to the node's overlay address and loopback never the underlay, so nothing it runs is reachable off-mesh regardless of firewall policy. The only thing that must face the underlay is WireGuard itself (UDP), which you open yourself like for any VPN.

greasewood uses four ports, one contiguous block — two WireGuard udp ports on the underlay, and two TCP service ports on the overlay.

	UDP — WireGuard transport	TCP — service inside the tunnel
mesh	51900 — overlay data plane	51902 — control plane
door	51901 — ephemeral join tunnel	51903 — enroll exchange

create and join check the local nftables ruleset and loudly warn if a needed port looks blocked by a default-drop policy, printing the exact rule to add. That's all greasewood does. You apply the printed rules yourself (put them in your nftables config, or however you configure your firewall).

No firewall at all? Then there's nothing to do — and nothing extra is exposed. greasewood binds nothing to the underlay except its WireGuard UDP port(s): 51900 (mesh) on any node, plus 51901 (the enrollment door) on the anchor. Those are WireGuard, which is designed to face the internet — it silently drops any packet that isn't a valid handshake from a configured peer (no reply, no info leak). Everything else — the control plane (51902) and the enrollment exchange (51903) — binds to the overlay address or the door tunnel, so it's structurally off the underlay whether or not you run a firewall. A greasewood host with no firewall is therefore no more exposed than a plain WireGuard host with no firewall. The rules below matter only on a host that runs a default-drop policy and so must explicitly allow those ports through.

On a default-drop host, allow (nftables). With port enforcement on (the default), greasewood's own nftables table filters the overlay interfaces (control plane, enrollment + door lockdown, and the grant-derived ports), so your firewall just opens the two underlay UDP ports and admits the overlay — greasewood does the rest:

Interface	Rule	Purpose
underlay	udp dport 51900 accept	mesh WireGuard
underlay	udp dport 51901 accept	enrollment door (during join)
lo	iifname "lo" accept	the host talks to itself (::1:51902)
gw-*	iifname "gw-*" accept	admit the overlay; greasewood's table filters the ports on it

udp dport { 51900, 51901 } accept
iifname "lo" accept
iifname "gw-*" accept

That coarse iifname "gw-*" accept is required in a default drop context to allow traffic to reach greasewood's overlay table (greasewood's table can only tighten what your firewall admits, never open it). Greasewood's table then scopes the control plane to gw-<mesh>, locks gw-door to enrollment only, and applies the grant table's port scopes.

If you turn enforcement off (enforce_ports = false), greasewood installs no table. It's purpose is to enforce access control to ports on this machine from a central location (the anchor), which requires cooperation from to enforce. It is opt-in.

Multi-user hosts: the overlay is host-wide, any local user can use the tunnel once it's up (identity is per-machine, not per-user). To restrict which users may originate overlay traffic, add an nftables owner-match on the output chain; see the "Multi-user hosts" section of SECURITY.md.

Reachability

WireGuard has no client/server roles. Both peers try to handshake and the direction that physically works wins, then endpoint roaming pins it. So a link forms as long as at least one side is reachable: a firewalled node dials an open one, and the reply returns via the NAT hole it punched. Two fully-blocked nodes can't pair (direct-or-fail — no relays). Greasewood inherits this semi-tolerance of NAT/firewall issues from WireGuard directly, but it makes no serious effort to reason about the underlying network state and automatically just work (that is the domain of Tailscale, etc). The best greasewood can do is to examine IP address and determine when a node obviously has no externally reachable address (LAN NAT, CGNAT, etc) and print that in gw diagnose. So, when a reachability issue arise it is very likely to be that BOTH peers are behind a NAT/Firewall that does not provide them an externally reachable endpoint. The only solution in that case would be to try and make changes to the underlying network state.

An anchor must be reachable (it serves the control plane), so in order for a node to become the anchor, it must have a reachable external address. So anchor-promote refuses on a node that knows it advertises no endpoint.

Access control (roles & grants)

A fresh anchor ships default-closed: the grant table is a secure star — the anchor (which holds role:admin) can SSH every node, nodes reach only the anchor's control plane, and nodes cannot reach each other at all. You open what you need by adding grants.

To control who talks to whom, on what, give nodes roles and write a grant table. The mesh then derives its tunnel topology from the policy. Three roles are built in:

Role	Who holds it	Meaning
node	every ordinary member (the default)	an ordinary fleet node
anchor	the anchor, and only the anchor	single-member reserved. Addressable in grants as to = ["anchor"]
admin	the anchor by default, tag any box	terminal access — SSH to every node.

Roles are CA-signed caps (role:<name>), and crucially, the anchor assigns them, a node cannot assert its own. However, the anchor can choose to provide new nodes with a "menu" of available roles. This is helpful for auto-provisioning, where multiple roles can join with a single standing token.

# Fixed role — the invite decides; join takes what the token granted:
TOKEN=$(sudo gw invite --roles web)          # this token → a role:web node
sudo gw join "$TOKEN" --hostname web1

# ...or a MENU — one standing token, and the joiner picks a role from it:
MENU=$(sudo gw invite --self-roles web,worker,db)
#   ...or derive the menu from the policy itself — every role grants.toml
#   references, minus built-ins (*, anchor, node, admin):
# MENU=$(sudo gw invite --self-roles-from-grants)
sudo gw join "$MENU" --roles worker --hostname worker1   # anchor signs iff 'worker' is on the menu

# Change roles later without re-joining (effective at the node's next renewal):
`sudo gw set-roles web1 web,worker`

Defaults for new nodes live in the anchor's config — [anchor] default_roles (["node"]) and default_caps (["tls"]). A plain gw invite uses them; the flags override per token; they're read fresh at each invite, so editing the config changes future enrollments with no restart.

The grant table derives the topology (examples)

# <data_dir>/grants.toml on the anchor  (full reference: grants.toml.example)
[[grant]]
from  = ["web", "worker"]
to    = ["api"]
ports = ["tcp/8000"]
# why: the app tier calls the API; nothing else does.

[[grant]]
from  = ["metrics"]
to    = ["*"]
ports = ["tcp/9100"]
# why: prometheus scrapes everyone — hub-and-spoke tunnels, not a clique.

A single grant can be a whole service: worker -> files : tcp/2049 is a complete file share, with the grant table acting as the share's ACL — see the NFS worked example.

Declarative role assignments ([assign])

Roles are assigned imperatively by default (gw invite --roles, gw set-roles). Add an optional [assign] table to grants.toml and the file also declares who holds which roles — making the one policy file a complete, diffable description of the mesh: membership shape and topology:

[assign]
nas = ["nfs_srv"]
bb  = ["nfs_usr", "web"]

gw policy apply reconciles listed hosts' roles to the table — previewing role diffs and the resulting tunnel delta exactly like a grant edit, then sending one fleet renew hint so every change lands within a poll interval. Listed hosts refuse gw set-roles (an imperative edit would silently drift from the file), and the gw watch role editor writes this table rather than the registry — the TUI is a hand on the file, never a bypass of it. Unlisted hosts keep the imperative flow, so menu-invite auto-provisioning composes untouched; a listed host that hasn't joined yet warns at apply and reconciles automatically once it does. No [assign] section — no change at all.

Grants to specific machines (host:)

When a grant really means this one box, name it directly instead of minting a single-member role:

[[grant]]
from  = ["host:bb"]
to    = ["host:nas"]
ports = ["tcp/2049"]
# why: exactly one workstation mounts the NAS — no role ceremony for a pair.

host:<name> matches the node's CA-attested hostname — the same signed credential field behind /etc/hosts names and TLS CNs — so a host grant rides the identical trust chain as a role: nothing self-asserted enters the policy. (Mechanically it's a derived tag, computed from the credential at match time; it is never stored and can never be assigned or spoofed as a cap.) Roles and host entries mix freely in one grant: from = ["worker", "host:laptop"].

The one caveat: a host grant is only as strong as name assignment. Roles are always anchor-chosen, but by default a node names itself at join — so a grant written for a name nobody currently holds (or a name freed by decommissioning) would be inherited by whichever machine the anchor next admits under it. gw policy apply warns on both cases (an unheld host: name, and a granted node whose name is self-chosen), and the fix is one flag: pin any name you grant by (gw invite --hostname nas — anchor-chosen, un-renameable). Relatedly, the grant follows the name: gw rename-node warns and asks before renaming a node out of its own grants (fail closed).

# the low-friction path — finds the file, opens your editor ($SUDO_EDITOR/
# $VISUAL/$EDITOR, else nano), validates on save, then offers the apply preview:
sudo gw policy edit

# or edit the anchor's <data_dir>/grants.toml by hand, then apply — with a preview:
sudo gw policy apply
#   this will change the policy: v1 → v2
#     - grant  * -> * : *
#     + grant  web -> api : tcp/8000     ← the rule change (X → Y)
#     - tunnel web1 ↔ web2               ← what actually connects/disconnects
#   apply? [y/N]
gw policy show              # on any node: the active table (flags unapplied edits)

grants.toml is the source of truth. gw create writes it (the default-closed baseline — admin -> anchor,node : tcp/22) and signs it into the distributed, CA-signed policy.json — the form nodes actually receive and trust (a node can't trust a plaintext file from another host). To change policy you edit grants.toml and run gw policy apply, which previews the change and asks you to confirm before signing it: a policy change tears down tunnels, so it is never applied silently by a stray file save. gw policy show flags an edited-but-unapplied grants.toml so a forgotten apply is visible. A joining node is handed the current signed policy at enrollment, so it enforces the real table from its first run — the mesh never operates on an implicit default.

With a policy applied, a tunnel exists between two nodes only if some grant connects their roles (either direction — tunnels are symmetric; the grant's direction is for port filtering). Tunnels are minimal by construction: delete a grant and its tunnels are torn down on the next sync; peers, keepalives, and handshake exposure all shrink to the grant graph. Two web nodes have no tunnel unless someone writes web -> web, client and server are roles that coexist without talking sideways.

Segments are emergent, not configured. There is no segment cap, flag, or config key: a "segment" is the unnamed connected structure the grant graph produces — role:web and role:api nodes granted an interface share one, and it dissolves when the grant does. gw watch --by-role shows the groups and flags policy-expected links that are down (a real fault) without false-alarming on pairs the policy correctly keeps apart.

Properties to rely on:

• Allow-only, by schema. A flow passes iff some grant covers it; there's no deny rule (no action field) — grants are a set, not an ordered program, so no conflicts.
• The anchor is hardwired beneath the table. Every node always tunnels to the anchor (role:*), and no grant can prune it — the policy rides the directory sync, which rides the anchor tunnel, so the channel that carries the policy can't be severed by the policy.
• Signed and replay-proof. The table is CA-signed with a monotonic version; nodes adopt only newer, validly-signed tables and keep last-known-good on disk across reboots.
• Anchor-assigned, mutually enforced. A tunnel needs both ends to install each other, each reading the other's roles from its CA-signed credential — a node can't talk its way into a role it wasn't issued, nor be forced into a link it denies.

Port enforcement is on by default. The daemon realizes each grant's ports in greasewood's own table inet greasewood_<mesh>, scoped to the mesh interface: it default-denies mesh traffic and admits only the granted flows (server-side inbound; a client's replies ride ct established, so the asymmetry needs no rule). A fresh anchor ships default-closed — a secure star where only role:admin (the anchor) can SSH nodes; you open services by writing grants. Enforcement is a policy state, always installed, not a mode you switch on.

It writes only its own table on gw-<mesh> — never your host firewall, never a physical NIC — so it can only ever tighten, and it presupposes you've admitted the overlay (iifname "gw-<mesh>" accept, or no host firewall; gw firewall advises exactly that). The table persists across daemon restarts (fail closed); gw purge removes it.

Because enforcement is on by default, nftables must be usable — the daemon refuses to start rather than run silently unenforced. A host without it sets enforce_ports = false under [network] (or a one-off gw run --no-enforce-ports): grants still gate which tunnels exist, but port scopes go advisory.

Command reference

Bare gw is the dashboard, not a usage error: sudo gw in a terminal opens the live watch view; without root (or piped) it prints the static snapshot with an everyday-commands index below it; on an unconfigured or multi-mesh host it tells you how to start (or which -c). gw --help keeps the full reference.

Command	sudo?	What it does
create	yes	One-shot anchor bootstrap: CA, door key, routing, self-cred.
run	yes	Start the daemon (WireGuard iface, control plane, loops). Port enforcement (grant port scopes, nftables) is on by default; --no-enforce-ports (or enforce_ports=false) disables it for an nft-less host. See Access control.
invite	yes	Open a 15-min door window, print a single-use join token. --standing opens a standing door instead: one token, any number of enrollments, until close-door.
close-door	yes	Close the current door window — permanently invalidates its token (standing or single-use); enrolled nodes unaffected.
join <token>	yes	Enroll this machine using a token from invite.
watch	sudo	Live mesh dashboard (redraws in place, so it needs sudo for live WireGuard state): the split roster + link state, per-second throughput, and a latency column that fills in as pings return. Ctrl-C to exit. --snapshot prints one static view and exits (no root; auto-used when piped); --json emits that snapshot as a stable, versioned schema for monitors/jq (add sudo for live WireGuard stats) — and the human roster is rendered from that very JSON, so the machine contract can't silently drift from what you see. --by-role groups by role and, per group, reports connectivity — connected components (the emergent segments), and policy-expected-but-down edges with a firewall/NAT hint — computed fleet-wide from each node's self-reported live links. Shows how fresh the view is (time since last sync) up top.
config [key]	no	Print resolved config facts machine-readably for scripting — gw config interface gives the mesh interface name (gw-<mesh>), no arg lists all as key<TAB>value.
firewall	no	Print the recommended firewall ruleset (a suggestion — greasewood never changes your firewall; nothing is applied). The same posture on every node; with sudo also flags anything that looks blocked.
diagnose [A [B]]	sudo	Pairwise link diagnosis: compare up to two nodes + the anchor side by side and explain whether a tunnel can form (policy/roles, reachability, firewall directionality with OPEN-inferred-from-handshake and upstream-router localization). No args = this host ↔ anchor.
revoke <node>	no	Revoke a node on the anchor (denies renew/publish, evicts it, frees its hostname). <node> = hostname, <host>.<mesh_domain> mesh name, or 64-char id_pub hex.
set-roles <node> <r>	no	Change a node's roles (on the anchor; effective next renewal).
policy	show: no · apply: sudo	The mesh's grant table (roles → roles : ports; derives which tunnels exist). show renders the active policy on any node; apply (anchor) validates grants.toml, previews the tunnel delta, signs with the CA key, publishes. See Roles & the grant table.
set-caps <node> <caps>	no	Change a node's full tag set (on the anchor; effective next renewal).
anchor-promote	yes	Turn this enrolled node into an anchor (generate its own CA key).
cert-request	no	Get an x509 TLS cert from the anchor for a local service. The daemon auto-renews it at ~half its TTL; --reload-cmd runs a command after each renewal, --no-auto-renew opts out. --profile <name|path> issues + places the key/cert/ca where the service wants them (right owner/mode) and re-places on every renewal; --profile <name> --show prints a bundled template to adapt.
cert-profiles	no	List the bundled cert profile templates (postgres, nginx, haproxy, redis, nats, minio, mosquitto) — starting points to copy and adapt.
cert-remove <name>	sudo	Stop managing a cert (drop it from auto-renewal + remove its profile snapshot). Leaves the placed files by default; --delete-files removes them too.
cert-status	no	Show every daemon-managed TLS cert (expiry, renewal state, SANs, placed files, profile) from the manifest — wherever the files live.
narrate	no	Translate the ip/wg command trail (audit.log) into a plain-English story of what greasewood did and why. Filters: --since, --peer, --grep, --failures, --stats, --raw.
rename-node <name>	yes	Change this node's mesh hostname (anchor-validated, no re-join; refused if the anchor pinned the name).
rename-mesh <name>	yes	Rename this mesh — domain, config, data dir, interface, and service move together. Run on the anchor, then on each member (surfaced in its gw watch). Old names resolve + verify in TLS through a one-TTL grace window. See the RUNBOOK SOP.
renew	yes	Force an immediate credential renewal for this node (applies an anchor-side set-caps/set-roles now, instead of at the ~half-TTL renewal).
renew-all	no	On the anchor: request a fleet-wide renewal (advertise renew_after=now; cooperating nodes renew, jittered so the anchor's rate stays ~constant with mesh size).
anchor-backup	no	On the anchor: write one passphrase-encrypted archive of the CA key, node registry, revoke list, door key, and anchor identity. Store it offline.
anchor-restore	yes	Restore a anchor-backup archive onto a replacement host (same CA key → a restore, not a re-root).
anchor-transfer <host>	yes	On the anchor: hand the anchor role to another host over SSH (same CA → no re-root; the CA never touches the mesh). Streams the encrypted state, copies the config, then stops here / starts there / verifies — rolling back if the target doesn't come up. See the RUNBOOK SOP.
purge	yes	Remove all greasewood state from this machine.

Global flags: -c/--config FILE (default: discovered — with one mesh on the host every command finds its config unaided; with several, gw lists them and asks for -c) and -v/--verbose. Both must precede the subcommand (gw -v run, not gw run -v).

Enrollment is door-only: invite on the anchor, join on the node. There is no manual credential-copy path.

Configuration

gw create and gw join write /etc/greasewood_<name>.toml for you; see greasewood.toml.example for the full annotated schema. Key fields:

[node]
hostname = "node01"
role     = "node"          # "anchor" | "node"
caps     = ["role:mesh"]     # role:<x> tags are the grant-table vocabulary; "tls" allows certs

[network]
interface  = "gw-myfleet"
listen_port = 51900
overlay_prefix = "fd8d:e5c1:db1a:7::"        # the fleet's overlay /64 (ULA)
seeds    = ["http://[<anchor-overlay>]:51902"]  # directory URLs to pull (the anchor)
root_url = "http://[<anchor-overlay>]:51902"    # where to publish / renew
hosts_sync  = true                           # manage /etc/hosts names (on by default)
mesh_domain = "myfleet.internal"             # the mesh's ONE domain (from create <name>)

[ca]
trusted_pubs = ["<hex Ed25519 CA pubkey>"]   # a set, to allow CA migration

[anchor]                        # anchor role only
ca_key_file    = "/var/lib/greasewood/ca.key"
control_listen = ":51902"
credential_ttl = "24h"

Roles

• anchor — holds the CA private key; serves the control plane and the enrollment door; participates in the mesh.
• node — a plain mesh participant.

One host on two meshes

The overlay /64 is configurable ([network] overlay_prefix, set at create --overlay-prefix; a node learns it from its credential at join). A node learns and verifies addresses prefix-agnostically — the self-certifying part is the host bits, blake2s(id_pub), and the CA signature attests the prefix — so one host can be a plain node on two independent meshes at once (anchor-in-two-meshes is not supported). Joining a second mesh is just:

sudo gw join "$TOKEN_B"        # that's it

join routes by the token's CA: a token for a mesh you're already on refreshes that membership; an unknown CA auto-provisions the next membership slot — config /etc/greasewood2.toml, data /var/lib/greasewood2, interface gw-<name>, UDP 51910 (then +10 each). The mesh's name domain rides in the token (declared once at gw create <name> → <name>.internal), so every member of a mesh, including multi-mesh hosts, mounts it under the SAME suffix, and TLS names agree fleet-wide with no flags.

Domain collisions: a node cannot bridge two meshes with the same domain — no local aliasing exists (a per-host alias would diverge from the names in the mesh's TLS certs, a debugging trap; and rewriting is off the table since names are CA-attested). The join refuses before the door dance (the token is not consumed) and tells you the fix: rename one mesh on its anchor. Requiring a mesh name at create (becomes subdomain) should makes this a rare coincidence.

Every derived value is still overridable — pass any of the explicit knobs and the auto-slotting steps aside entirely:

sudo gw -c /etc/gw-b.toml join "$TOKEN_B" --data-dir /var/lib/gw-b \
    --interface gw-b --listen-port 51920 --mesh-domain beta

The mesh domain must differ between the two, for the same reason the interface name must — both are flat, host-global namespaces with no scoping. The /etc/hosts block is keyed by mesh_domain, so two meshes sharing one would (a) clobber each other's block every reconcile, each daemon strips and rewrites the same-tagged block, and (b) collide on the names themselves: both meshes' db.mymesh.internal would claim the same name for two different addresses.

Names

Every node has a stable overlay address, and gw watch shows each node's resolvable name↔address map. Name resolution is encouraged: the daemon keeps a marked /etc/hosts block mapping each node's address to <hostname>.<mesh>.internal (e.g. db.mymesh.internal), built from the records that pass the reconcile loop's full verification — the same gate that decides WireGuard peers, so a revoked or expired node's name stops resolving on the same cycle its tunnel comes down. It's re-checked each reconcile but only rewritten when the block actually changes (a join, departure, revocation, or rename) — in steady state it never touches the file, so it won't hammer /etc/hosts

# BEGIN greasewood — managed, do not edit
fd8d:e5c1:db1a:7:…  db.mymesh.internal
fd8d:e5c1:db1a:7:…  node01.mymesh.internal
# END greasewood

So ping db.mymesh.internal, ssh db.mymesh.internal, etc. just work — no DNS server, and it keeps resolving even if the anchor is down (it's from the cache, for as long as the cached credentials remain valid — one credential TTL, the same horizon as the tunnels themselves). It only ever touches the region between its markers; your own /etc/hosts lines are left alone, and --no-hosts-sync (or hosts_sync = false + restart) or gw purge removes the block.

A node can also publish extra service names under its own name via [network] aliases (or automatically from a subdomain TLS cert) — a label pg becomes an extra pg.<hostname>.myfleet.internal line pointing at that node. See the TLS section for how this ties cert SANs to resolvable names.

Who chooses the name. By default a node names itself at gw join (defaulting to its machine hostname) and can change it later with gw rename-node. If you'd rather the anchor control it, pin it at invite: gw invite --hostname db fixes the name at enrollment (the joiner's requested name is ignored) and marks the credential so the node can't gw rename-node itself — to change a pinned name, re-invite with a new --hostname. Either way the name is CA-attested; pinning just moves the decision from the node to the anchor.

Two things make defaulting this on safe:

• Names are CA-attested (the hostname lives in the signed credential), so a member can't publish a record claiming another node's name to poison your hosts.
• Names are namespaced under the mesh's own <name>.internal label.

The domain is shared with TLS: gw cert-request with no --san defaults the cert to this node's <hostname>.myfleet.internal plus its overlay address. So the name a node is reached by is exactly the name its certificate is valid for — resolve db.myfleet.internal → connect over WireGuard → TLS validates the db.myfleet.internal SAN (Subject Alternative Name — the x509 field listing the names a certificate is valid for).

A node's hostname defaults to the machine's own hostname at enrollment; change it later with sudo gw rename-node <newname> (then restart the daemon). Rename goes through the anchor, so it's uniqueness-checked and frees the old name. The keys and overlay address don't change. (Editing hostname in the config directly is not enough: the anchor wouldn't know, so always use gw rename-node.)

Names are sanitized to a DNS-safe form (ops@node01 → ops-node01) and must be unique. For a self-chosen name, uniqueness is checked at enrollment: a join whose (sanitized) name is already taken is refused. However, the token isn't immediately burned, so the joiner is told how many attempts remain and can retry with a different --hostname (a few tries per window). For a anchor-pinned name (gw invite --hostname), uniqueness is checked at invite instead, so a pinned name is guaranteed free before the token goes out and can't collide at enrollment (the joiner couldn't fix it anyway). Either way, a decommissioned node keeps its name until its nodes/<id>.json is removed on the anchor, which frees it for reuse.

TLS certificates for services

The same CA that gates the mesh also issues ordinary x509 TLS certificates, so a service on a node (Postgres, an internal API, etc) gets a cert that every peer validates against one trust root — no second PKI (public-key infrastructure). These are real x509 certs with SANs, distinct from the mesh credential, but signed by the same Ed25519 CA key.

What this is for (and isn't). WireGuard already encrypts and authenticates traffic between nodes, so TLS here is not about adding encryption — that part would be redundant. Its value is at the layers WireGuard doesn't cover:

• Service identity by name. WireGuard authenticates the node you reached, not that you reached the right node. The db.myfleet.internal→address mapping lives outside its crypto. A cert with SAN=db.myfleet.internal, validated by the client, is what proves "this endpoint is authorized for that name."
• Process/tenant identity. The mesh interface is host-global, so any process on a node can use the tunnel. mTLS (client certs) narrows a connection to a specific identity and surfaces it into the app (e.g. Postgres cert→role) for authz and audit.
• A free, mesh-rooted PKI. Services that require TLS anyway (sslmode=verify-full, HTTPS clients) get certs without you running a second CA.

The value requires the client to verify — use verify-full/mTLS. Using the cert only for opportunistic encryption (no SAN check) is just redundant with WireGuard.

A node may request certs only if its credential carries the tls capability. It's granted by the anchor, and is on by default ([anchor] default_caps = ["tls"]), so a plain gw invite already yields a cert-capable node — no extra flag:

TOKEN=$(sudo gw invite)                 # tls is in the default caps
sudo gw join "$TOKEN" --hostname dbnode

To make tls opt-in instead, set default_caps = [] in [anchor] (effective on the next invite) and grant it per-node with gw invite --caps tls or later with gw set-caps <node> …. Either way tls is bounded by SAN authorization (below), so a cert-capable node can still only get certs for its own names.

Then, on that node. A node can only get a cert for names it owns: its own <hostname>.<mesh_domain>, any subdomain of that, and its own overlay address. The anchor (the CA) enforces this, so a node can never obtain a valid cert for another node's name and impersonate its service to TLS clients.

# On node "dbnode" — postgres.dbnode.myfleet.internal is a subdomain it owns:
sudo gw cert-request --san postgres.dbnode.myfleet.internal --name postgres
#   → writes <data_dir>/tls/postgres.key, postgres.crt, and ca.crt, AND
#     registers the label so peers can resolve postgres.dbnode.myfleet.internal

# With no --san, the cert defaults to the node's own name + overlay address:
sudo gw cert-request                 # SAN = dbnode.myfleet.internal (and its addr)

# The three files need not share a directory — override any of them, e.g. put
# the key where the service expects it and the CA in the system trust store:
sudo gw cert-request --name postgres \
     --key-out  /etc/postgresql/ssl/postgres.key \
     --cert-out /etc/postgresql/ssl/postgres.crt \
     --ca-out   /usr/local/share/ca-certificates/mesh-ca.crt

gw cert-status                       # list issued certs and their expiry

Profiles — one command, files in the right place. Assembling the per-file flags (and getting the ownership right so the service can read its own key) is the fiddly part — and worse, plain --cert-out leaves files root:root, so auto-renewal months later rewrites them as root:root and silently breaks a service running as postgres. A profile fixes the whole lifecycle: a small TOML that says where each file goes, who owns it, and how to reload — and the daemon re-places and re-owns on every renewal, not just the first issue.

gw cert-profiles                              # list bundled templates
gw cert-request --profile postgres --show     # print one to copy + adapt
sudo gw cert-request --profile ./postgres.toml   # issue + place + register reload

A profile is a set of [[file]] entries (role = key/cert/ca/fullchain/ bundle, plus path, owner, mode) and a reload command. Bundled templates ship for postgres, nginx, haproxy, redis, nats, minio, mosquitto — they're starting points, not turnkey: each records the OS/software version it was written against, and a wrong path or missing service user fails loudly at request time rather than mis-placing a cert. Copy one, adapt the paths to your system, pass it in.

cert-request is idempotent: an unchanged re-request of a still-valid cert is a no-op (safe to run from config management), so a change (new SAN, edited profile path) is what triggers a re-issue; --renew forces one. The profile you pass is snapshotted to <data_dir>/tls/profiles/<name>.toml for record-keeping (the manifest already holds the effective config). gw cert-status lists everything the daemon manages, and gw cert-remove <name> stops managing one (keeping the placed files unless --delete-files).

The leaf private key is generated locally and never sent to the anchor; only its public key goes in the request, which is signed by the node's identity key. The anchor returns the leaf cert plus the CA cert. Point the service at them — e.g. Postgres ssl_cert_file=postgres.crt, ssl_key_file=postgres.key, and clients sslrootcert=ca.crt with sslmode=verify-full. Certs are short-lived (default 7 days, [anchor] tls_cert_ttl), and the daemon auto-renews each one at ~half its TTL into whatever paths you chose — pass --reload-cmd "systemctl reload postgresql" so the service picks up the rotation (or --no-auto-renew for a one-shot). Managed certs are keyed by --name, so re-running cert-request with the same name relocates it (the daemon renews into the new paths and flags the old files as orphaned) rather than leaving a duplicate. See RUNBOOK.md. Revocation is passive — stop renewing and it expires.

Subdomain names resolve too. A cert for postgres.dbnode.myfleet.internal is only useful if clients can resolve that name — so when a --san is a subdomain of the node's own mesh name, cert-request also publishes it: it adds the label (postgres) to [network] aliases, and the daemon advertises postgres.dbnode.myfleet.internal → <dbnode's address> into every node's /etc/hosts block (restart the daemon, or wait for the next renewal, to propagate). Aliases travel as bare labels in the (self-signed) NodeRecord and every reader expands them under the record's CA-attested mesh name — so a node can only ever publish names inside its own namespace, pointing at its own address; it can't name or hijack anything else. You can also set aliases = ["pg", "metrics"] directly in [network] without a cert.

Where things live — three files, don't conflate them:

File	Role	Location
the leaf key/cert + CA cert	what your service reads	wherever you point them (--key-out/--cert-out/--ca-out, else <data_dir>/tls/); the three need not share a directory
greasewood.toml	the daemon config; cert-request only reads it (for data_dir + the default SAN) and never writes it	wherever you pass gw -c … (default: the discovered /etc/greasewood_<name>.toml)
<data_dir>/tls/managed.json	the renewal source of truth — records each managed cert's three paths; the daemon reads it to know where to re-issue	pinned to data_dir (there's no separate flag; move data_dir in the config and it follows)

So the file that actually "controls" where renewed certs land is the manifest, not greasewood.toml: the TOML only locates the manifest (via data_dir), and the manifest holds the per-cert paths. cert-request prints both so you always know which config it read and where the renewal record is.

SANs are constrained to what the node owns (its CA-registered <hostname>.<mesh_domain>, subdomains, and its overlay address) — the anchor refuses a cert for another node's name, so a tls-capable node can't impersonate a service it doesn't run. The tls capability is still the gate; grant it only to nodes that run services. The anchor's CA cert is also at GET /ca-cert. (After a re-root the CA changes; re-request to pick up the new issuer.)

Worked example: mutual TLS for Postgres

This wires up a Postgres server that authenticates clients by their mesh identity, with certs that rotate transparently. Nothing below hardcodes a hostname: greasewood binds the cert's CN and SAN to the node's own attested <hostname>.<mesh_domain> automatically — you can't set them to another identity (the anchor refuses a SAN the node doesn't own and forces the CN to the node's own name), so each host gets a cert for exactly its own identity with no name typed.

The one thing to understand about CN. greasewood makes the CN attested, not cosmetic, and it's the mesh FQDN — the same on the server and client cert:

• Server cert: clients validate SAN = DNS:<db-host>.<mesh_domain> under sslmode=verify-full. Connecting by overlay address works too — the node's own address is a SAN by default.
• Client cert: the CN is the identity Postgres maps to a role. It is the connecting node's <hostname>.<mesh_domain> (e.g. nats01.myfleet.internal), so that FQDN — not a bare label — is what your pg_ident.conf map keys on.

On the database host. Point the three files at fixed, host-agnostic paths (the Debian ssl-cert group layout, which satisfies Postgres's key-permission check with a root-owned key):

sudo gw cert-request --name pg-server \
  --key-out  /etc/ssl/private/gw-postgres.key \
  --cert-out /etc/ssl/certs/gw-postgres.crt \
  --ca-out   /etc/ssl/certs/gw-myfleet-ca.crt \
  --reload-cmd /usr/local/sbin/gw-pg-reload
# CN + SAN default to THIS node's mesh name — no --san needed.
# --name is just the manifest key (so cert-status is readable and a re-request
# relocates in place); it does NOT affect the cert's identity.

One-time ownership so postgres can read a root-owned key (needs a restart):

sudo adduser postgres ssl-cert
sudo chgrp ssl-cert /etc/ssl/private/gw-postgres.key
sudo chmod 640      /etc/ssl/private/gw-postgres.key
sudo systemctl restart postgresql

postgresql.conf — set once, never touched again:

ssl = on
ssl_cert_file = '/etc/ssl/certs/gw-postgres.crt'
ssl_key_file  = '/etc/ssl/private/gw-postgres.key'
ssl_ca_file   = '/etc/ssl/certs/gw-myfleet-ca.crt'   # verifies client certs

Why rotation just works. greasewood renews in place — it truncates and rewrites each file at its path and never re-chmods an existing one — so the ownership and mode you set that first time are preserved on every rotation. Postgres doesn't watch the files; it re-reads them on SIGHUP. So the --reload-cmd only needs to reload (a restart is unnecessary and drops connections). Make it a script that asserts the key perms first, as a guard against a botched change:

#!/bin/sh
# /usr/local/sbin/gw-pg-reload   (chmod 0755, root-owned)
set -e
key=/etc/ssl/private/gw-postgres.key
test "$(stat -c '%U:%G %a' "$key")" = "root:ssl-cert 640" || {
  echo "gw-pg-reload: $key is $(stat -c '%U:%G %a' "$key"), want root:ssl-cert 640" >&2
  exit 1   # refuse to reload with wrong perms rather than break TLS
}
exec systemctl reload postgresql

On each client node. Request its own cert (again, identity is automatic) and point libpq at it:

sudo gw cert-request --name pg-client \
  --key-out  /etc/gw/pg-client.key \
  --cert-out /etc/gw/pg-client.crt \
  --ca-out   /etc/gw/gw-myfleet-ca.crt
# connect: sslmode=verify-full sslrootcert=/etc/gw/gw-myfleet-ca.crt \
#          sslcert=/etc/gw/pg-client.crt sslkey=/etc/gw/pg-client.key

Map identities to roles on the server. pg_hba.conf:

hostssl all all ::/0 cert map=mesh clientcert=verify-full

pg_ident.conf — the map keys on each client's mesh FQDN (its automatic CN):

# MAPNAME   CERT CN (= client's <hostname>.<mesh_domain>)   PG ROLE
mesh        nats01.mymesh.internal                              nats
mesh        chat01.mymesh.internal                              chat

This is the only place client hostnames appear, it's the allow-list of which identities may connect, and each entry is that node's own automatic name.

CA rotation. The ssl_ca_file matters only because of client-cert auth. A re-root changes the CA, and both the server's CA file and every client cert re-issue under the new CA on their next renewal, so rotate the CA (re-root) and let the fleet re-issue together; don't swap a CA independently, or client certs signed by the old one stop validating.

Worked example: a shared directory over the mesh (NFS)

greasewood doesn't ship a file-sharing feature, on purpose — it provides the substrate (identity, addresses, names, per-port access control) and the operator owns the service, same as with Postgres above. But the substrate does almost all of the work here, so the recipe is short. What you get: any granted node mounts files01.mymesh.internal:/srv/workspace and the mesh supplies the encryption, the peer authentication, and the per-role firewall that NFS setups usually bolt on by hand (Kerberos, per-host exports, firewall rules per pair).

The one thing to understand: the grant table is the share ACL. NFSv4.1+ speaks on a single port (tcp/2049 — no rpcbind/portmapper dance), and the port filter already decides, per role, who can reach that port over addresses that can't be spoofed (cryptokey routing pins address ↔ key). So the NFS config below is static — written once, never edited as the fleet changes. Export to the whole overlay /64 and let grants do the gating: adding a node to role:worker grants it the share; revoking a node revokes it, on the same reconcile cycle that tears down its tunnel. No per-client exports lines, ever.

On the file server (say files01, holding role:files) — install the server, make it NFSv4-only, and export to the overlay prefix (yours is overlay_prefix in the config; the default is shown):

sudo apt install nfs-kernel-server        # Debian/Ubuntu; use your pkg mgr

# NFSv4-only: v3 off means no rpcbind, no mountd port — 2049 is the whole surface
printf '[nfsd]\nvers3=n\n' | sudo tee /etc/nfs.conf.d/greasewood.conf
sudo systemctl mask --now rpcbind.socket rpcbind.service

# a dedicated owner for the workspace tree
sudo adduser --system --group --uid 1500 workspace
sudo mkdir -p /srv/workspace && sudo chown workspace:workspace /srv/workspace

/etc/exports — one line, scoped to the mesh, and it never changes again:

/srv/workspace fd8d:e5c1:db1a:7::/64(rw,sync,no_subtree_check,all_squash,anonuid=1500,anongid=1500)

sudo exportfs -ra && sudo systemctl enable --now nfs-server

Why all_squash. WireGuard authenticates hosts, not users: over sec=sys (plain NFS auth), root on any granted node can claim any UID. For a shared workspace, don't fight that — squash every client to the workspace user, so the share behaves like a common drop-box and UID games are moot. If you instead need per-user ownership across nodes, you need matched UIDs fleet-wide and root_squash, and you should know you're trusting every granted host's root — that's inherent to NFS-without-Kerberos, not to the mesh.

On the anchor — open the one port, with the usual preview:

# append to <data_dir>/grants.toml
[[grant]]
from  = ["worker"]
to    = ["files"]
ports = ["tcp/2049"]
# why: workers mount the shared workspace; nothing else reaches NFS.

sudo gw policy apply     # shows the grant diff + tunnel delta, asks to confirm

On each client (any role:worker node) — one fstab line, using the mesh name (from the managed hosts block, so it resolves anchor-down too):

# /etc/fstab
files01.mymesh.internal:/srv/workspace  /mnt/workspace  nfs4  vers=4.2,_netdev,noauto,x-systemd.automount,x-systemd.mount-timeout=30s,x-systemd.requires=[e-mail]  0  0

sudo mkdir -p /mnt/workspace && sudo systemctl daemon-reload
sudo systemctl start mnt-workspace.automount
ls /mnt/workspace        # first touch mounts it through the tunnel

The automount is doing real work in that line: the path mounts on first access instead of at boot (so boot never blocks on the mesh), and x-systemd.requires= pins the ordering to this mesh's daemon.

Honesty about failure. NFS is the one place a mesh outage stops looking like direct-or-fail: a hard mount (the default above) never errors — processes touching the path when the server is unreachable block in uninterruptible sleep until it returns. That's the correct default for a read-write workspace (soft can corrupt writes that were in flight), but know the trade. If a share is read-mostly and you'd rather get I/O errors than hangs, add soft,timeo=100,retrans=3 to the options and accept the risk; either way the mount-timeout above already keeps a new access from hanging a service forever when the tunnel is down.

The underlay is your job, as always. greasewood never touches your main firewall, and nfsd listens on all addresses — so whether tcp/2049 is reachable from the underlay is decided by your own host firewall, exactly like SSH. Two things protect you meanwhile: the exports line admits only the overlay /64 (an underlay client is refused the mount), and the port filter admits only granted roles within the mesh. But scope your underlay firewall as you would for any service.

What about NFS over TLS? Linux 6.4+ can run NFS inside TLS (RPC-with-TLS, RFC 9289): xprtsec=mtls on the mount, xprtsec=mtls in the exports line, and the tlshd daemon (ktls-utils) on both ends — and gw cert-request can issue the certs, so the mesh CA drives it for free. Inside the mesh, skip it: it duplicates what WireGuard already provides (the tunnel encrypts; the address pins the host), costs a second encryption pass, adds a daemon whose failure mode is "mounts stop working" — and it does not fix the sec=sys caveat above, because certs authenticate the machine while UIDs stay client-asserted, so the all_squash guidance stands unchanged. It earns its keep only when a share must also be reachable from hosts outside the mesh, where there's no tunnel underneath — there, mTLS with mesh-issued certs is exactly the right tool.

Moving the anchor (re-root)

No node is forever-critical: because a node trusts a CA key, not a machine, any node that holds the CA authority and serves the directory is the anchor. Moving the anchor is therefore a deliberate re-root — swap which CA key the fleet trusts — not an automatic handover. trusted_pubs is a set, so you trust the old and new CA during an overlap and the move is non-disruptive.

The CA private key never moves: the new anchor generates its own key, and you push the new public key into every node's trusted_pubs. Migrating from anchor A to a new node B:

1. Enroll B as an ordinary node (gw join …) and start it — so it's a reachable mesh member every node can renew against over the overlay.
2. Promote B: sudo gw anchor-promote generates B's own CA key and flips it to role=anchor (keeping trust in A), printing B's CA pubkey + control endpoint; then sudo gw run, and B serves the control plane on its own overlay address.
3. On every node (Ansible): add B's CA pubkey to [ca] trusted_pubs (keep A's) and repoint root_url + seeds to B; restart the daemon. The fleet now trusts A and B — this is the overlap.
4. Nodes renew under B. B never enrolled them, but re-issues from each node's still-trusted directory record (hostname/caps are CA-attested) — nothing to copy. sudo gw renew-all on B pulls the fleet over promptly. Re-apply any gw revoke on B first (a fresh CA doesn't inherit A's revoke list).
5. Once every node holds a B-signed credential, drop A's CA pubkey from trusted_pubs fleet-wide, then decommission A.

Throughout, A and B both run as anchors — that's the point of the overlap, and there's no clash: each anchor's control plane binds to its own overlay address (the hash of its own identity key), the fleet trusts both CAs at once, and the directory is eventually-consistent (records merge by highest sequence number, and each node renews against the one anchor its root_url points at). Existing tunnels stay up throughout (the data plane never depends on the anchor), so the handover is non-disruptive. Plan the overlap to last at least one credential TTL so every node renews under B in time. See RUNBOOK.md for the full graceful vs emergency (compromised/lost-key) procedures.

Testing

Unit tests run in ~30s with no privileges (pip install -e '.[test]' && python -m pytest). The Podman-based integration/stress suite and the nightly Hypothesis "deep" tier are documented in TESTING.md.

Design notes & non-goals

The non-goals — routing/relays, NAT traversal, IPv4 overlay, cross-platform — aren't missing, they're the point. A few internal ideas are deferred rather than overlooked — named here with the trigger that would make them worth building:

• Gossip between nodes — if the network ever genuinely partitions (today every node pulls the directory from the anchor).
• Lazy, on-demand tunnels — at hundreds of nodes, if a full peer mesh becomes too many links to hold open.
• Threshold CA — if single-anchor-key compromise becomes unacceptable.
• CA cross-signing to smooth re-root — let the old CA sign a short-lived, directory-distributed "also trust the new CA" delegation, so a graceful re-root doesn't require pushing the new key into every node's trusted_pubs up front (the config edit becomes a calm, batchable follow-up instead of a race against credential expiry). Trigger: re-root friction in practice. Would be opt-in, short-lived, and logged, since it loosens the config-only trust root; emergency re-root (old CA lost) still needs the out-of-band config push.

Clock integrity is part of the security posture. Every allow/deny is a timestamp comparison against a credential expiry, so run NTP/chrony on every node.

CA trust is a set, not a single key. The CA (and anchor) is moved by a re-root — trust the new key alongside the old during an overlap, then drop the old — don't move the private key to a new machine. See Moving the anchor.

Security & operations

• SECURITY.md — trust boundaries, what the 7-step check enforces, accepted risks, and the results of the security review.
• RUNBOOK.md — disaster SOPs: compromised node, lost/leaked CA key, destroyed anchor, fleet-wide teardown, and how to read gw diagnose.

AI Disclaimer

Greasewood is a greasy project, and was built with the assistance of SWE LLM agents.