pyinfra-lxd-api-connector

A pyinfra connector that targets LXD containers via the LXD HTTPS API directly — no SSH hop, no paramiko, no lxc CLI subprocess, no websockets. One kept-alive HTTPS connection per host, exec via the container_exec_recording API extension, file transfers via the native files API.

Why?

The obvious way to drive pyinfra against LXD is to shell out to the lxc CLI for every command. That approach pays a per-command cost of ~6–7 fresh TCP+TLS connections (capabilities probe + events websocket + exec POST + 4× stdio websockets + operation poll) — measured at ~870 ms per command over Tailscale from a remote laptop.

Talking to the API directly with record-output: true mode collapses all of that to a single kept-alive HTTPS connection with zero websockets. Measured at ~150 ms per run_shell_command from the same vantage point — ~5–6× faster, and within the same order of magnitude as warm SSH-multiplex.

Tracks pyinfra issue #677. Per pyinfra's contributing guide, connectors live as separate packages rather than in the pyinfra core repo.

Performance

Per-call latency over Tailscale from a remote laptop (~27 ms RTT to the cluster), measured via smoke_test.py against a real container:

For comparison, an lxc exec-based connector pays ~870 ms per run_shell_command from the same vantage point. From a node inside the cluster the difference doesn't matter; from a laptop driving deploys over a WAN it dominates wall time.

Install

uv tool install pyinfra --with pyinfra-lxd-api-connector

Usage

Prereq: an lxc remote configured locally:

lxc remote add mycluster https://your-cluster:8443 --token <token>
lxc list mycluster:        # verify

The connector reads the standard LXD client config at ~/.config/lxc/:

• config.yml — remote URL
• client.crt + client.key — mTLS client identity
• servercerts/<remote>.crt — pinned server cert

Inventory:

hosts = [
    "@lxd_api/mycluster:php01",       # explicit remote
    "@lxd_api/web1",                   # uses default-remote from lxc config
    "@lxd_api/some-other-cluster:web1",
]

A bare @lxd_api/<container> resolves the remote via the default-remote field in ~/.config/lxc/config.yml — the same field lxc itself consults when called without a remote qualifier. Switch the default with lxc remote switch <name>. If no default is set, the connector raises an InventoryError pointing you at the qualified form.

Requirements

• LXD server with the container_exec_recording API extension (LXD 5.0+).
• Local LXD client config at ~/.config/lxc/. lxc remote add sets all of this up.

Status

Beta. In production use against a 32-container LXD cluster since 2026-04-28. Feedback / bug reports welcome.

Known limitations

• No interactive / PTY support — the connector raises NotImplementedError if _get_pty=True. pyinfra never needs PTY for facts/operations, so this is fine in practice; if you need an interactive shell, use lxc shell directly.
• Per-command stdout/stderr is buffered, not streamed — record-output mode means output arrives at the end of the command. For pyinfra's typical workload (facts and one-shot operations) this is invisible; for long-running commands you won't see live progress.
• POST /exec retries only connect-class errors by default — run_shell_command retries ConnectError / ConnectTimeout / PoolTimeout (the request provably never reached the server), but fails fast on ReadTimeout / 5xx, since the command may already be running on the container. File transfers (put_file / get_file) and the operation/output polls retry on any transient error. Set lxd_exec_retry_on_read_errors=True on a host (or globally) if all your deploy steps are independently idempotent and duplicate execution is safe.

AI assistance

Per the pyinfra AI usage policy, disclosing how this package was authored:

The initial draft of pyinfra_lxd_api_connector.py was generated by Claude (Anthropic) in collaboration with the maintainer (Christian Rishøj). Specifically:

• Christian identified the original problem — a silent SFTP-truncation bug in an earlier lxc exec-based driver — and ran the empirical analysis showing that the LXD record-output: true API path was the right fast alternative.
• Claude drafted the connector module against pyinfra's BaseConnector interface.
• Christian reviewed every line, integrated and ran it against a 32-container production cluster, and iterated through several rounds of correctness, latency, and ergonomics fixes.
• All subsequent maintenance is human-driven.

The code in this repository is fully understood and reviewed by the maintainer; AI assistance is a drafting tool, not a substitute for human judgment.

License

MIT — see LICENSE.