Skip to main content

HTTP-controlled NBD-export multiplexer for a small lab (boot Linux images over the network with overlayfs+tmpfs writes)

Project description

nbdmux

HTTP-controlled NBD-export multiplexer for a small lab. Register local disk-image files as named NBD exports over an HTTP control plane; nbdmux keeps an nbd-server subprocess alive that serves all registered exports on a single TCP port. Targets nbd-client against that port from an initramfs and boot the image with overlayfs over tmpfs for writes (see bty's ramboot boot mode for the canonical consumer).

Designed as a peer to withcache: small lab, single sidecar container, no third-party Python deps. Operationally:

[ bty-web ] --HTTP--> [ nbdmux ]  --supervises-->  [ nbd-server ]
                          |                              |
                          |                            TCP 10809
                          |                              |
                          v                              v
                     SQLite state                  [ target's
                     (exports table)                nbd-client ]

Components

Path What it is
src/nbdmux/server.py The daemon. HTTP control plane + nbd-server subprocess management + operator UI (Bootstrap 5 + Bootstrap Icons + HTMX, matches bty's chrome)
src/nbdmux/client.py Stdlib-only Python client library for other tools
deploy/Containerfile Single-image deploy (Python + nbd-server)
deploy/compose.yml Reference compose stack

System dependency

nbdmux runs nbd-server (from the classical nbd project) as a subprocess. Install at the OS level:

# Debian / Ubuntu
sudo apt install nbd-server

# Fedora
sudo dnf install nbd

The container deploy bundles it. Also make sure the nbd kernel module + nbd-client are available on the consuming Linux box (the target you're booting); they're in the same nbd package.

Install

pipx install nbdmux            # or: uv tool install nbdmux

Run the daemon (development; the container deploy is the recommended production path):

nbdmux-server --data-dir ./data --port 8082 --nbd-port 10809

Register an image. Two body shapes are accepted:

Pre-warmed -- point at a file the operator has already placed on disk:

curl -X POST http://localhost:8082/exports \
     -H 'Content-Type: application/json' \
     -d '{"name": "debian-sysdev", "file": "/path/to/debian-sysdev.img", "readonly": true}'

Warm via withcache -- nbdmux fetches src_url through the configured withcache, decompresses on the fly (gzip / zstd / xz), and lands the raw .img under <images-dir>/<name>.img. Requires NBDMUX_WITHCACHE_URL set on the daemon:

curl -X POST http://localhost:8082/exports \
     -H 'Content-Type: application/json' \
     -d '{"name": "debian-sysdev", "src_url": "https://catalog/debian-sysdev.img.zst", "readonly": true}'

Then on a target Linux box:

modprobe nbd
nbd-client <nbdmux-host> 10809 -name debian-sysdev /dev/nbd0
fdisk -l /dev/nbd0   # the .img's partition table

HTTP control plane

Method Path Body Returns
GET /exports - array of exports
POST /exports {name, file, readonly?: bool} (pre-warmed) OR {name, src_url} (warm via withcache) the new export
DELETE /exports/{name} - 204 (warm-created also unlinks the .img)
POST /admin/create_export form-encoded name=...&src_url=... 303 to / (dashboard)
GET /healthz - ok
GET / - operator dashboard

POST /admin/create_export is what the operator UI's New Export subnav form submits to; it's the form-encoded counterpart of the JSON POST /exports {name, src_url} warm path. Validation failures 303 back to /?err=<kind> and the dashboard renders an alert banner with a friendly reason.

Operator UI

The dashboard at http://<host>:8082/ is a one-page view of the nbd-server process, all registered exports, and (top-right of the sub-navigation strip) a New Export form. It uses Bootstrap 5 + Bootstrap Icons + HTMX bundled offline; the same chrome as bty and withcache, only the primary hue differs (magenta -- the terminus of the trio's navy -> dark-magenta -> magenta gradient) so operators tell the three consoles apart at a glance.

Login is a signed session cookie gated on NBDMUX_ADMIN_PASSWORD (same pattern as withcache). With no password set, the UI is open and the daemon logs a startup warning.

Auth

Single-tenant, server-signed cookie -- same pattern as withcache. Set NBDMUX_ADMIN_PASSWORD to gate the operator UI + the HTTP control plane. Unset = open with a startup warning.

The NBD port itself is unauthenticated (nbd-server's classical model); LAN-only assumption, firewall is the operator's responsibility.

License

BSD-3-Clause.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nbdmux-0.2.13.tar.gz (419.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

nbdmux-0.2.13-py3-none-any.whl (405.5 kB view details)

Uploaded Python 3

File details

Details for the file nbdmux-0.2.13.tar.gz.

File metadata

  • Download URL: nbdmux-0.2.13.tar.gz
  • Upload date:
  • Size: 419.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for nbdmux-0.2.13.tar.gz
Algorithm Hash digest
SHA256 5a632a3455231f36035326d748248249d95cf61f5f7606214f1482d0af6cb85f
MD5 e638c2637c5ad280c8fc8f70a7c25686
BLAKE2b-256 ceab240270fb3fccb4c2644217292a1363320fb0fed760be5ae3079267b53ca6

See more details on using hashes here.

Provenance

The following attestation bundles were made for nbdmux-0.2.13.tar.gz:

Publisher: ci-cd.yml on safl/nbdmux

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nbdmux-0.2.13-py3-none-any.whl.

File metadata

  • Download URL: nbdmux-0.2.13-py3-none-any.whl
  • Upload date:
  • Size: 405.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for nbdmux-0.2.13-py3-none-any.whl
Algorithm Hash digest
SHA256 5c81996377847f187633c49c5b138838d3e2f5c8fd8d3fca519f9501bcdd0d5a
MD5 eaf4961a684a4cc381e24a35e59859a3
BLAKE2b-256 70fd4bd1906de81e74fed32d3d8941a3ac9a48f0a0088832d2bfeac8206a1d3f

See more details on using hashes here.

Provenance

The following attestation bundles were made for nbdmux-0.2.13-py3-none-any.whl:

Publisher: ci-cd.yml on safl/nbdmux

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page