vhotseat

Want your local LLM agent to drive ComfyUI on the same consumer graphics card? Without coordination, the first video render lands on top of the loaded LLM, and something runs out of memory. Boom, OOM!

vhotseat makes Strata and ComfyUI take turns on VRAM and RAM: when the agent queues a render, the LLM is unloaded; when it asks the LLM its next question, ComfyUI is freed. Neither ever loads on top of the other, and running work is never interrupted: not an answer the LLM is writing, not a render in progress.

v-hotseat: VRAM hotseat. Hot-seat multiplayer: one device, players take turns, nobody is kicked out mid-turn.

Tested on:

With this setup, pi drove long-running video workflows unattended, over several rounds of render → review → render, overnight.

Status: Linux, NVIDIA. Windows is designed for and planned. MIT licence.

→ Quick start for this exact setup: a step-by-step guide that you, or your coding agent, can follow. Or just prompt your coding agent to set up the whole workflow:

Set up vhotseat so my local LLM agent can drive ComfyUI on this GPU without running out of memory. Follow https://github.com/wandermyz/vhotseat/blob/main/docs/guides/quickstart-strata-pi-comfyui.md, adapting it to the servers and paths on this machine, and run each step's check before going on.

How it works

The servers stay as they are, and every client is covered: the agent, its scripts, a web UI, curl. Nobody calls vhotseat; each workload asks for the GPU through a gate right before it loads, and vhotseat makes room by releasing the others:

• The LLM runs vhotseat gate from its pre-load hook. While ComfyUI has work queued the load is refused (a 503 the agent's harness retries).
• ComfyUI runs vhotseat switch before every job through a small custom node, the guard. It waits for any answer the LLM is writing, then unloads it.
• Admission checks there's enough free VRAM and RAM before a load, and restarts an idle ComfyUI if its leftover RAM is what's in the way.
• comfy-render lets the agent run a batch of workflows in one call, so a batch costs two handovers instead of two per job.

Strata and ComfyUI are just the built-in examples: anything that uses the GPU (another LLM server, a TTS server, a training run) can join through a config entry or a small driver, with a hook, plugin, proxy or hold as its gate.

How it differs. llama-swap and Ollama's scheduler swap models between LLM servers they control; they don't take turns with a non-LLM app like ComfyUI, and don't wait for a running job. vhotseat arbitrates between different servers, of any kind, and can sit on top of either (they can be one of its workloads).

Install

uv tool install vhotseat      # or: pipx install vhotseat

That installs two commands, vhotseat and comfy-render, with one dependency (PyYAML). From a checkout, ./vhotseat and ./comfy-render run as they are; symlink them into ~/.local/bin.

Then set it up for your servers: follow the quick start, or let a coding agent do it with the vhotseat-setup skill. config.example.yaml is commented key by key, and examples/ has configs for other servers.

vhotseat doctor     # check the config, the GPU, every workload and gate
vhotseat status     # who owns the GPU and what each workload holds
vhotseat web        # dashboard: RAM and VRAM over time, handovers, events, logs

Documentation

• Quick start: Strata + pi + ComfyUI on one 16 GB card, step by step.
• How it works: a full agent → ComfyUI → agent flow, the races and deadlocks, and how often the GPU changes hands.
• Reference: commands, gates, admission, the ComfyUI guard, comfy-render, the dashboard and telemetry.
• Agent kit: comfy-render and Agent Skills for the agent that renders, and for the coding agent that sets vhotseat up and keeps it working.
• Writing a driver for another server, and building a proxy gate for one with no pre-load hook.
• After a ComfyUI update: checking the guard still works. Fixing a VRAM leak in a custom node is a worked example.

Development

uv run --group dev pytest       # ~2 min; fake LLM and ComfyUI servers, no GPU needed
uvx ruff check .

The tests run the real CLI and arbiter against fake servers over HTTP on localhost; the config they use is config.example.yaml, and every file in examples/ is validated. CI runs both on Python 3.10 to 3.13.

Licence

MIT. See LICENSE.