vupai 0.9.0

Voice UI for AI panes - push-to-talk voice control over tmux agent panes

Voice UI for AI panes: push-to-talk voice control for your tmux agent panes, on macOS, with on-device speech.

vupai (say "voo-pie") is a Voice UI for your AI panes.

Hold a key, speak, and what you say is typed into the right tmux pane: the one you're looking at, or an agent you call by name ("atlas, run the tests"). Speech-to-text runs on-device with NVIDIA Parakeet (via Apple MLX): no cloud, no API keys.

Built for a tmux-centric workflow where you keep several coding agents and shells open at once and want to drive them by voice without reaching for the mouse. New panes launch an agent by default (claude out of the box) and should work with other agentic coding tools (Codex, Gemini, …), though testing so far has focused on Claude Code.

📚 Full documentation: itsjrsa.github.io/vupai · 🎬 demo clips

Install

vupai is macOS Apple-Silicon only (it depends on Apple MLX for on-device speech). It needs tmux, sox, and uv:

brew install tmux sox uv
uv tool install vupai
vupai setup        # one-time bootstrap: mic, hotkeys, model download, macOS permissions

The Parakeet model (~2 GB) downloads automatically on first use. Full details (requirements, from-source install, permissions) are in the install and setup docs.

Quickstart

vupai              # attach-or-create the session named after the cwd

Then hold a push-to-talk key and speak:

• Right-Option (dictation): what you say is typed into the focused pane. Start with a pane's name ("atlas, run the tests") to address it anywhere.
• Right-Command (system): voice commands that act on the panes instead of typing: "create 3 panes", "focus atlas", "board", "note: fix the retry logic", "everyone, pull main".

It's still tmux: every normal binding keeps working, and vupai runs on its own tmux server so your existing setup is untouched.

Learn more

The documentation site covers everything:

• Usage · Voice commands · Silent mode (no mic)
• Multi-agent supervision: board · activity ledger · review · task pile · squad
• Reference: configuration · SSH hosts · tmux tips · uninstall · roadmap

Contributing? See AGENTS.md. Ideas and PRs are welcome: open an issue.

License

MIT. (Note: pynput is LGPL-3.0 and the Parakeet model weights are CC-BY-4.0; both are runtime dependencies, not part of this repo's code.)