mavor — local, low-latency voice dictation

Tap a hotkey, talk, tap again: the words are transcribed on your own machine and typed into whatever window has focus. Your voice never leaves the box — no cloud API, no account, nothing to sign up for. Optional additional copying is controlled by output.clipboard (off by default), and a small "● Recording" pill sits at the top of the screen, clear of your bar, showing a live waveform and a running preview of the words while you speak. The preview is never what gets typed — the text you keep is transcribed once, when you let go.

$mod + ` ──▶  ● Recording   ▂▃▅▆          (HUD overlay with live audio meter)
              talking talking talking
$mod + ` ──▶  ⟳ Transcribing  ● ● ●        (in-process CGO / whisper.cpp)
              ──▶ wtype + wl-copy         (text lands in focused app)
              overlay closes

CLI subcommands:

• mavor setup — one-shot first run: scaffold the config, install missing runtime tools, and download every model the config names.
• mavor daemon — long-lived process. Owns the overlay, audio capture, speech-to-text, PipeWire audio ducking, and a Unix-socket IPC server.
• mavor start / mavor stop — push-to-talk keybind controls (hold to speak).
• mavor toggle — toggle mode control (press once to start, press again to stop).
• mavor doctor — self-diagnostic health check for Wayland, audio, and tools.
• mavor config init — scaffold ~/.config/mavor/config.toml with documented defaults.
• mavor service install — install and enable systemd user service (mavor.service).

GNOME Wayland: copy, then paste

Set this in your config and restart the daemon:

[output]
driver = "clipboard"
clipboard_backend = "x11"

This always replaces CLIPBOARD, even with output.clipboard = false, and never injects keys or modifies PRIMARY. Install xclip and supply the desktop session's DISPLAY/XAUTHORITY environment; wtype, wl-copy, and wl-paste are not required for this backend. The default remains paste. In GNOME Settings → Keyboard → Custom Shortcuts (labels vary), bind the absolute installed path followed by toggle, for example /home/you/.local/bin/mavor toggle. Press once to record, again to transcribe; then paste normally (Ctrl+V in editors, usually Ctrl+Shift+V in terminals).

GNOME automatically selects the passive XWayland HUD: the same pill, waveform and preview painter as Sway, without a Shell extension. XWayland, the current session DISPLAY/XAUTHORITY, D-Bus and XDG_CURRENT_DESKTOP=GNOME must reach the daemon. mavor doctor separately probes the HUD; constructor failure logs a no-overlay fallback without disabling transcription/copy. Use mavor status and mavor logs; Idle means the cycle ended, not that copying succeeded. Notifications are not implemented. See the manual-paste guide for clipboard limitations and verification, and HUD QA for presentation and geometry limits. Isolated GNOME Shell/Mutter 50.4 passes native GTK transfer/Paste and continuous-focus checks with xclip 0.13; complete desktop lifecycle is not certified. Wayland clipboard output remains the default backend and its wl-copy fallback is not focus-safe on measured GNOME.

Why this exists

Dictation on Linux tends to arrive as an appliance: one vendor's model, one opinionated UI, a tray icon, and a service that wants to own your microphone. Meanwhile the interesting work is happening in the models — whisper.cpp, Parakeet, Zipformer, Moonshine, SenseVoice, Paraformer, a new one every few months, each with its own build, its own model layout, and its own idea of what an API is.

mavor is the boring layer under that. Three goals, in order:

One interface in front of every model. A single Transcriber contract with a supervised warm whisper-server, a one-shot whisper-cli, and in-process sherpa-onnx behind it. You do not choose between them: switching models is the one line model = "…" in a config file, and the model decides the rest — a whisper model runs on whisper.cpp, everything else on ONNX Runtime through sherpa-onnx. mavor models list shows the whole catalog — size, languages, whether it streams — and mavor models pull puts it where mavor will find it. Trying a new model should cost a minute, not an afternoon.

Everything runs on your machine. Your voice never leaves it. Transcription is whisper.cpp or sherpa-onnx running locally against a model on your own disk — there is no cloud API behind it, no account, no API key, and nothing to sign up for. The only network call in the program is mavor models pull, which you invoke, to fetch a model from Hugging Face or a GitHub release. Whisper models do go over HTTP, and it is loopback: mavor starts and supervises its own whisper-server child to keep the model warm. advanced.server can point that at a server you run yourself instead — still yours, still not a vendor. No telemetry, no analytics, no crash reporting. Unplug the network after mavor setup and dictation still works.

Minimal and unintrusive. No tray icon, no window, no background service listening for a wake word. A daemon that idles until you press a key, a floating pill that appears while you speak and disappears when you stop, and text in the window you were already typing in. It holds the microphone only between start and stop, and the one piece of UI it draws is deliberately the smallest thing that still tells you it is listening.

A good citizen of a Wayland tiling session. Built for compositors rather than in spite of them: a wlr-layer-shell overlay that floats clear of your bar instead of stealing focus or spawning a window your tiler has to place, wtype for input so text lands in the focused surface through the compositor's own protocols, and PipeWire ducking so whatever you were listening to gets out of the way while you talk. It is driven entirely by keybinds and a Unix socket, which is what makes it scriptable rather than clickable.

Install

Go

go install github.com/mschulkind-oss/mavor/cmd/mavor@latest

Release binary

Each tagged release publishes a linux/amd64 tarball and a checksums.txt on the releases page. The tarball holds three files, and mavor needs the other two: it is dynamically linked against sherpa-onnx. Keep them together — either beside the binary, or one directory up in lib/, which is where the binary looks.

tar -xzf mavor_v0.1.0_linux_amd64.tar.gz
install -m 0755 mavor ~/.local/bin/mavor
install -m 0644 -D -t ~/.local/lib lib*.so     # ~/.local/bin/../lib

linux/arm64 is not published yet: mavor links sherpa-onnx through cgo, which the amd64 release builder cannot cross-compile. Build from source on the machine instead.

From source

git clone https://github.com/mschulkind-oss/mavor
cd mavor
mise install                 # gets the right toolchain
just install                 # binary to ~/.local/bin, libraries to ~/.local/lib

# Or deploy binary + install systemd user service in one step:
just deploy

Quick Start & Verification

The five-step version, with what each command prints and what to do when a step does not land, is docs/quickstart.md. The short form:

mavor setup      # config, missing tools, every model the config names, systemd unit
mavor doctor     # what this machine will actually do with that config

mavor setup makes the current config runnable: it downloads the main model and the streaming model the live preview runs alongside it — on a scaffolded config, whisper-base.en and zipformer-streaming — skips whatever is already in the cache, and is safe to re-run after you edit config.toml. The preview companion adds 161 MB resident, held for as long as the daemon runs; it is picked for how continuously it updates rather than for accuracy, because the main text remains authoritative on success. Set preview.source to zipformer-streaming-20m (112 MB) if you want that smaller still, or preview.enabled = false for no companion at all. mavor config init scaffolds the file on its own if you would rather start there.

For experimental recognition during capture, see the opt-in final modes. Native streaming and offline segment processing are different capabilities; after-stop remains the default; only finalized text emits after release. The paced observations include the quality drift observed with offline segments.

Model initialization and request failures

The daemon loads models quietly while serving status/hotkeys as initializing. Only start or toggle during loading reveals INITIALIZING, explaining that recording is unavailable until models are ready. The notice stays until readiness hides it or a genuine failure shows ERROR. Status and stop do not reveal the notice. Nothing queues recording: press start/toggle again after ready. Readiness includes actual inference on a short generated, non-private voiced fixture, with the result discarded and streams reset. Default budgets are 120 seconds for GPU server startup, 180 seconds overall and 30 seconds per probe. A native model call may outlive cancellation while cleanup waits safely for it. Startup failure shows the actual diagnostic and exits; no recording starts first.

Automatic main-model CPU recovery remains off (advanced.cpu_fallback = false). A failed supervised local GPU-enabled main request can instead use the already-running streaming companion's finalized text from the same recording, only when complete audio coverage and final drain succeed. The HUD first shows ERROR, then BACKUP TRANSCRIPT; status and history retain the actual source and warning. Provisional/stale previews never emit, and successful main results always win. Missing/empty/incomplete backup retains the original error. Startup, canceled/invalid, remote and CPU requests do not qualify. No fresh backup model is loaded and the failed main is not replayed on CPU by this path. See the failure and coverage contract and measured acceptance limits.

Compositor integration

Push-to-Talk Mode (Recommended)

# ~/.config/sway/config
exec mavor daemon
bindsym $mod+grave exec mavor start
bindsym --release $mod+grave exec mavor stop

Toggle Mode

# ~/.config/sway/config
exec mavor daemon
bindsym $mod+grave exec mavor toggle

On layer-shell desktops, the overlay is a surface on the top layer and does not request an exclusive zone, which means two things: it floats over your content without resizing windows, and the compositor places it inside the space other bars have reserved. overlay.top_margin is therefore a gap below Waybar, not an offset from the screen edge — a bar of any height, or no bar at all, works without configuring anything.

Recovering a transcript

Synthetic typing can be swallowed by a window that lost focus or an app that was still starting, and the transcript goes with it. Every completed transcription is appended to a log first, so it is always recoverable.

mavor history --pick is the whole recovery round trip in one command: it renders the log, hands it to a picker, and copies whatever you chose to the clipboard. Bind it to a key:

# ~/.config/sway/config
bindsym $mod+shift+grave exec mavor history --pick

Any dmenu-compatible picker works — rofi, wofi, fuzzel, dmenu, fzf — because the contract is only "read rows on stdin, echo the chosen one to stdout". It defaults to rofi -dmenu; set $MAVOR_PICKER once in your profile, or pass --picker:

export MAVOR_PICKER='fuzzel --dmenu'
mavor history --pick --picker 'wofi --dmenu'

Selection is resolved by the index in each row's first column, not by the text, so a picker that truncates or reformats what it displays still recovers the right transcript.

Picker rows are numbered and otherwise bare — a full RFC3339 timestamp is 25 columns of prefix that never helps you tell two transcripts apart. Add --timestamps if you want the column back.

Without a picker, the log is an ordinary listing you can pipe anywhere:

mavor history                     # 20 newest, timestamped
mavor history -n0 --timestamps=false # all of them, text only
mavor history --number            # prefixed with the index `history copy` takes
mavor history --json              # JSON Lines, for scripts
mavor history copy                # copy the newest — the one that just vanished
mavor history copy 3              # copy a specific entry

Each transcript is flattened to exactly one row, so a multi-line transcription stays one selectable entry.

Systemd User Service

Alternatively, run the daemon as a systemd user unit that starts automatically with your graphical session:

mavor service install --start
mavor service status

Configuration

$XDG_CONFIG_HOME/mavor/config.toml (defaults to ~/.config/mavor/config.toml). All paths support ~ and $ENVIRONMENT_VARIABLES. Run mavor config show to inspect the resolved values, and mavor config init to scaffold the commented file with every default in it.

One top-level key and six tables. The first line is the one a first-time user touches; everything below it has a working default, and deleting a line gets that default back.

model = "whisper-base.en"   # `mavor models list` shows every choice

[preview]
enabled = true              # live text in the overlay while you speak
source = "auto"             # "auto" | "phrases" | a model name

[ducking]
enabled = false             # lower other audio while recording
volume = "0%"               # "0%" mutes; "25%" merely lowers
# apps = ["spotify", "firefox"]

# Mute a networked mixer over OSC while recording — it reaches where pactl
# cannot. Independent of ducking.enabled above.
[ducking.osc]
enabled = false
# address = "192.168.1.6"           # the device's IP; make it static
# paths   = ["/ch/15/mix/on"]       # the parameters to mute (X Air spelling)

[overlay]
top_margin = 8              # px below the top of the usable area

[vocabulary]
# words = ["mavor", "wlroots", "Schulkind"]

# Chosen for you. Override only if `mavor doctor` gives you a reason to.
[advanced]
# threads = 6               # default: this machine's physical core count
# gpu = "auto"              # "auto" or "off"; whisper only

[paths]
# models = "~/.cache/mavor/models"

There is no engine key. The model decides its runtime, and where that runtime runs is derived too — a warm supervised whisper-server for whisper models, in-process sherpa-onnx for the rest. mavor doctor prints what it picked and why.

Every key, with its units and failure modes, is in the User Guide.

Models

Models are downloaded explicitly, never at runtime — the daemon fails at startup with a mavor models pull hint rather than stalling a dictation on a multi-gigabyte fetch.

mavor models list shows everything available, with what is already in the cache marked:

Model cache: /home/you/.cache/mavor/models

NAME                         ENGINE       SIZE  LANGUAGES            STREAM  STATUS
whisper-tiny.en              whisper   74.1 MB  en                   no      –
whisper-base.en              whisper  141.1 MB  en                   no      ✓ 141.1 MB  ★
whisper-large-v3-turbo       whisper   1.51 GB  multi (99)           no      –
parakeet-tdt-0.6b            sherpa   464.6 MB  multi (25)           no      –
sensevoice-small             sherpa   999.3 MB  zh, en, ja, ko, yue  no      –
zipformer-streaming          sherpa   296.0 MB  en                   yes     ✓ 320.2 MB
zipformer-streaming-20m      sherpa   122.0 MB  en                   yes     –
…

★ active   ✓ downloaded   – not downloaded
SIZE is the download; sherpa archives expand to roughly twice that on disk.
Download one with `mavor models pull <name>`.

That is seven of thirty-one rows; mavor models list prints them all.

• Every name carries its model family, and there are no aliases — one name per model. whisper-base.en, not base.en; a name that is not in the catalog is an error naming the closest entries, never a silent fallback.
• STREAM marks models that decode incrementally as you speak. Whisper is encoder-decoder over 30-second windows, so it always transcribes after you stop; the streaming sherpa transducers do not.
• SIZE is the download. The sherpa archives expand to roughly twice that on disk.

mavor models list                              # the catalog above
mavor models list --installed                  # only what is downloaded
mavor models list --verbose                    # a block per model, with the detail below
mavor models pull whisper-base.en              # production default
mavor models pull whisper-tiny.en              # smallest; what the test suite uses
mavor models pull zipformer-streaming          # the live-preview companion

--verbose adds the properties that do not fit a column:

zipformer-streaming
  Streaming Zipformer transducer — decodes while you speak
  engine      sherpa (in-process sherpa-onnx, CGO)
  download    296.0 MB
  languages   en
  streaming   yes — decodes incrementally while you speak
  speed       fast (relative tier, not measured)
  vocabulary  hotwords supported (transducer)
  gpu         none in practice — the bundled ONNX Runtime is a CPU-only build
  status      ✓ downloaded (320.2 MB)
  source      https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-streaming-zipformer-en-2023-06-26.tar.bz2

• speed is a relative tier across the catalog, estimated from architecture and parameter count. Where a real benchmark exists it is labelled measured and carries the real-time factor. Every model in the catalog has now been benchmarked for speed, memory and accuracy — docs/choosing-a-model.md says which to use, and docs/reports/model-benchmarks.md has the numbers. Rerun them on your hardware with just bench.
• vocabulary is what biasing the model can take from the [vocabulary] table. Whisper models take it as an initial prompt; transducers (parakeet, zipformer) get it as hotwords boosted while decoding, because that is the only place sherpa-onnx implements biasing. The CTC, paraformer, moonshine and sensevoice models can use none of it, and mavor doctor says so rather than failing.
• gpu depends on the build you are running, not on the model. Run mavor doctor, which reports what your whisper.cpp and ONNX Runtime can actually use rather than what the config asks for.

Whisper models are fetched from the whisper.cpp GGML repository and land in paths.models under the name upstream serves them by — whisper-base.en becomes ggml-base.en.bin. Sherpa models come from the sherpa-onnx release assets and unpack into paths.models/sherpa/<name>/.

Documentation

docs/ is indexed, and the index says which tree to trust for what — guides describe use, reference/ describes the built system, reports/ are generated measurements, and design/ and planning/ are proposals rather than descriptions.

• Quickstart · User Guide · Choosing a Model
• How mavor works — the daemon as built
• Measured model benchmarks — regenerate with just bench
• Roadmap — open decisions and known blockers

Desktop storybooks

Generate two linked local reports with real isolated desktop screenshots:

just storybook-nix
just storybook-gnome-nix

The wrappers use the existing hash-pinned Nix environment, unset inherited LD_LIBRARY_PATH, and supply the pinned C++ runtime. No host session, installation, service restart, or downloaded image assets are needed. Run these sequentially; each replaces its generated report.

With dependencies already available, use just storybook and just storybook-gnome directly. Sway staging additionally needs swaybg, GTK4 (4.12 or newer), pkg-config, and a C compiler; existing sway, waybar, grim, D-Bus, Go, and speech build dependencies remain required. GNOME needs Shell/Mutter with headless virtual-monitor support, working Mesa driver paths, gdbus, Python, GTK4, D-Bus, and xclip. Missing prerequisites fail, not skip. Ordinary overlay tests still start with their original black backdrop.

Open Sway HTML or GNOME HTML after generation. These and their PNG directories are ignored local artifacts, not committed files. GNOME also writes capture metadata beside its HTML. MAVOR_GNOME_ARTIFACTS chooses retained private-session diagnostics; keep that path short enough for Unix bus sockets.

Both HUD reports use one twelve-state catalog, the same production Go painter, and one Sway-derived HTML template: Hidden, six recording levels, Transcribing, Error, and short/long/cleared preview. Both stage the unchanged soft wallpaper and native session-notes editor at 1920×1080. These are controlled fixture inputs to actual production backends, not microphone capture or inference. Preview never reaches output. Frame receipts acknowledge submission only; the shared image validator independently measures screenshot bounds and checks labels, waveform, preview, clearing and duplicate HUDs. Wayland receipts have no global position; GNOME receipts use X protocol coordinates, not screenshot coordinates.

Captures come from Grim on Sway and Shell Screenshot on GNOME, never composites. Full frames, top crops, theme/filter/view controls and lightbox are shared. Native chrome and clock/pulse timing may differ. Shell fixture staging uses unsafe mode; production never uses Shell Eval. Missing prerequisites fail, not skip.

The separate just storybook-gnome-clipboard-nix recipe writes the seven-state gnome-clipboard-qa.html report: production copy and an explicit test-driven GTK Paste action, not automatic typing. just test-gnome-overlay-nix includes real HUD/clipboard coexistence and geometry checks. See HUD QA.

Development

Dev container

yolo-jail.jsonc is a committed yolo-jail definition: yolo from the repo root drops you in a container with the whole toolchain already present — sway and waybar for the headless integration tests, grim for the screenshot assertions, PipeWire and pulseaudio utilities for audio capture, wtype and wl-clipboard, and whisper-cpp. Optional; nothing in the build depends on it.

just --list for the full set. The interesting ones:

Test layout

• Unit tests (go test ./...): mocked Recorder/Transcriber/ Overlay/Output. Fast, run under -race.
• Integration tests (go test -tags=integration ./test/integration/...): spin up a real headless wlroots sway, optionally waybar, the real daemon binary, a host PipeWire null-sink, and assert against grim screenshots and wl-paste output.
• End-to-end test (go test -tags=e2e ./...): real whisper-cli plus a downloaded model.

The integration test rig lives in test/integration/harness.go. Each test gets its own XDG_RUNTIME_DIR, dbus session, headless sway, and optionally waybar + null-sink. Cleanup happens in t.Cleanup.

Project layout

cmd/mavor/                   # CLI entrypoint & subcommands (daemon, doctor, config, service, models)
internal/state/              # Idle ⇄ Recording ⇄ Transcribing FSM
internal/audio/              # Recorder interface + parec impl + VAD + PipeWire and OSC ducking
internal/speech/             # STT runtimes: whisper.cpp (server/cli) and in-process sherpa-onnx
internal/overlay/            # Layer-shell HUD: paint.go renders, overlay_wl.go presents
internal/wayland/            # Minimal hand-written Wayland client (wire protocol, layer-shell, shm)
internal/ipc/                # JSON-over-Unix-socket protocol
internal/output/             # wtype + wl-copy dispatch
internal/config/             # XDG_CONFIG_HOME/mavor/config.toml loader with ~/$VAR expansion
internal/daemon/             # wires everything; main.go is a thin caller
test/integration/            # headless sway + audio-stack test harness

The build is cgo

There is one build and it links the in-process sherpa-onnx recognizers, so a C compiler is required, CGO_ENABLED=0 does not work, and cross-compiling needs a cross toolchain. The two shared objects sherpa-onnx brings with it are vendored in the Go module cache and copied next to the binary by just build; the binary is linked with an $ORIGIN rpath so it finds them beside itself or in a sibling lib/. bin/ is the artifact, not bin/mavor.

The remaining build tags are test-only:

• integration: build the headless-sway test harness.
• e2e: opt in to tests that exercise real whisper-cli + a downloaded model.

Built on

mavor is a thin daemon around other people's hard work. In rough order of how much of the heavy lifting they do:

Speech recognition

• whisper.cpp — the whisper-server binary mavor supervises for every whisper model, the whisper-cli it falls back to, and the GGML model format the catalog pulls.
• sherpa-onnx and its Go bindings, sherpa-onnx-go — the in-process runtime behind every non-whisper model, including the streaming transducers the live preview uses. (ONNX Runtime rides along inside the platform modules, and is the 26 MB libonnxruntime.so that every mavor release ships beside the binary.)
• OpenAI Whisper, NVIDIA NeMo (Parakeet), k2-fsa/icefall (Zipformer), Useful Sensors (Moonshine) and FunASR (SenseVoice, Paraformer) — the model families the catalog carries.

Desktop integration

• wlroots and the wlr-layer-shell protocol — what lets the overlay be an anchored surface rather than a window your tiler has to place. mavor speaks the protocol directly rather than through a library.
• golang.org/x/image — the rasterizer the pill is drawn with, and the Go font it is typeset in. Embedding the font is what makes the overlay render identically on every machine.
• wtype — synthetic keystrokes over virtual-keyboard-unstable-v1.
• wl-clipboard — wl-copy and wl-paste.
• PipeWire and PulseAudio utilities — parec for capture and pactl for ducking.
• sway and wlroots — the compositor this is built for, and the headless one the integration tests run against.

Go modules

Testing

• grim for the screenshots the integration suite and the UI storybook assert against, and Waybar as the bar the overlay has to stay clear of.