LiveKit WakeWord

Why LiveKit WakeWord · Comparison · Guides and examples · Train your own · Models · Development

LiveKit WakeWord is an open-source toolkit for training and running wake word models. You provide a phrase, and the training CLI generates speech with LiveKit Inference TTS and adapts a pretrained encoder to recognize it. The resulting model runs locally in Python, Swift or on an ESP32, without sending audio to a server. You do not need to record your own training data.

Why LiveKit WakeWord

Training a standard model for “hey livekit” took 11 minutes on one RTX PRO 6000 GPU: about 6 minutes to generate speech and 5 minutes to train, calibrate and export the model. Training used 1,000 generated wake-word clips. The same workflow produces a micro model in about 18 minutes, including the standard model it learns from.

The standard model is about 7 MB and runs in Python or Swift through ONNX Runtime. The micro model is 451 KB and runs on ESP32 through TensorFlow Lite Micro. You can start with a ready-made model, or train a model for your own phrase. Running detection requires neither a GPU nor LiveKit Cloud credentials.

Comparison

We trained “hey livekit” separately in each engine using its own recipe, then evaluated the models on the same benchmark. At 0.5 false alarms per hour, LiveKit WakeWord detected 89% of wake words, nanowakeword 49%, and openWakeWord 46%. Their training times were 11, 34 and 94 minutes, respectively. Streaming inference used 16.5, 30.5 and 29.5 ms of CPU time per second of audio.

The micro model detected 78% of wake words at the same false-alarm rate. microWakeWord scored 0% at that operating point, although it used less CPU: 2.2 ms per second of audio versus 18 ms for LiveKit micro. These results compare the complete training workflows for one phrase, including their different speech generators; they do not isolate encoder performance.

Guides and examples

Each client has a README with installation and usage instructions, along with examples you can run. Use a ready-made model or one you trained yourself.

Models

The latest release has ready-made models and the pretrained encoders that training starts from:

gh release download -R livekit/livekit-wakeword -p "*.onnx" -p "*.tflite" -p "*.json"

hey livekit is scored on the bench with party scenes (the one in the comparison above), and hey jarvis on the plain bench. standard is ONNX (about 7 MB, about 1 ms per 80 ms step on one CPU thread). micro is int8 TFLite (451 KB, 11.5 million multiply-accumulates per second of audio) for TensorFlow Lite Micro. At the default threshold of 0.5, these four models give 0.5 to 2.5 false alarms per hour on the bench's conversational speech; raise the threshold if you see too many.

Train a wake word

You need Python 3.12+, a CUDA GPU or an Apple silicon Mac, uv and a LiveKit Cloud project (the free plan works) for the TTS.

1. Get LiveKit credentials. Sign up at cloud.livekit.io and create a project, then install the LiveKit CLI and log in:

brew install livekit-cli                         # macOS
curl -sSL https://get.livekit.io/cli | bash      # Linux
winget install LiveKit.LiveKitCLI                # Windows

lk cloud auth                                    # log in in the browser and pick your project
lk app env -w                                    # writes LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET to .env.local
set -a && source .env.local && set +a            # export them in this shell

You can also create an API key in the project settings of the LiveKit Cloud dashboard and export the three variables yourself.

2. Get the code:

git clone https://github.com/livekit/livekit-wakeword && cd livekit-wakeword
uv sync --extra train                            # or --extra tflite for --type micro

setup downloads the pretrained encoders from the v2.0.0 release once.

3. Train:

uv run livekit-wakeword setup --wakeword "hey robot" --eval  # training speech + held-out test set
uv run livekit-wakeword train --wakeword "hey robot" --type standard
uv run livekit-wakeword eval --wakeword "hey robot"    # bench score on a held-out test set
uv run --with pyaudio livekit-wakeword listen hey_robot.onnx   # try it on your microphone

A new phrase takes about 11 minutes on one data-center GPU (6 minutes of speech generation, 5 of training). The first run downloads about 4 GB of noise, room and speech sources once.

For ESP32, also train a micro model (about 7 more minutes):

uv run livekit-wakeword train --wakeword "hey robot" --type micro   # -> hey_robot_micro.tflite

More speech, or a different output folder:

uv run livekit-wakeword setup --wakeword "hey robot" --clips 4      # 4x the default training speech
uv run livekit-wakeword train --wakeword "hey robot" --out models   # writes models/hey_robot.onnx

Multilingual wake words. A phrase can be in any language and script. Pass its language with --language (-l, an ISO 639-1 code such as es, de, vi or ja) so the speech is generated by voices that speak it; train and eval read it from the saved config:

uv run livekit-wakeword setup --wakeword "hola asistente" --language es --eval
uv run livekit-wakeword setup --wakeword "xin chào robot" --language vi
uv run livekit-wakeword setup --wakeword "ねえ ロボット" --language ja
uv run livekit-wakeword train --wakeword "ねえ ロボット"             # -> ねえ_ロボット.onnx

The model file is named after the phrase; set model_name in the config (for example ne_robot) for an ASCII file name, such as for ESP32 builds.

In English, words outside the dictionary (names, brands, made-up words) need a pronunciation in ARPAbet, so every voice says them the same way; setup names any word it is missing. Words made of dictionary words, such as "livekit", work without one. Phrases in other languages need none: their speech is generated from the text, by every TTS model that speaks the language.

uv run livekit-wakeword setup --wakeword "hey zork" --pronounce "zork=Z AO1 R K"

The training guide lists every flag, with a full example from setup to evaluation.

Repository layout

Development

uv sync --extra tflite
uv run pytest                                    # Python: client and trainer
uv run ruff check . && uv run mypy -p livekit.wakeword -p livekit.wakeword_train
swift test                                       # Swift client

License

Apache 2.0, see LICENSE.