kenzy 5.0.0

Distributed home voice-assistant: kenzy-node, kenzy-server, kenzy-stt, kenzy-tts, kenzy-llm, kenzy-speaker

KENZY ·

kenzy.ai · Documentation · Install

A distributed home voice assistant built as six independently deployable microservices. Kenzy runs wake-word detection locally on room nodes (Orange Pi Zero 3 / 3W or Raspberry Pi 3 / 4 / 5), streams audio to a central server for transcription, runs it through an LLM with tool-calling skills, and streams synthesized speech back to the room. It also keeps a live, per-room sense of where people are, built from your Home Assistant sensors and from who it last heard in each room.

Architecture

Node (mic) ──PCM over WebSocket──► Server
                                      │
                    ┌─────────────────┘ on session end
                    ▼
            STT  ──┐  (parallel)
            Speaker ID ──┘
                    │
                    ▼
                   LLM  ◄──► Skills (weather, news, home control, …)
                    │
                    ▼
                   TTS
                    │
             PCM over WebSocket ──► Node (speaker)

Alongside the voice pipeline, the server holds a persistent subscription to Home Assistant's WebSocket API, feeding the room-presence model described under Room presence. It's the one inbound stream that isn't audio.

Service	Command	Default port	Role
node	kenzy-node	—	Wake word + audio capture, TTS playback
server	kenzy-server	8765	WebSocket hub, pipeline orchestrator
stt	kenzy-stt	8767	Speech-to-text via faster-whisper
tts	kenzy-tts	8769	Text-to-speech via OpenAI or local Kokoro
llm	kenzy-llm	8766	LLM + skill tool-calling via LiteLLM
speaker	kenzy-speaker	8768	Speaker identification via SpeechBrain

Requirements

• Python 3.11+
• On Raspberry Pi OS / Debian: sudo apt-get install libportaudio2 portaudio19-dev
• API keys: OpenAI (TTS + LLM), Home Assistant (home control skill). The weather skill uses the National Weather Service API — no key required.

Setup

Kenzy installs from PyPI — the default configs, built-in skills, and .env.example ship as package data, so a service runs from a bare install with no source checkout:

pipx install "kenzy[node]"           # or use the one-line installer at kenzy.ai/install.sh
kenzy-setup                          # download wake-word / speaker-ID models (run once)
kenzy-init                           # scaffold a config home (~/.config/kenzy)

For development from a checkout, use an editable install instead:

# Create and activate a virtualenv
python3 -m venv .venv
source .venv/bin/activate

# Install the services you need
pip install -e ".[node]"                          # room node only
pip install -e ".[server,stt,tts,llm,speaker]"   # full server stack
pip install -e ".[node,server,stt,tts,llm,speaker,dev]"  # everything

# Download wake-word and speaker-ID models (run once after install)
kenzy-setup

# Configure API keys
cp .env.example .env
# Edit .env and fill in OPENAI_API_KEY, HA_API_KEY (as needed)

Running

The config-path argument is optional — each service resolves its config from the config home automatically. Start the server first: the backend services and nodes pull their config from it on startup and block until it answers.

# Server host first
kenzy-server  [configs/server.yaml]
kenzy-stt     [configs/stt.yaml]
kenzy-tts     [configs/tts.yaml]
kenzy-llm     [configs/llm.yaml]
kenzy-speaker [configs/speaker.yaml]

kenzy-node    [configs/node.yaml]     # then each room device (discovers + pulls from the server)

Speaker enrollment

To enable speaker identification, enroll each person once:

kenzy-enroll [configs/speaker.yaml]

To identify the correct audio device and sample rates for a node:

kenzy-devices

Remote deployment

kenzy-deploy manages installation and updates across a fleet of remote hosts over SSH. See configs/deploy.yaml for host configuration.

kenzy-deploy init       # one-time OS setup on all hosts
kenzy-deploy install    # first full deployment (source or PyPI mode)
kenzy-deploy upgrade    # install updates and restart services
kenzy-deploy status     # check service health
kenzy-deploy uninstall  # stop, remove units + venv (--purge also removes the install dir)

Prerequisites on each remote host: SSH key auth and passwordless sudo. Backend services are deployed in pull mode (they fetch config from the server) and per-node config lives in the server-owned central store (configs/nodes/, configs/services/), so a deployed fleet is managed from the dashboard. See the deployment docs for the central-config model, per-host node_id, and --reseed.

Dashboard

kenzy-server serves a web fleet manager — on by default in the shipped config (set dashboard.enabled: false in server.yaml to disable it entirely; nothing is then wired up). Open https://<server>:8770 — http:// if you installed without TLS. It gives you one place to:

• See live node + backend-service health and each host's installed version — including any room that has gone missing, with how long it has been unreachable (an orphaned node still answers its wake word, so it looks fine from inside the room)
• Configure each node, rename its room, and run guided calibration (also available by voice: "Hey Kenzy, calibrate") — it measures the room, detects echo cancellation, and applies the thresholds itself
• Manage skills (enable/disable live) and speaker profiles (rename / delete / enroll from a room)
• Watch room presence — which rooms have someone in them, what each belief rests on, and how fresh it is
• Watch pipeline activity (transcripts, latency, fast-path hit rate) and read server / service / node logs
• Trigger / stop / restart nodes and send TTS announcements to every room
• Upgrade the server, backend services, and nodes in place — one click, with an "update available" check against PyPI
• Edit a safe subset of the server's own config and change the dashboard password

The install.sh installer enables TLS by default (generating a self-signed pair into the config home; --no-tls opts out). A plain pip install starts plaintext — a trusted-LAN posture — until you add tls: {cert, key} to server.yaml. Either way TLS covers both the dashboard (https) and the node audio stream (wss); a self-signed cert works because Kenzy's own clients connect encrypted-but-unverified by default. See the server configuration docs.

All /api reads and actions require login; mutating actions also need controls. Login defaults to admin / password — change it with kenzy-passwd (server host only) or from the Settings page. It's a LAN appliance either way, so do not port-forward it. The Settings page also shows the node join token to copy when provisioning new nodes. See the Dashboard guide.

Configuration

The server is the configuration authority for the whole fleet. Nodes and the backend services pull their config from it at boot and are edited from the dashboard; the YAML files below are the server-side store and the seed defaults.

Key settings:

• configs/node.yaml — bootstrap-only (identity + how to reach the server + early logging). A node auto-generates a stable node_id, then blocks until the server pushes its full operational config (audio device, wake-word threshold/VAD, sounds, room name) and initializes audio from that. Per-node overrides live in configs/nodes/<node_id>.yaml; the room name is server-owned and set from the dashboard.
• configs/server.yaml — URLs for each downstream service (omit a URL to disable that stage), node_defaults, discovery, room presence (occupancy.enabled), and the dashboard block
• configs/services/<svc>.yaml — server-owned overrides for the backend services (stt/tts/llm/speaker), edited from the dashboard (Fleet → the service's chip); each service pulls its effective config (packaged default + this override, secrets stripped) from the server at boot
• configs/llm.yaml / stt.yaml / tts.yaml / speaker.yaml — packaged seed defaults for those services (model/voice/thresholds/etc.)

Secrets stay in each host's environment / .env — never in the config store.

Skills

Skills are async Python functions in skills/ decorated with @skill. They are discovered and loaded automatically at startup — no registration required. The LLM calls them as tools based on their docstrings and type signatures.

Included skills:

Skill file	What it does
weather.py	Current conditions and forecast via NWS
news.py	RSS headlines and article summaries
web_search.py	General web search for current or niche questions the model can't answer alone
stocks.py	Stock quotes via yfinance
home_assistant.py	Smart home control via Home Assistant REST API (secure actions require a recognized speaker)
lists.py	Shopping / to-do lists, backed by Home Assistant's todo entities — add, read, check off, create (no Kenzy-side storage, so your phone already has them)
schedule.py	Timers, alarms, and reminders — including "turn on the lights in 30 seconds", replayed through the pipeline at fire time
memory_skill.py	Remember / recall / forget, per person, with private / personal / shared tiers (recognized voices only)
presence.py	"Is Mom home?" — a live read of Home Assistant's person entities via each person's linked HA user
datetime_skill.py	Current date and time (with a deterministic fast path)
announce.py	Speak a message in every room (broadcast)
intercom.py	Start a live two-way voice call between two rooms (consent-gated at the far end)
volume.py	Set / adjust a room's playback volume or mute
enroll.py	Voice speaker enrollment ("enroll me as Alice")
calibrate.py	"Hey Kenzy, calibrate" — runs the guided audio calibration on the asking node
social.py	Instant greetings and "never mind" — fast path only, no model round-trip
random_tools.py	Coin flip, dice, random number, pick from list
knock_knock.py	Knock-knock jokes, both directions — she tells them and plays along with yours
about.py	Reports the installed Kenzy version

Adding a skill

# skills/my_skill.py
from kenzy.llm.skills import skill

@skill
async def my_skill(query: str) -> str:
    """One-line description the LLM uses to decide when to call this."""
    return "result"

Per-skill config lives under skills.<name> in llm.yaml. Secrets come from environment variables in .env.

Home Assistant device topology

Smart home control pulls your device topology live from Home Assistant — which entities exist, their friendly names, domains, and floor/area placement — so there's no device-map file to maintain. Add a device in HA and it's voice-controllable on the next refresh. Commands resolve in two tiers: a deterministic fast path (padacioso + rapidfuzz, no LLM) for everyday imperatives, and a sub-LLM fallback that reads the live floor → area → type → entity outline for harder requests.

Covered domains: lights, switches, fans, covers, locks, climate — plus scenes, scripts, buttons, and toggle helpers ("activate movie night", "run the goodnight routine", "turn on guest mode" — resolved by name house-wide), the robot vacuum ("start the vacuum", "send it home"), and media-player transport ("pause the TV", "skip this song", "turn the music down" — targeting whatever's actually playing; starting new music by name arrives with the Music Assistant integration).

The only hand-authored input is an optional data/home_assistant/curation.yaml — the voice layer HA can't store: spoken aliases, per-device notes, room group-defaults, voice-control exclusions, which to-do list "the list" means, and which sensors count as presence. Edit it directly or from the dashboard's Home Assistant tab (sub-tabs: Devices, Presence sensors, Lists). Run kenzy-ha-devices to print the live tree with each entity ID and its included/excluded status.

Room presence

Kenzy keeps a live picture of which rooms have someone in them. The server holds a persistent subscription to Home Assistant's WebSocket API and folds motion, occupancy and presence sensors — plus whose voice was just heard where — into a per-room belief that decays with time. The dashboard's Presence tab shows it: each room's state, what the claim rests on, and how old it is.

Two things it does on purpose:

• "Unknown" is a real answer. No sensor and no recent voice means Kenzy doesn't know, which is not the same as the room being empty. Rooms read unknown after a restart until something says otherwise.
• It acts on nothing. Presence changes no behavior yet — nothing speaks unprompted, no delivery is re-targeted. It's a world model you can watch, and catch being wrong, before anything depends on it. Acting on it comes next.

It needs Home Assistant configured; without it nothing starts and the tab stays hidden. Turn it off with occupancy.enabled: false. Which entities count is automatic (motion, occupancy and presence sensors, plus person entities for home/away) and adjustable per sensor under Home Assistant → Presence sensors — useful when a hallway sensor the cat trips keeps a room "occupied".

Development

source .venv/bin/activate
ruff check src/      # lint
ruff format src/     # format
mypy src/            # type-check
pytest               # run tests

Environment variables

See .env.example for the full list. Required variables:

Variable	Used by
OPENAI_API_KEY	TTS service, LLM service (if using OpenAI models)
HA_API_KEY	Home Assistant skill