livekit-plugins-rumik

Rumik AI text-to-speech plugin for LiveKit Agents.

Streams low-latency 24 kHz speech from Rumik's Silk models over a reusable WebSocket session:

• muga — emotion-controlled via a leading [tone] tag (e.g. [happy], [sad]) plus optional <laugh>/<chuckle>/<sigh> events. Tuned for Romanized Hinglish.
• mulberry — steered by a natural-language voice description or a preset speaker, with optional pitch shift (f0_up_key).

Migrating from livekit-plugins-rumik-ai

This package is the continuation of livekit-plugins-rumik-ai (last release 0.2.2) under a new name. The API is unchanged. Swap the requirement and one import line:

- livekit-plugins-rumik-ai==0.2.2
+ livekit-plugins-rumik>=0.2.3

- from livekit.plugins import rumik_ai
+ from livekit.plugins import rumik as rumik_ai   # the rest of your code stays the same

The two packages install under different module names, so they never overwrite each other. You can uninstall the old one at your convenience.

Install

pip install livekit-plugins-rumik

This depends on livekit-agents (1.5+). Set your key:

export RUMIK_API_KEY="your-rumik-api-key"

Quickstart

from livekit.agents import AgentSession
from livekit.plugins import rumik

# muga: the LLM should start each reply with one tone tag, e.g. "[happy] ..."
session = AgentSession(
    stt=...,
    llm=...,
    tts=rumik.TTS(model="muga"),
)

Mulberry, steered by a voice description (or a preset speaker):

tts = rumik.TTS(
    model="mulberry",
    description="warm, gentle female friend",
    # speaker="ira",            # optional preset voice, overrides description
    # f0_up_key=2.0,            # optional pitch shift, -12..12 semitones
)

Changing the voice at runtime

description, speaker, f0_up_key, and the sampling params are sent on every request, so you can change mulberry's voice between turns without reconnecting — the pooled WebSocket is reused (only a model change re-mints the session):

tts.update_options(description="excited young man, fast and energetic")
# the next synthesis request uses the new voice

Latency vs. smoothness

The default is model-aware:

• muga buffers the full LLM reply and synthesizes it in one request, so its leading [tone] tag conditions the whole utterance (and there are no per-request TTFB gaps).
• mulberry streams sentence-by-sentence for lower time-to-first-word, since it has no tone tag to protect.

Override either with full_response_aggregation:

rumik.TTS(model="muga", full_response_aggregation=False, tone="neutral")  # muga, lower latency
rumik.TTS(model="mulberry", full_response_aggregation=True)               # mulberry, smoother

When you turn aggregation off for muga, set a fallback tone= so every sentence keeps a tone tag.

Barge-in & cancel

Built for the live "call with AI" case. When the caller talks over the agent, LiveKit interrupts the TTS and the plugin sends an explicit cancel to Rumik, so the in-flight generation stops immediately (and billing is finalized cleanly). The pooled WebSocket is kept warm across the interruption, so the next utterance doesn't pay a reconnect.

Connection lifecycle

The session stays open between replies, including long pauses: the plugin answers the server's pings and sends a keepalive before the server's idle timeout. Sessions are refreshed every few minutes, and the replacement is opened in the background, so a reply never waits on a new connection. prewarm() (called for you by AgentSession) opens the first one before the first reply. A TTS instance that receives no requests for 10 minutes closes its session; the next request opens a new one. Call aclose() when you are done.

Configuration

Examples

See examples/ for a full voice agent (rumik_agent.py) and a record-to-WAV demo (rumik_tts.py).

License

Apache-2.0