smart-turn-livekit

Run Smart Turn end-of-turn models on LiveKit Agents. Given the last 8 seconds of a speaker's audio, the model predicts whether they have finished talking — from prosody, without waiting for STT.

pip install 'smart-turn-livekit[livekit]'

from livekit.agents import AgentSession
from livekit.plugins import silero
from smart_turn_livekit import SmartTurnDetector

session = AgentSession(
    vad=silero.VAD.load(min_silence_duration=0.25),   # must be >= 0.25
    turn_handling={
        "turn_detection": SmartTurnDetector(model="smart-turn-tamil-tiny"),
        "endpointing": {"min_delay": 0.3, "max_delay": 2.5},
    },
)

Weights download from HuggingFace on first use and are cached. Nothing to configure, nothing bundled in the wheel.

Models

SmartTurnDetector()                                  # smart-turn-tamil-tiny
SmartTurnDetector(model="smart-turn-tamil-base")
SmartTurnDetector(model="smart-turn-v3")

Note: Accuracies are not directly comparable as they use different data sources and test sets.

Every Smart Turn checkpoint shares one ONNX signature — input_features (batch, 80, 800) in, a logits output that is already a sigmoid out — which is why one code path runs all of them.

Tamil

End-of-turn detection for Tamil, expanding the languages supported by Smart Turn. These two models are fine-tuned specifically for Tamil from real Tamil telephone conversations.

tiny is the same architecture as v3, so it is a true drop-in. FP/N is Smart Turn's convention (FP/N + FN/N = error rate), not FP/(FP+TN) — they differ by ~3×. Latency is 1 thread, batch 1, idle i5-12450H, inference only.

Replayed through the real Silero VAD and this adapter, the model gives the identical verdict on 90.9% of boundaries and costs 2.6 accuracy points against scoring the pre-cut clips.

Method and ablations: https://github.com/santhosh-005/tamil-eot

Two things that break it

• min_silence_duration >= 0.25. LiveKit will not request a prediction below MIN_SILENCE_DURATION_MS + 50 ms and raises at session start if the VAD is faster. A floor, not a ceiling — it fails loudly.
• The language must be one the model claims, or unknown. supports_language() returns False otherwise, using the languages the model was benchmarked on. This one fails silently — the detector is skipped and the agent falls back to fixed timing.

Under LiveKit the threshold picks how long to wait — below it the session takes max_delay instead of min_delay — so a false complete costs a shorter pause, not an interruption.

Threshold

0.5 is what every number above is quoted at. The Tamil models also ship two tuned operating points, both picked on a held-out dev split:

SmartTurnDetector(operating_point="polite")
SmartTurnDetector(threshold=0.72)

Tuning for accuracy did not transfer: base's dev-argmax scores 85.10% on test against 86.23% at plain 0.5. That is why 0.5 is the default. smart-turn-v3 offers only inherited — there is no dev split of its data to pick one on.

Other uses

from smart_turn_livekit import SmartTurn

st = SmartTurn("smart-turn-tamil-tiny")
st.probability(wave_16k)      # P(speaker has finished), 0.0 .. 1.0
st.is_complete(wave_16k)      # bool, against st.threshold

16 kHz mono float, last 8 seconds only. Other rates need soxr.

Pipecat needs no adapter — its built-in analyzer takes any Smart Turn ONNX:

from smart_turn_livekit import resolve_model
from pipecat.audio.turn.smart_turn.local_smart_turn_v3 import LocalSmartTurnAnalyzerV3

analyzer = LocalSmartTurnAnalyzerV3(
    smart_turn_model_path=str(resolve_model("smart-turn-tamil-tiny")),   # the file
)

Pipecat hardcodes the 0.5 threshold, and there the decision ends the turn.

Pinned or air-gapped:

SmartTurnDetector(model_path="/opt/models/smart-turn-tamil.onnx")
SmartTurnDetector(local_only=True)                  # cached weights only

or SMART_TURN_MODEL=/path/to.onnx, or HF_HUB_OFFLINE=1. An explicit model_path claims no language until you pass languages=(...).

Tests

pip install -e '.[dev,livekit]'
pytest                                  # 22 tests
python -m smart_turn_livekit.bench

Checks features against Pipecat's own extractor (<1e-3), the LiveKit Protocols by isinstance, and that every registered model loads and returns a probability — including upstream's, which we do not control.

Limitations

• The Tamil models are narrowband telephony. Wideband or close-mic speech is out of distribution.
• Not a VAD. It answers "did they finish?", not "is anyone talking?" You still need Silero upstream.
• Audio-only, no language token. Nothing stops any of these models running on any language — supports_language() reflects what was measured.
• Run-to-run spread at identical config is ~0.9 points; treat smaller differences as noise.

Author

Built and maintained by Santhosh (@santhosh-005).

Licence

This package is BSD-2-Clause. It bundles Whisper's mel filterbank (MIT, OpenAI); nothing else in it is third-party.

The weights are downloaded, not distributed here, and licensed separately — smart-turn-v3 is BSD-2-Clause by Daily; the Tamil models are BSD-2-Clause, trained on data derived from SPRING_INX Tamil R1 (CC BY 4.0), SPRING Lab, IIT Madras.