Skip to main content

Speechmatics STT plugin for LiveKit Agents

Support for Speechmatics STT.

See https://docs.livekit.io/agents/integrations/stt/speechmatics/ for more information.

Installation

pip install livekit-plugins-speechmatics

Diarization

Speechmatics STT engine can be configured to emit information about individual speakers in a conversation. This needs to be enabled using enable_diarization=True. The text output of the transcription can be configured to include this information using the macros speaker_id and text, as shown in the examples below.

  • <{speaker_id}>{text}</{speaker_id}> -> <S1>Hello</S1>
  • [Speaker {speaker_id}] {text} -> [Speaker S1] Hello

You should adjust your system instructions to inform the LLM of this format for speaker identification.

Turn detection modes

The turn_detection_mode parameter controls how end-of-turn is detected:

  • EXTERNAL (default) — Speechmatics does not endpoint on its own; turn boundaries are driven by an external VAD or by calling finalize(). If no vad is passed, Silero is auto-loaded (requires livekit-plugins-silero). Pass vad=None to opt out and drive finalize() yourself.
  • ADAPTIVE — Speechmatics controls end of turn using its own VAD and the pace of speech.
  • SMART_TURN — Speechmatics ML-based endpointing.
  • FIXED — Endpoints after a fixed silence duration set by end_of_utterance_silence_trigger.

Usage (LiveKit Turn Detection)

The default EXTERNAL mode pairs naturally with LiveKit's turn detector. The format for the output text needs to be adjusted to not include any extra content at the end of the utterance. Using [Speaker S1] ... as the speaker_active_format should work well. You may need to adjust your system instructions to inform the LLM of this format for speaker identification. You must also include the listener for when the VAD has detected the end of speech.

The end_of_utterance_silence_trigger parameter controls the amount of silence before the end of turn detection is triggered. The default is 0.5 seconds.

Usage:

from livekit.agents import AgentSession, inference
from livekit.agents.inference import TurnDetector
from livekit.plugins import speechmatics

agent = AgentSession(
    stt=speechmatics.STT(
        end_of_utterance_silence_trigger=0.2,
        speaker_active_format="[Speaker {speaker_id}] {text}",
        speaker_passive_format="[Speaker {speaker_id} *PASSIVE*] {text}",
    ),
    vad=inference.VAD(),
    turn_detection=TurnDetector(),
    min_endpointing_delay=0.3,
    max_endpointing_delay=5.0,
    ...
)

Usage (Speechmatics end of utterance detection and speaker ID)

To delegate end-of-turn detection to Speechmatics, set turn_detection_mode=TurnDetectionMode.ADAPTIVE (or SMART_TURN / FIXED) and pair it with turn_detection="stt" on the AgentSession.

from livekit.agents import AgentSession
from livekit.plugins import speechmatics

agent = AgentSession(
    stt=speechmatics.STT(
        turn_detection_mode=speechmatics.TurnDetectionMode.ADAPTIVE,
        speaker_active_format="[Speaker {speaker_id}] {text}",
        speaker_passive_format="[Speaker {speaker_id} *PASSIVE*] {text}",
        additional_vocab=[
            speechmatics.AdditionalVocabEntry(
                content="LiveKit",
                sounds_like=["live kit"],
            ),
        ],
    ),
    turn_detection="stt",
    ...
)

Pre-requisites

You'll need to specify a Speechmatics API Key. It can be set as environment variable SPEECHMATICS_API_KEY or .env.local file.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

livekit_plugins_speechmatics-1.6.9.tar.gz (15.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

livekit_plugins_speechmatics-1.6.9-py3-none-any.whl (16.9 kB view details)

Uploaded Python 3

File details

Details for the file livekit_plugins_speechmatics-1.6.9.tar.gz.

File metadata

File hashes

Hashes for livekit_plugins_speechmatics-1.6.9.tar.gz
Algorithm Hash digest
SHA256 04c5ede9f7f88f99c0710ec6d1f22a20852a15e78f034fec5bbca39673378721
MD5 9c1103eae3f0fc0bbd347990528181c3
BLAKE2b-256 a848ee08ad22156eec5dda0c8437f2992a643bbcea100d2d07800a1d84f3a64c

See more details on using hashes here.

Provenance

The following attestation bundles were made for livekit_plugins_speechmatics-1.6.9.tar.gz:

Publisher: publish.yml on livekit/agents

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file livekit_plugins_speechmatics-1.6.9-py3-none-any.whl.

File metadata

File hashes

Hashes for livekit_plugins_speechmatics-1.6.9-py3-none-any.whl
Algorithm Hash digest
SHA256 42f50f004990cdb73345e6461c9e0d9f3e45d4328fd50b89a2572375cd41b0fb
MD5 80614ba7d4a9fc0af9546ea41f6464df
BLAKE2b-256 94de747d468c9684d81b587b861e1adcea4fb76c4a8853716260da80b23f11b5

See more details on using hashes here.

Provenance

The following attestation bundles were made for livekit_plugins_speechmatics-1.6.9-py3-none-any.whl:

Publisher: publish.yml on livekit/agents

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

1.6.9

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page