AssemblyAI Plugin
Streaming Speech-to-Text (STT) plugin for Vision Agents using AssemblyAI's Universal-3 Pro model.
Features
- Real-time streaming transcription via async WebSocket
- Built-in punctuation-based turn detection with configurable silence thresholds
- Streaming diarization — identify speakers in real time
- Native
SpeechStartedevent support - Custom prompt and keyterms boosting support
- Sub-300ms time to complete transcript latency
- Built-in reconnection with exponential backoff
Installation
uv add "vision-agents[assemblyai]"
# or directly
uv add vision-agents-plugins-assemblyai
Usage
from vision_agents.plugins import assemblyai
stt = assemblyai.STT(
speech_model="u3-rt-pro", # Default model
sample_rate=16000,
)
With streaming diarization
Enable speaker_labels to identify speakers in a mixed audio stream. Each transcript event will carry a distinct participant per speaker and the raw label in response.other["speaker_label"].
stt = assemblyai.STT(
speaker_labels=True,
max_speakers=2, # optional hint (1-10)
)
With keyterms boosting
stt = assemblyai.STT(
keyterms_prompt=["AssemblyAI", "Vision Agents"],
)
With custom turn silence thresholds
stt = assemblyai.STT(
min_turn_silence=100, # ms before speculative EOT check
max_turn_silence=1200, # ms before forcing turn end
)
Configuration
| Parameter | Description | Default |
|---|---|---|
api_key |
AssemblyAI API key (falls back to ASSEMBLYAI_API_KEY env var) |
None |
speech_model |
Model identifier | "u3-rt-pro" |
sample_rate |
Audio sample rate in Hz | 16000 |
min_turn_silence |
Silence (ms) before speculative end-of-turn check | API default |
max_turn_silence |
Maximum silence (ms) before forcing turn end | API default |
prompt |
Custom transcription prompt (cannot be combined with keyterms_prompt) |
None |
keyterms_prompt |
List of terms to boost recognition for (cannot be combined with prompt) |
None |
speaker_labels |
Enable streaming diarization for multi-speaker identification | False |
max_speakers |
Hint for expected number of speakers, 1-10 (requires speaker_labels=True) |
None |
max_reconnect_attempts |
Maximum reconnect attempts on transient failures | 3 |
reconnect_backoff_initial_s |
Initial backoff delay in seconds | 0.5 |
reconnect_backoff_max_s |
Maximum backoff delay in seconds | 4.0 |
Environment Variables
Set ASSEMBLYAI_API_KEY in your environment or pass api_key to the constructor.
Dependencies
aiohttp>=3.9.0vision-agents
Docs
2 maintainers
Metadata
2 maintainers
Release files for vision-agents-plugins-assemblyai 0.6.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| vision_agents_plugins_assemblyai-0.6.9.tar.gz | 6.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| vision_agents_plugins_assemblyai-0.6.9-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 12.7 kB
Release files / vision_agents_plugins_assemblyai-0.6.9.tar.gz
| Download URL | vision_agents_plugins_assemblyai-0.6.9.tar.gz |
|---|---|
| Size | 6.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
45c5c8c65d0a36dc2f86edefc0c1b1622f15f5fe9fc9b4b22029cbeb1a1bd967
|
|
BLAKE2b-256 checksum How to use checksums |
3f3da937934f5592cde1ea8eb581e349503277ff29624fe2fdbbc37860f7da5d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / vision_agents_plugins_assemblyai-0.6.9-py3-none-any.whl
| Download URL | vision_agents_plugins_assemblyai-0.6.9-py3-none-any.whl |
|---|---|
| Size | 6.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
55265f1724dce67ed61d27cbbaffcc168ff1d6d389a6b22b43013df21befefc3
|
|
BLAKE2b-256 checksum How to use checksums |
9e1e752ad389b835d2e14dba96de5b635c89dd9f930c7e0dd55d67be5eb10dfd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
