AssemblyAI Plugin
Streaming Speech-to-Text (STT) plugin for Vision Agents using AssemblyAI's Universal-3 Pro model.
Features
- Real-time streaming transcription via async WebSocket
- Built-in punctuation-based turn detection with configurable silence thresholds
- Streaming diarization — identify speakers in real time
- Native
SpeechStartedevent support - Custom prompt and keyterms boosting support
- Sub-300ms time to complete transcript latency
- Built-in reconnection with exponential backoff
Installation
uv add "vision-agents[assemblyai]"
# or directly
uv add vision-agents-plugins-assemblyai
Usage
from vision_agents.plugins import assemblyai
stt = assemblyai.STT(
speech_model="u3-rt-pro", # Default model
sample_rate=16000,
)
With streaming diarization
Enable speaker_labels to identify speakers in a mixed audio stream. Each transcript event will carry a distinct participant per speaker and the raw label in response.other["speaker_label"].
stt = assemblyai.STT(
speaker_labels=True,
max_speakers=2, # optional hint (1-10)
)
With keyterms boosting
stt = assemblyai.STT(
keyterms_prompt=["AssemblyAI", "Vision Agents"],
)
With custom turn silence thresholds
stt = assemblyai.STT(
min_turn_silence=100, # ms before speculative EOT check
max_turn_silence=1200, # ms before forcing turn end
)
Configuration
| Parameter | Description | Default |
|---|---|---|
api_key |
AssemblyAI API key (falls back to ASSEMBLYAI_API_KEY env var) |
None |
speech_model |
Model identifier | "u3-rt-pro" |
sample_rate |
Audio sample rate in Hz | 16000 |
min_turn_silence |
Silence (ms) before speculative end-of-turn check | API default |
max_turn_silence |
Maximum silence (ms) before forcing turn end | API default |
prompt |
Custom transcription prompt (cannot be combined with keyterms_prompt) |
None |
keyterms_prompt |
List of terms to boost recognition for (cannot be combined with prompt) |
None |
speaker_labels |
Enable streaming diarization for multi-speaker identification | False |
max_speakers |
Hint for expected number of speakers, 1-10 (requires speaker_labels=True) |
None |
max_reconnect_attempts |
Maximum reconnect attempts on transient failures | 3 |
reconnect_backoff_initial_s |
Initial backoff delay in seconds | 0.5 |
reconnect_backoff_max_s |
Maximum backoff delay in seconds | 4.0 |
Environment Variables
Set ASSEMBLYAI_API_KEY in your environment or pass api_key to the constructor.
Dependencies
aiohttp>=3.9.0vision-agents
Docs
Metadata
Release files for vision-agents-plugins-assemblyai 0.6.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| vision_agents_plugins_assemblyai-0.6.9.tar.gz | 6.2 kB | Without provenance | Details for vision_agents_plugins_assemblyai-0.6.9.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| vision_agents_plugins_assemblyai-0.6.9-py3-none-any.whl | Python 3 | none | any | Details for vision_agents_plugins_assemblyai-0.6.9-py3-none-any.whl |
Total release size: 12.7 kB
Release files / vision_agents_plugins_assemblyai-0.6.9.tar.gz
| Download URL | vision_agents_plugins_assemblyai-0.6.9.tar.gz |
|---|---|
| Size | 6.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
45c5c8c65d0a36dc2f86edefc0c1b1622f15f5fe9fc9b4b22029cbeb1a1bd967
|
|
BLAKE2b-256 checksum How to use checksums |
3f3da937934f5592cde1ea8eb581e349503277ff29624fe2fdbbc37860f7da5d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / vision_agents_plugins_assemblyai-0.6.9-py3-none-any.whl
| Download URL | vision_agents_plugins_assemblyai-0.6.9-py3-none-any.whl |
|---|---|
| Size | 6.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
55265f1724dce67ed61d27cbbaffcc168ff1d6d389a6b22b43013df21befefc3
|
|
BLAKE2b-256 checksum How to use checksums |
9e1e752ad389b835d2e14dba96de5b635c89dd9f930c7e0dd55d67be5eb10dfd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.10.10 {"installer":{"name":"uv","version":"0.10.10","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for vision-agents-plugins-assemblyai 0.6.9
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.