Skip to main content

Aavaaz

Production-grade speech-to-text platform built on WhisperLive.

Aavaaz (/aa-vaa-z/, आवाज़, "voice" in Hindi) is an open source extension of WhisperLive with enterprise features that compete with Deepgram, ElevenLabs, and AssemblyAI.

Features

Category Capabilities
Transcription Real-time WebSocket streaming, REST API (OpenAI-compatible), batch inference, multichannel audio (Lambda/Modal)
Intelligence Speaker diarization, sentiment analysis, topic detection, entity extraction, summarization
Post-processing Smart formatting, PII redaction, profanity filtering, noise reduction, filler word removal, utterance/paragraph segmentation
Platform Webhook delivery, transcript search & tagging (SaaS API), S3 output storage (Lambda), API-key and JWT auth, JWT/SSO (JWKS), GDPR export/erasure, Prometheus metrics
Deployment Docker, Helm charts, Terraform (AWS), serverless (Lambda), Modal (GPU), GPU auto-detection, model caching, SSE streaming

Quick Start

# Create a virtualenv (Python 3.12 or 3.13)
python3.12 -m venv .venv && source .venv/bin/activate

# Install aavaaz with WhisperLive + ML stack
pip install "aavaaz[whisper]"

# Start the server
aavaaz serve --model large-v3

# Transcribe a file
aavaaz transcribe audio.wav

Option 2: Using uv (Fast & Reproducible)

# Install uv: https://docs.astral.sh/uv/
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create venv with Python 3.12 or 3.13
uv venv .venv --python python3.12
source .venv/bin/activate

# Install from PyPI
uv pip install "aavaaz[whisper]"

# Start the server
aavaaz serve --model large-v3

# Transcribe a file
aavaaz transcribe audio.wav

Option 3: Local Development Install

git clone git@github.com:collabora/aavaaz.git
cd aavaaz
python3.12 -m venv .venv && source .venv/bin/activate

# Local editable install
pip install -e .

# With WhisperLive + dev tooling
pip install -e ".[whisper,dev]"

Option 4: Using pip with Requirements Files

# Create a virtualenv (Python 3.12 or 3.13)
python3.12 -m venv .venv && source .venv/bin/activate

# Install base + ML stack (about 8 GB, mostly torch)
pip install -r requirements/whisper.txt

# Or install just base (fast, no ML):
# pip install -r requirements/base.txt

# Start the server (requires ML stack)
aavaaz serve --model large-v3

# Transcribe a file
aavaaz transcribe audio.wav

# OpenAI-compatible REST endpoint. The `model` form field takes a stock size,
# a local path, or an HF repo, and defaults to the server's `--model`.
curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F model=small \
  -F file=@audio.wav

Note on Storage

The full ML stack (torch, torchaudio, CUDA libraries) needs about 8 GB of disk. If you hit disk quota errors, consider:

  • Using uv which is faster and handles large downloads better
  • Installing on a machine with more space
  • Using serverless deployments (AWS Lambda / Modal) instead of local

Requirements Files

  • requirements/base.txt — Core dependencies only (fastapi, uvicorn, boto3)
  • requirements/whisper.txt — Full ML stack (torch, whisper-live>=0.10.0, etc)
  • requirements/dev.txt — Development tools (pytest, ruff, etc)

Architecture

Aavaaz uses WhisperLive as its transcription engine and extends it via the plugin system:

┌─────────────────────────────────────────┐
│              Aavaaz Server               │
│  ┌─────────────────────────────────┐    │
│  │      REST API / WebSocket       │    │
│  └──────────────┬──────────────────┘    │
│  ┌──────────────┴──────────────────┐    │
│  │        Plugin Pipeline          │    │
│  │  formatting → PII redaction →   │    │
│  │  profanity → intelligence       │    │
│  └──────────────┬──────────────────┘    │
│  ┌──────────────┴──────────────────┐    │
│  │    WhisperLive Core Engine      │    │
│  │  faster-whisper / TensorRT /    │    │
│  │  OpenVINO, VAD, diarization     │    │
│  └─────────────────────────────────┘    │
└─────────────────────────────────────────┘

Advanced Features

Word-Level Timestamps

Enable per-word timing and confidence scores in transcription segments:

from aavaaz import AavaazServer

server = AavaazServer()
server.serve(word_timestamps=True)

When enabled, each segment includes a words array:

{
  "segments": [{
    "start": "0.000", "end": "2.500", "text": "Hello world", "completed": true,
    "words": [
      {"word": "Hello", "start": "0.000", "end": "0.800", "probability": 0.95},
      {"word": " world", "start": "0.900", "end": "2.500", "probability": 0.88}
    ]
  }]
}

Custom Vocabulary / Hotwords

Boost recognition of specific terms (product names, acronyms, domain jargon):

from aavaaz import AavaazServer

server = AavaazServer()
server.serve(hotwords="Aavaaz,TensorRT,OpenVINO")

The hotwords parameter is a comma-separated string passed directly to faster-whisper's keyword boosting. Also available in the REST API via the hotwords form field.

Speaker Diarization

Real-time speaker identification using pyannote.audio embeddings:

pip install pyannote.audio
from aavaaz import AavaazServer

server = AavaazServer()
server.serve(enable_diarization=True, max_speakers=4)

When enabled, completed segments include a speaker field:

{"start": "0.000", "end": "2.500", "text": "Hello", "speaker": "SPEAKER_00", "completed": true}

Streaming Post-Processing Flags

Post-processing options for aavaaz serve:

aavaaz serve --model large-v3 \
  --noise-reduction near_field \
  --profanity-filter --profanity-mode full --profanity-words foo,bar \
  --filler-removal --filler-aggressive \
  --intelligence --paragraphs \
  --callback-url https://example.com/hook
  • --noise-reduction {near_field,far_field} — reduce noise on live audio frames before transcription.
  • --profanity-mode {partial,full,remove} — how --profanity-filter masks a match.
  • --profanity-words a,b — extra words added to the profanity list.
  • --filler-removal — drop filler words ("um", "you know") from each segment.
  • --filler-aggressive — also drop borderline fillers ("like", "actually", "right").
  • --intelligence — per-segment sentiment/topics/entities, plus a final {"intelligence": ...} message with summary and highlights for the whole transcript.
  • --paragraphs — a final {"paragraphs": [...]} message at stream end.
  • --callback-url URL — POST the final transcript (segments, paragraphs, intelligence) to URL, with retries.

Authentication

Protect both REST API and WebSocket connections with a shared API key:

aavaaz serve --model large-v3 --api-key "my-secret-key"
  • REST API: Requires Authorization: Bearer my-secret-key header
  • WebSocket: Requires either Authorization: Bearer my-secret-key header or ?token=my-secret-key query parameter

Unauthenticated connections receive HTTP 401 before any GPU resources are allocated.

Set AAVAAZ_JWT_SECRET to the HS256 secret the rest of the platform signs with and the WebSocket requires a platform token on every handshake as well. A browser offers it as new WebSocket(url, ["bearer", token]), other clients send an Authorization: Bearer <token> header. The token must carry exp and sub and must not carry aud, which is what stops a session token minted for another service being replayed here. With the variable unset the WebSocket accepts any client and says so in a warning at startup.

Rate Limiting

Limit REST API requests per client IP (sliding 60-second window):

aavaaz serve --model large-v3 --rate-limit-rpm 60

Clients exceeding the limit receive HTTP 429.

Batch Inference

Batch multiple client sessions into single GPU calls for higher throughput:

aavaaz serve --model large-v3 --batch-inference --batch-max-size 16 --batch-window-ms 50

--batch-max-queue-wait (default 0.5 seconds) turns new clients away with a WAIT message once requests sit in the batch queue longer than that. --batch-max-admissions-per-s (default 5) caps how many new clients get in per second, so a burst of reconnects cannot overshoot the queue before the wait reflects it. --batch-beam-size (default 1, greedy) sets the decoder beam width, 5 matches the non-batched path at about 2.5x the decode time. --batch-temperature-fallback (default off) re-decodes chunks that fail the quality check at higher temperatures, each such pass costs about as much as the first and halves throughput at beam 1.

Prometheus Metrics

Monitor server health with a Prometheus /metrics endpoint:

aavaaz serve --model large-v3 --metrics-port 9091

Tracks active connections, transcription latency, segment counts, and error rates.

SSE Streaming

Stream transcription results via Server-Sent Events from the REST API:

curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F file=@audio.wav -F stream=true

Returns real-time segment events as text/event-stream.

Plugin System

Extend the transcription pipeline with custom post-processors:

from aavaaz import AavaazServer, PluginRegistry

registry = PluginRegistry()
registry.add("my_plugin", my_post_processor_fn, priority=50)

server = AavaazServer(plugin_registry=registry)
server.serve()

Plugins receive each transcription segment dict and return the modified dict. Passing your own registry replaces the built-in plugins; use from aavaaz.plugins import registry and registry.add(...) to extend the built-ins instead.

Scaling Guide

Single GPU

aavaaz serve --model large-v3 --batch-inference

Docker Compose

services:
  aavaaz:
    build: .
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
    ports:
      - "9090:9090"
      - "8000:8000"

Kubernetes (Helm)

helm install aavaaz deploy/helm/aavaaz \
  --set model=large-v3 \
  --set replicaCount=3

Each replica requests one nvidia.com/gpu (see values.yaml resources).

AWS (Terraform)

cd deploy/terraform
terraform init
terraform apply -var="model=large-v3" -var="api_key=my-secret"

Provisions VPC, ALB, ECS with GPU instances (g5.xlarge), ECR, and CloudWatch. See deploy/terraform/README.md for full options.

AWS Lambda (Serverless)

For batch file transcription without managing servers:

Production role: Lambda is the batch transcription path.

# Build and push the Lambda container image
docker build -f Dockerfile.lambda --build-arg WHISPER_MODEL=small -t aavaaz-lambda .

# Deploy infrastructure
cd deploy/terraform-lambda
terraform init
terraform apply

# Upload audio — transcript appears automatically in the output bucket
aws s3 cp recording.wav s3://$(terraform output -raw audio_input_bucket)/

# Or use the REST API
curl -X POST $(terraform output -raw api_endpoint) \
  -H "Content-Type: application/json" \
  -d '{"audio_url": "s3://my-bucket/recording.wav"}'

See docs/SERVERLESS.md for full configuration, model selection, cost estimates, and limitations.

Modal (GPU Serverless)

Deploy on Modal for on-demand GPU transcription with zero infrastructure:

Production role: Modal is used for live WebSocket transcription (deploy/modal/app_live.py). The deploy/modal/app.py endpoint is an optional GPU batch API.

cd deploy/modal
pip install modal
modal setup
modal deploy app_live.py

# Optional: deploy the GPU batch API endpoint
# modal deploy app.py

# Transcribe
# Live websocket URL is exposed by app_live.py deployment output.

Scales to zero after two idle minutes; the first connection after that waits for a container cold start. See docs/MODAL.md for full configuration.

Development

git clone git@github.com:collabora/aavaaz.git
cd aavaaz
pip install -e ".[dev]"
pytest

License

MPL-2.0

Metadata

Release files for aavaaz 0.10.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for aavaaz 0.10.0
File Size Uploaded Provenance Actions
aavaaz-0.10.0.tar.gz 115.2 kB With provenance Details for aavaaz-0.10.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for aavaaz 0.10.0
File Interpreter ABI Platform Actions
aavaaz-0.10.0-py3-none-any.whl Python 3 none any Details for aavaaz-0.10.0-py3-none-any.whl

Total release size: 193.9 kB

Release files / aavaaz-0.10.0.tar.gz

Download URL aavaaz-0.10.0.tar.gz
Size 115.2 kB
Tags Source
SHA-256 checksum
How to use checksums
7f40fbf12657619ae6d4544ab28b189332b3dce1713b1d77a17a9b8b1967c1a9
BLAKE2b-256 checksum
How to use checksums
a2201653cad905e57e74236d8eb7c52e6ad23e50915a3c7a4d2ac06a95c90308
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 24, 2026.

Transparency log

Release files / aavaaz-0.10.0-py3-none-any.whl

Download URL aavaaz-0.10.0-py3-none-any.whl
Size 78.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b0fd74c3c2c25967ff41b870514eed72c970089129ab5b0d08585488666562ee
BLAKE2b-256 checksum
How to use checksums
ddd1040cb33a030eb27154ae70efc1c3d04643b73f2a58873ea1323207257895
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 24, 2026.

Transparency log

Security for aavaaz 0.10.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for aavaaz-0.10.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 24, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page