AudioQL

A semantic layer for audio.

AudioQL wraps Whisper, pyannote, YAMNet and friends behind one small API. You point it at an audio file and get a semantic timeline — a single stream of events ("speech", "laughter", "music", "silence", ...) you can query with plain English:

from audioql import Audio

audio = Audio("meeting.mp3")

audio.find("laughter")               # all the laughs
audio.find("music after applause")   # sequencing works too
audio.find("speaker:SPEAKER_00")     # one speaker's turns
audio.search("someone discussing AI")  # keyword search over the transcript

No model-specific APIs leak out. Every backend produces the same TimelineEvent, so swapping Whisper for something else doesn't change your code.

Install

The core package is small and has no AI dependencies:

pip install audioql

Model backends are optional extras — install only what you use:

pip install "audioql[whisper]"        # speech-to-text
pip install "audioql[diarization]"    # speaker diarization
pip install "audioql[events]"         # laughter, applause, music detection
pip install "audioql[all]"            # everything

Quickstart

from audioql import Audio

audio = Audio("meeting.mp3")

print(audio.duration)                  # seconds
print(audio.sample_rate)

transcript = audio.transcript()        # needs audioql[whisper]
print(transcript.text)
for seg in transcript.segments:
    print(seg.start, seg.end, seg.text)

print(audio.speakers())                # needs audioql[diarization]

for event in audio.find("laughter"):
    print(event.start, event.end, round(event.confidence, 2))

Analysis runs lazily and the results are cached, so calling timeline(), find(), transcript() and search() on the same Audio only analyzes the file once.

Configuration

Backends are configured with keyword arguments to Audio:

import os

audio = Audio(
    "meeting.mp3",
    whisper={"model": "small", "device": "cpu"},
    diarization={"hf_token": os.getenv("HF_TOKEN")},
)

hf_token also falls back to the HF_TOKEN environment variable, so most setups just work.

Hugging Face setup (required for diarization)

Speaker diarization downloads several models from the Hugging Face Hub, and two of them are gated: you need a Hugging Face account and the model owners must approve your access before the models can be downloaded. This is a one-time setup:

1. Create a free account at huggingface.co/join.

2. Open each model page below and click Agree and access repository to accept the user conditions:

   • pyannote/speaker-diarization-3.1 — the diarization pipeline (required)
   • pyannote/segmentation-3.0 — the speaker segmentation model the pipeline loads (required)

   The speaker embedding model (pyannote/wespeaker-voxceleb-resnet34-LM) is public and needs no signup. If you instead use the community edition (pyannote/speaker-diarization-community-1), that repo is gated too and needs the same approval.

3. Create an access token at huggingface.co/settings/tokens (a Read token is enough), then make it available to AudioQL. Either set the HF_TOKEN environment variable:

   # Windows (PowerShell)
   setx HF_TOKEN "hf_xxxx"
   
   # macOS / Linux
   export HF_TOKEN="hf_xxxx"
   

   or pass it directly to Audio:

   audio = Audio("meeting.mp3", diarization={"hf_token": "hf_xxxx"})
   

Transcript-only usage

transcript() also runs speaker diarization to attribute each segment to a speaker whenever the diarization extra is installed. If you only want the transcript (or haven't done the Hugging Face setup yet), disable the speaker plugin:

audio = Audio("meeting.mp3", disabled_plugins=["speaker"])
transcript = audio.transcript()

How it works

audio file
  → loader                 (decodes to mono float32)
  → plugins                (speech, speaker, events, silence)
  → timeline builder       (merges events, fills gaps, detects questions)
  → query engine           (find / search over the timeline)

Plugins are auto-enabled when their dependencies are installed, and disabled when they're not. You can also pass plugins and disabled_plugins to take full control.

Development

python -m venv .venv
.venv\Scripts\activate     # or: source .venv/bin/activate
pip install -e ".[dev]"
pytest

License

MIT