Senko

閃光 (senkō) - a flash of light

A very fast and accurate speaker diarization pipeline.

1 hour of audio processed in 5 seconds (RTX 4090 + Ryzen 9 7950X).

On Apple M3, 1 hour in 7.7 seconds.

The pipeline achieves a best score of 13.5% DER on VoxConverse, 13.3% on AISHELL-4, and 26.5% on AMI-IHM. See the evaluation directory for more benchmarks and comparison with other diarization systems.

Senko powers the Zanshin media player.

Usage

import senko

diarizer = senko.Diarizer(device='auto', warmup=True, quiet=False, model_dir=None)

wav_path = 'audio.wav' # 16kHz mono 16-bit wav
result = diarizer.diarize(wav_path, generate_colors=False)

senko.save_json(result["merged_segments"], 'audio_diarized.json')
senko.save_rttm(result["merged_segments"], wav_path, 'audio_diarized.rttm')

See examples/diarize.py for an interactive script, and also read DOCS.md

For growing recordings, see streaming diarization.

Senko can also be used in a notebook, like Google Colab and Modal Notebooks.

Model Directory

Senko resolves each required source model with the following precedence:

• Explicit model_dir= argument or script --model-dir flag
• SENKO_MODEL_DIR
• Bundled default model directory

If a model is missing from the configured model directory, Senko falls back to the bundled copy for that specific asset.

On macOS, Senko also stores reusable compiled CAM++ CoreML artifacts under <model_dir>/cached. That cache directory is disposable and can be deleted safely if it becomes stale or you want to reclaim disk space.

export SENKO_MODEL_DIR=/path/to/models
python examples/diarize.py --model-dir /path/to/override

Installation

The following instructions are for Linux, macOS, and WSL. For Windows, see WINDOWS.md.

Prerequisites:

• gcc/clang - on Linux/WSL, a separate install; on macOS, have the Xcode Command Line Tools installed
• uv

Create a Python virtual environment and activate it

uv venv --python 3.13 .venv
source .venv/bin/activate

Then install Senko

# For NVIDIA GPUs with CUDA compute capability >= 7.5 (~GTX 16 series and newer)
uv pip install "senko[nvidia]"

# For NVIDIA GPUs with CUDA compute capability < 7.5 (~GTX 10 series and older)
uv pip install "senko[nvidia-old]"

# For NVIDIA GPUs on native Windows with CUDA compute capability >= 7.5
uv pip install "senko[nvidia-windows]"

# For NVIDIA GPUs on native Windows with CUDA compute capability < 7.5
uv pip install "senko[nvidia-old-windows]"

# For Mac (macOS 14+) and CPU execution on all other platforms
uv pip install senko

For NVIDIA, make sure the installed driver is CUDA 12 capable (should see CUDA Version: 12+ in nvidia-smi). On Linux/WSL, a matching Kaldifeat CUDA wheel optionally enables GPU Fbank extraction.

PyPI alpha wheels are smoke-tested on GitHub-hosted runners, which cover packaging and CPU/default initialization but not GPU end-to-end execution.

For setting up Senko for development, see DEV_SETUP.md.

Accuracy

See the evaluation directory.

Technical Details

Senko is a heavily optimized and slightly modified version of the speaker diarization pipeline found in the excellent 3D-Speaker project. It consists of four stages: VAD (voice activity detection), Fbank feature extraction, speaker embeddings generation, and clustering (spectral or UMAP+HDBSCAN).

The following modifications have been made:

• VAD model has been swapped from FSMN-VAD to either Senko's local pyannote backend (powered by the bundled segmentation-3.0 assets) or Silero VAD
• Fbank feature extraction is done fully upfront with the bundled C++ extractor, or with Kaldifeat on GPU when installed.
• Batched inference of the CAM++ embedding model
• Clustering when on NVIDIA (with a GPU of CUDA compute capability 7.0+) can be done on the GPU through RAPIDS

On Linux/WSL, Senko's local segmentation-3.0 backend and CAM++ run using PyTorch, but on Mac, both models run through CoreML. The CAM++ CoreML conversion was done from scratch in this project (see tracing/coreml), but the segmentation-3.0 converted model and interfacing code is taken from the excellent FluidAudio project by Fluid Inference. No pyannote.audio package install is required at runtime.

Showcase

Create a PR or message on Discord if you'd like your application that uses Senko added here too.

FAQ

Troubleshooting

If you run into Numba related errors after upgrading/downgrading the numba package or other packages that use it (umap-learn, pynndescent, etc.), they might be caused by failed Numba cache invalidation. In such a case, clear the cache manually like so:

rm -rf ~/.cache/senko

Such errors may also appear if you have Zanshin installed, with different package versions installed in its Python environment compared to the development venv that you're using for Senko.

If you are using a custom model directory and want to clear reusable CoreML artifacts, delete the <model_dir>/cached directory. Senko will recreate it automatically on the next run if needed.

Community & Support

Join the Discord server to ask questions, suggest features, talk about Senko and Zanshin development etc.

Future Improvements & Directions

• Overlapping speaker segments support
• Improve speaker colors generation algorithm
• Support for Intel and AMD GPUs
• Experiment with torch.compile()
• Experiment with Modular MAX engine (faster CPU inference speed?)
• Background noise removal (DeepFilterNet), speech enhancement
• Live progress reporting
• VBx-based clustering (DiariZen)