seprq 0.4.3

Université de Toulon · LIS · CNRS · ILLS · pyannoteAI

📄 Paper  ·  🌐 Website  ·  🤗 Hugging Face  ·  💻 GitHub

🗣️ This framework is intended to provide an easy, ready-to-use and strong SepRQ feature extractor for 🍸 cocktail-party 🍸 and multi-talker conversational scenarios 🔀 👥, while also facilitating the use of other feature extractors from the literature. We also release a BEST-RQ model trained at 50 Hz as a valuable contribution to the community.

SepRQ features are pulled from 🤗 SevKod/SepRQ; a call returns the 12 Conformer layer features usable for training or cocktail-party representation analysis. Requires PyTorch and upstream speechbrain>=1.0.3; runs on CPU or GPU.

👉 Please visit https://sevkod.github.io/SepRQ/ for the full usage and demo.

Install

pip install seprq

Usage (frozen ❄️ or fine-tuned 🔥)

from seprq import SepRQEncoder

wav = "hf.co/SepRQ/SepRQ/blob/main/ex/mix.wav"
# wav = torchaudio.load("path/to/audio.wav")

# "SepRQ" — see "Available models" for every name
encoder = SepRQEncoder("SepRQ",
                       streams=2)

# ❄️ frozen feature extraction
encoder.requires_grad_(False).eval()  # or True for 🔥 fine-tuning
features = encoder(wav)

It accepts a file path (any format/rate — decoded, mono, resampled to 16 kHz) or a waveform tensor [num_samples], [batch, num_samples] or [batch, channel, num_samples].

SepRQ ships in variants trained to separate a different number of speakers — select one with streams (default 2):

encoder = SepRQEncoder("SepRQ",
                       streams=3)
# 3-speaker variant

Available models

The SepRQ / BEST-RQ weights come from the Hub (private — authenticate once with huggingface-cli login or HF_TOKEN); the HuBERT / WavLM baselines are fetched from the torchaudio pipelines, nothing extra to host.

Run a task

SUPERB separation, diarization, and enhancement, plus target-speaker extraction, personalized VAD, and target-speaker ASR. The head is chosen from the upstream. Published for SepRQ (streams=2), HuBERT_BASE, WavLM_BASE, and WavLM_BASE_PLUS. Target-speaker tasks take an enrollment utterance.

from seprq import SepRQPipeline

wav = "hf.co/SepRQ/SepRQ/blob/main/ex/mix.wav"
noisy = "hf.co/SepRQ/SepRQ/blob/main/ex/noisy.wav"

save_to = "./output"

# Separation — writes ./output/0.wav, ./output/1.wav
pipe = SepRQPipeline("separation",
                     upstream="SepRQ",
                     streams=2)
sources = pipe(wav,
               save_to=save_to)
# [num_speakers, num_samples]

# Diarization — writes ./output/0.rttm
pipe = SepRQPipeline("diarization",
                     upstream="SepRQ",
                     streams=2)
activations = pipe(wav,
                   save_to=save_to)
# [time, num_speakers]

# Enhancement — writes ./output/0.wav
pipe = SepRQPipeline("enhancement",
                     upstream="SepRQ",
                     streams=2)
clean = pipe(noisy,
             save_to=save_to)
# [num_samples]

# Target speaker extraction — writes ./output/0.wav
mix = "hf.co/SepRQ/SepRQ/blob/main/ex/target-speaker-extraction/1188-133604-0025_4992-23283-0016/mixture.wav"
enrollment = "hf.co/SepRQ/SepRQ/blob/main/ex/target-speaker-extraction/1188-133604-0025_4992-23283-0016/enrollment.wav"
pipe = SepRQPipeline("target-speaker-extraction",
                     upstream="SepRQ",
                     streams=2)
target = pipe(mix,
              enrollment=enrollment,
              save_to=save_to)
# [num_samples]

# Personalized VAD — writes ./output/0.rttm
mix = "hf.co/SepRQ/SepRQ/blob/main/ex/personalized-vad/mix_0000001/mixture.wav"
enrollment = "hf.co/SepRQ/SepRQ/blob/main/ex/personalized-vad/mix_0000001/enrollment.wav"
pipe = SepRQPipeline("personalized-vad",
                     upstream="SepRQ",
                     streams=2)
activations = pipe(mix,
                   enrollment=enrollment,
                   save_to=save_to)
# [time]

# Target-speaker ASR — writes ./output/0.txt
mix = "hf.co/SepRQ/SepRQ/blob/main/ex/target-speaker-asr/4446-2271-0014_3570-5694-0003/mixture.wav"
enrollment = "hf.co/SepRQ/SepRQ/blob/main/ex/target-speaker-asr/4446-2271-0014_3570-5694-0003/enrollment.wav"
pipe = SepRQPipeline("target-speaker-asr",
                     upstream="SepRQ",
                     streams=2)
text = pipe(mix,
            enrollment=enrollment,
            save_to=save_to)

🔀 SepRQ — separation as the pretext task

SepRQ swaps masked prediction for pseudo source separation. It predicts one stream of discrete units per speaker in the mixture, at several temporal resolutions.

Fig. 1. SepRQ performs pseudo source separation in the discrete space. Random vector quantizers (RVQs) give speaker-specific discrete labels at several resolutions.

• Pseudo source separation — each separation head predicts the frozen BEST-RQ codewords of one clean speaker from the mixture. Utterance-level PIT resolves speaker order.
• Mask-free — masking can hide exactly the content needed to disentangle a mixture. SepRQ predicts units over the full utterance, which uses every frame for training.
• Multi-resolution — a separation objective follows every two Conformer layers, with frame folding going from 20 ms to 320 ms and one codebook per scale.

👥 Results

Frozen upstreams with SUPERB / TS-SUPERB downstream heads.

Multi-speaker SUPERB

Speaker diarization (SD), speech separation (SS) and speech enhancement (SE). LS = LibriSpeech, LL = Libri-Light, Mix = LL-60k + GigaSpeech-10k + VoxPopuli-24k.

TS-SUPERB

Target speaker extraction (TSE), personalized speech extraction (PSE), personalized VAD (PVAD), target-speaker ASR (TS-ASR).

Out-of-domain

EEND diarization on DIHARD 3 and ConvTasNet separation on WSJ0-2/3mix.

Citation

If SepRQ helps your research, please cite the paper: arXiv:2610.04690.

@misc{baroudi2026seprqselfsupervisedspeech,
  title         = {SepRQ : Self-Supervised Speech Mixture Representation Learning
                   via Mask-Free, Multi-Scale Source Separation},
  author        = {Séverin Baroudi and Hervé Bredin and Ricard Marxer},
  year          = {2026},
  eprint        = {2610.04690},
  archivePrefix = {arXiv},
  primaryClass  = {eess.AS},
  url           = {https://arxiv.org/abs/2610.04690}
}