ASRX: Universal Forced Alignment & Speech Segmentation

ASRX is a high-performance, model-agnostic Forced Alignment, VAD Segmentation, and Speaker Diarization engine.

It allows developers to take text generated by ANY external ASR model, LLM, or API (e.g. Cohere, Qwen, Whisper, vLLM, OpenAI, Deepgram, or custom local scripts) and immediately generate:

• ⏱️ Exact millisecond-level word timestamps (start, end)
• 🎯 Word-level confidence scores
• 🎙️ VAD-guided natural speech segment grouping
• 👥 Speaker Diarization
• ⚡ Ultra-fast processing

🚀 Quickstart

1. Installation

ASRX uses a lazy-loaded backend architecture. You only install the heavy ML dependencies for the specific models you want to use.

# Core installation (fast, no heavy dependencies)
pip install -e asrx

# 1. Install for Wav2Vec2/MMS alignment (Recommended)
pip install -e asrx[transformers]

# 2. Install for Pyannote VAD and Diarization
pip install -e asrx[pyannote]

# 3. Install for NVIDIA NeMo (Aligner, VAD, Sortformer)
pip install -e asrx[nemo]

# 4. Install for CTC-Segmentation (Confidence scores)
pip install -e asrx[ctc,transformers]

2. Standalone Alignment in 1 Line (asrx.align)

import asrx

audio_file = "meeting.wav"
my_text = "I don't have a problem with where we go, the most important thing is that the place is quiet, has good coffee, and we can sit comfortably and talk."

# Generates word timestamps, confidence scores, and VAD-guided segments
result = asrx.align(
    audio=audio_file,
    text=my_text,
    language="en",
    aligner="wav2vec2", # "wav2vec2", "mms", "nemo", "ctc_segmentation"
    vad="silero"        # "silero", "pyannote", "nemo", "flashvad"
)

print("Total words aligned:", len(result["word_segments"]))
print("First word timestamp:", result["word_segments"][0])

🧰 Backend Registry

Because ASRX is entirely modular, you can mix and match the best tools for your specific use case.

Alignment Backends (aligner=...)

VAD Backends (vad=...)

Diarization Backends (diarize=...)

🛠️ Advanced Usage

1. Alignment with SOTA Diarization (NeMo Sortformer)

import asrx

result = asrx.align(
    audio="panel_discussion.wav",
    text=transcript_text,
    language="en",
    vad="nemo",
    diarize="sortformer"
)

# Words and segments will have "SPEAKER_00", "SPEAKER_01", etc.
for seg in result["segments"]:
    print(f"[{seg['speaker']}] {seg['start']}s -> {seg['end']}s: {seg['text']}")

2. Custom Pyannote Models (e.g., Community-1)

result = asrx.align(
    audio="audio.wav",
    text=my_text,
    language="ar",
    diarize="pyannote",
    model_name="pyannote/speaker-diarization-community-1",
    hf_token="your_hf_token"
)

3. Reusable Pipeline for High-Throughput Batching

import asrx

# Load pipeline once into memory
pipeline = asrx.load_aligner(
    language="en",
    aligner="mms",
    vad="pyannote",
    diarize="sortformer",
    device="cuda",
    hf_token="your_hf_token"
)

# Process multiple audio files efficiently
for audio_path, text in dataset:
    result = pipeline.align(audio=audio_path, text=text)

📊 Standardized JSON Output

{
  "segments": [
    {
      "start": 0.12,
      "end": 7.15,
      "text": "I don't have a problem with where we go, the most important thing is that the place is quiet.",
      "speaker": "SPEAKER_00",
      "words": [
        {
          "word": "I",
          "start": 0.12,
          "end": 0.34,
          "score": 0.769,
          "speaker": "SPEAKER_00"
        },
        {
          "word": "don't",
          "start": 0.381,
          "end": 0.461,
          "score": 0.976,
          "speaker": "SPEAKER_00"
        }
      ]
    }
  ],
  "word_segments": [
    {
      "word": "I",
      "start": 0.12,
      "end": 0.34,
      "score": 0.769,
      "speaker": "SPEAKER_00"
    },
    {
      "word": "don't",
      "start": 0.381,
      "end": 0.461,
      "score": 0.976,
      "speaker": "SPEAKER_00"
    }
  ],
  "language": "en"
}

🌍 Supported Alignment Languages

Supports Arabic (ar), English (en), French (fr), German (de), Spanish (es), Italian (it), Japanese (ja), Chinese (zh), Portuguese (pt), Russian (ru), Turkish (tr), Hindi (hi), Korean (ko), and 1000+ other languages via MMS and Multilingual models.

📄 License

Apache-2.0