CrisperWhisper 2.0

The most accurate verbatim speech recognition you can run in production: controllable, multilingual, and timed to the word.

Release post · Paper · Full documentation · Models · Benchmark

Most speech-to-text systems never actually decide whether to write down what was said or what was meant. They inherit that choice from their training data and apply it inconsistently. CrisperWhisper 2.0 makes it an explicit, controllable choice. One recording, two transcripts:

Verbatim, exactly what was said, in one consistent format: [um] so we we need to, to reschedule the th- thursday meeting to [uh] march third at nine thirty [laughter]

Intended, the clean version the speaker meant, with numbers, dates, and emails formatted the way you'd write them: So we need to reschedule the Thursday meeting to March 3 at 9:30.

On top of that:

• Word-level timings. Around 30 ms mean boundary error on read speech and 41 ms on conversational speech, the most precise word timing of any system we benchmarked, on both.
• Verbatimize. Upgrade transcripts you already have: given audio plus a trusted clean transcript, the model reproduces your content word-for-word and inserts only the disfluencies and vocal events actually present in the audio (rare-word recall jumps from 6.8% to 96.1% vs. re-transcribing). This turns the world's abundant clean corpora into verbatim ones, ready for TTS data, clinical speech analysis, and dataset construction.
• Multilingual. Verbatim and intended modes work across most languages Whisper supports. CrisperWhisper 2.0 tops the Nyra Verbatim Speech Benchmark leaderboard for disfluency F1 across ten languages, ahead of every closed-source alternative we tested.
• Seamless longform. Audio of any length, transcribed without the usual chunk-boundary artifacts: each window continues from the words already transcribed (conditional continuation), so there are no duplicated or dropped words at the seams and no fragile timestamp-token bookkeeping.
• Production inference. A CTranslate2 runtime with speculative decoding and built-in mitigation of Whisper's looping-hallucination failure mode.

Performance

The Nyra Verbatim Speech Benchmark scores fillers, repetitions, cut-offs, and vocal sounds as separate, typed metrics. Its headline number is disfluency F1: how reliably a system writes down the disfluencies that were actually spoken, without inventing ones that weren't. Averaged over ten languages:

_{* CrisperWhisper 1.0 is English/German-only; its average covers those two languages. English and German use human-labeled evaluation sets; the other eight languages use synthetic verbatim sets. Per-language breakdowns and how the metric is computed are in the benchmark post.}

Word-timing accuracy

Mean absolute word-boundary error on read speech (TIMIT), lower is better:

_{Scored on exactly the words each system gets right. How the timings are extracted from supervised cross-attention, plus results on conversational speech, are in the aligner post.}

Install

# NVIDIA GPU (Linux): fastest, includes speculative decoding.
# An NVIDIA driver is all you need; CUDA libraries arrive via pip.
pip install "crisperwhisper[ct2]>=2.1.0"

# Pure PyTorch: runs anywhere torch does (macOS, Windows, CPU)
pip install "crisperwhisper[transformers]>=2.1.0"

# Add [align] for the most precise English word timings (forced aligner)
pip install "crisperwhisper[ct2,align]>=2.1.0"

Quickstart

from crisperwhisper import CrisperWhisperModel

model = CrisperWhisperModel()          # nyralabs/CrisperWhisper2.0_large
# or pick a size: CrisperWhisperModel("turbo")  # turbo / medium / small

# Verbatim transcription (default): every filler, repetition, stutter,
# false start, and vocal event
result = model.transcribe("meeting.wav", language="en")
print(result.text)

# Intended: the clean, readable version
clean = model.transcribe("meeting.wav", language="en", mode="intended")

# Word-level timestamps (and confidences)
result = model.transcribe("meeting.wav", language="en", word_confidence=True)
for w in result.words:
    print(f"{w.start:6.2f}-{w.end:6.2f}  {w.word}  p={w.probability:.2f}")

# Verbatimize: upgrade an existing clean transcript with the
# disfluencies that are actually in the audio
result = model.verbatimize("clip.wav", "I think we should ship it Friday.")

Audio longer than 30 seconds is handled automatically (see longform below). The first load of a model downloads it from HuggingFace and, on the ct2 backend, converts it once into a local cache. That one-time conversion needs torch and transformers, which the lean [ct2] extra does not install -- add them once with:

pip install "crisperwhisper[ct2,convert]"   # or add [convert] later

A CPU-only torch is enough for the conversion, and the dependencies can be uninstalled once the converted model is cached. Loading a directory that already contains a converted CT2 model (model.bin) never needs them.

Models

The standard models are released under a non-commercial research license and are available for commercial licensing. The Pro models are available under commercial license only. For both, get in touch.

Most precise word timings (English): the forced aligner

For English, the most precise word timings come from the nyra forced aligner, installed with the [align] extra (pip install "crisperwhisper[ct2,align]"). CrisperWhisper transcribes as usual, then the aligner re-times the same words against the audio. result.words keeps its format and gains timing_confidence, kind (word / filler / sound / cutoff) and phone-level phones:

from crisperwhisper import CrisperWhisperModel, ForcedAligner

model = CrisperWhisperModel("large")
result = model.transcribe(
    "meeting.wav",
    language="en",
    timing_method="forced_aligner",   # implies word_timestamps=True
    word_confidence=True,             # optional: ASR word probabilities too
)
for w in result.words:
    print(f"{w.start:7.3f}-{w.end:7.3f}  {w.word:14} {w.kind:7} "
          f"timing_conf={w.timing_confidence:.2f}")

# Noisy audio: the noise-robust aligner model
model = CrisperWhisperModel("large", forced_aligner=ForcedAligner(pro=True))

# Long recordings with many pauses: about 2x faster alignment, slightly
# less precise (aligns each stretch between >5 s pauses separately)
model = CrisperWhisperModel("large", forced_aligner=ForcedAligner(segment_gap=5.0))

# A transcript you already have, no ASR model needed
result = ForcedAligner().align("clip.wav", "so I [UM] went there")

The forced aligner supports English today, and we will add other languages soon. Until then, timing_method="forced_aligner" with another language raises a warning and keeps the attention timings, which work for every language.

Faster inference: speculative decoding (ct2)

A small draft model proposes tokens and the main model verifies them. Same output, 1.3 to 1.4x faster:

model = CrisperWhisperModel("large", draft_model="turbo")
result = model.transcribe("meeting.wav", language="en",
                          speculative_decoding=True)

What else is in the box

Everything below works out of the box and is covered in depth in DOCS.md:

How it works

Each mechanism has a deep-dive post:

• Measuring verbatimness: the Nyra Verbatim Speech Benchmark. Typed metrics for fillers, repetitions, cut-offs, and vocal sounds instead of one opaque WER number.
• How we unlocked multilingual style-controlled transcription at scale. Verbatim/intended control across languages.
• Turning emergent cross-attention into a precise aligner. Supervising Whisper's alignment heads for word boundaries around 30 ms.
• Longform transcription with conditional continuation. Resolving window seams with the words already transcribed instead of fragile timestamp tokens.
• Closing the verbatim data gap with Verbatimize. Upgrading clean corpora into verbatim ones at scale.
• Faster inference and mitigating hallucinations. The CTranslate2 stack, speculative decoding, and the anti-looping decoder.

Documentation

DOCS.md covers the full API: backends and their trade-offs, every transcribe() option (including word timestamps, timing methods and confidence), dual-mode transcription, forced alignment, longform strategies, speculative-K tuning, hallucination-repair thresholds, quantization, model conversion, and the result object.

License

The inference code in this repository is MIT-licensed (see LICENSE): use it freely, commercially or otherwise. crisperwhisper/features.py is vendored from faster-whisper (MIT, SYSTRAN).

The model weights are not MIT. They are released under the Nyra Health Non-Commercial Research License: free for research and other non-commercial use; any commercial use requires a commercial license. The Pro models are available under commercial license only. For commercial licensing of either, contact Nyra.