whisper-guard

English ｜ 繁體中文

whisper-guard is a small Python package that removes common Whisper hallucinations before you ship subtitles or transcripts downstream.

Problem

Whisper output can degrade on silence-heavy or low-confidence clips:

• repeated phrases
• phantom subtitles on silence
• short character loops such as 哈哈哈哈 or xyzxyzxyz

This package extracts the anti-hallucination logic from arkiv into a reusable package with a minimal API.

4-Layer Guard

A/B Test Results

These numbers are from the arkiv guard benchmark set on April 2026.

Accuracy Benchmark

⚠️ This is a curated fixture benchmark, not an independent third-party one. The labels reflect what we already expect the guard to catch, so its value is transparency + regression guard (numbers move if a change starts eating real speech or letting hallucinations through), not a leaderboard score. Reproduce with python bench/benchmark.py; invariants are pinned in tests/test_bench.py.

On a hand-labelled corpus modelling real Chinese Whisper output (genuine speech mixed with each hallucination type):

Limitations

whisper-guard is metric/pattern-driven — it reasons over no_speech_prob, avg_logprob, compression_ratio, and character-loop patterns. It therefore cannot catch a fluent phantom: a single, grammatical hallucinated sentence with healthy metrics (e.g. 請按讚訂閱開啟小鈴鐺 emitted over silence) is indistinguishable from real speech by metrics alone.

→ Those are caught by upstream VAD (voice-activity detection). By design, whisper-guard pairs with VAD: VAD removes silence-phantoms, whisper-guard removes repetition / low-confidence / loop hallucinations. The 80% recall above honestly reflects that division of labour.

Install

pip install whisper-guard

For local development:

pip install -e .

Quick Start

from faster_whisper import WhisperModel
from whisper_guard import WhisperGuard

model = WhisperModel("small")
segments, info = model.transcribe("sample.wav")

guard = WhisperGuard()
result = guard.process([segment._asdict() for segment in segments])

if result.passed:
    print(result.text)

API

from whisper_guard import WhisperGuard, GuardConfig, filter_hallucinations

WhisperGuard.process() expects segment dictionaries shaped like:

{
    "text": "hello world",
    "no_speech_prob": 0.12,
    "avg_logprob": -0.44,
    "compression_ratio": 1.2,
    "start": 0.0,   # optional — enables dynamic logprob threshold
    "end": 2.5,      # optional — enables dynamic logprob threshold
}

When start/end are provided, segments shorter than 1.6s use a more lenient logprob threshold (-1.7 vs -1.5) — genuine brief utterances naturally score lower confidence, so loosening the bar avoids wrongly dropping real short speech. Segments without timing info fall back to the normal threshold.

Compatible With

• faster-whisper
• openai-whisper
• mlx-whisper

Optional Vocab Helpers

from whisper_guard.vocab import build_hotwords_prompt, filter_filler_words

Part Of

Built for the arkiv transcription pipeline and split out as a standalone package for reuse.

License

MIT