voi-oiwer 0.1.0

Open-lattice Indic Word Error Rate (OIWER) with a fast Rust core

voi_oiwer

Open-lattice Indic Word Error Rate (OIWER) for ASR evaluation across Indian languages, with a fast Rust core.

Unlike plain WER, OIWER scores a hypothesis against a lattice of accepted reference variations per word/phrase, so legitimate spelling / transliteration variants aren't counted as errors.

Install

pip install voi_oiwer

Works on CPython 3.8+ (Linux, macOS, Windows). Prebuilt wheels — no Rust toolchain needed.

Usage

from voi_oiwer import oiwer

score, final_h, final_r, final_ops, (ins, dele, sub), \
    total_ref_words, metadata, std_refs = oiwer(
        hypothesis="नमस्ते दुनिया",
        reference_lists=[["नमस्ते"], ["दुनिया", "विश्व"]],   # lattice: variations per slot
        input_language="hindi",
    )

print(score)   # OIWER, as a percentage

Arguments

arg	type	description
hypothesis	str	the ASR output text
reference_lists	list[list[str]]	the reference lattice — one list of accepted variations per slot (a variation may be multiple words)
input_language	str	a language name, e.g. "hindi", "tamil", "urdu" (see list below)

Returns

A tuple:

(oiwer_score,                    # float — error percentage
 final_h,                        # list[str] — aligned hypothesis words
 final_r,                        # list[str] — aligned reference words (" " = insertion)
 final_ops,                      # list[str] — per slot: 'c' correct, 's' sub, 'd' del, 'i' ins
 (insertions, deletions, substitutions),
 total_reference_words,          # int
 final_metadata,                 # list — [lat_idx, var_idx, word_idx] per reference word (None for insertions)
 standardized_reference_lists)   # list[list[str]] — the normalized lattice used internally

Supported languages

assamese, bengali, bodo, dogri, gujarati, hindi, kannada, kashmiri, konkani, maithili, malayalam, manipuri, marathi, nepali, odia, punjabi, sanskrit, santali, sindhi, tamil, telugu, urdu, chattisgarhi, bhojpuri.

How it works

Text normalization (indicnlp-based) runs in Python; the edit-distance alignment over the lattice runs in a compiled Rust core — so results are identical to the reference implementation while running several times faster.

Development

pip install maturin
maturin develop --release   # build the Rust core into your active venv
pytest -q                   # run the test suite

License

MIT © JoshTalks