Normalization
A lightweight library for normalizing speech transcripts before computing WER.
Quick Start · Documentation · Contributing
Why normalization matters
Word Error Rate (WER) is the standard metric for evaluating speech-to-text systems. But WER operates on raw strings — it has no notion of meaning. Two transcriptions that say the same thing in different surface forms get penalized as errors:
| Ground truth | STT output | Match without normalization |
|---|---|---|
| It's $50 | it is fifty dollars | 0/3 words match |
| 3:00 PM | 3 pm | 0/2 words match |
| Mr. Smith | mister smith | 0/2 words match |
These aren't transcription errors — they're formatting differences. Without normalization, WER scores become unreliable and comparisons across engines are meaningless.
gladia-normalization solves this by reducing both the ground truth and the STT output to a shared canonical form before WER is computed, so that only genuine recognition errors affect the score.
Used in Gladia's benchmarks
This library is the normalization step behind the WER figures Gladia publishes. Every reference and every provider's output goes through the gladia-3 preset before WER is computed, so engines are compared on recognition errors, not formatting.
- Speech-to-text benchmarks: WER per dataset and provider.
- Speech-to-text API comparison: blind test and ELO leaderboard.
What it does
The library runs your text through a configurable pipeline of normalization steps — expanding contractions, converting symbols to words, removing fillers, casefolding, and more — to produce a clean, canonical output.
Input: "It's $50.9 at 3:00PM — y'know, roughly."
Output: "it is 50 point 9 dollars at 3 pm you know roughly"
The pipeline is deterministic, language-aware, and fully defined in YAML — run the same preset and get the same output every time.
Quick start
Installation
pip install gladia-normalization
Install from source
git clone https://github.com/gladiaio/normalization.git
cd normalization
uv sync
Usage
from normalization import load_pipeline
# Load a built-in preset by name
pipeline = load_pipeline("gladia-3", language="en")
pipeline.normalize("It's $50 at 3:00PM")
# => "it is 50 dollars at 3 pm"
CLI
# Normalize a single text
gladia-normalization "A tea cost £2 at 3:30PM" --language en
# Normalize a text file
gladia-normalization --file transcript.txt --language en
# Pipe from stdin
echo "A tea cost £2 at 3:30PM" | gladia-normalization --language fr
# Use a custom preset
gladia-normalization "A tea cost £2 at 3:30PM" --preset path/to/my-preset.yaml --language en
# Inspect the pipeline
gladia-normalization --describe --language en
If you don't want a permanent installation, run it directly with uvx:
uvx gladia-normalization "A tea cost £2 at 3:30PM" --language en
Compute WER
Normalize both the reference and the hypothesis with the same pipeline, then score them. This example uses jiwer:
pip install gladia-normalization jiwer
import jiwer
from normalization import load_pipeline
pipeline = load_pipeline("gladia-3", language="en")
reference = "It's $50 at 3:00PM, Mr. Smith."
hypothesis = "it is fifty dollars at 3 pm mister smith"
jiwer.wer(reference, hypothesis)
# => 1.33 (formatting differences counted as errors)
jiwer.wer(pipeline.normalize(reference), pipeline.normalize(hypothesis))
# => 0.0
How it works
Every pipeline runs exactly three stages, always in this order:
- Stage 1 — Text pre-processing: Full-text transforms: protect symbols, expand contractions, convert numbers, casefold, remove symbols
- Stage 2 — Word processing: Per-token transforms: word replacements, filler removal
- Stage 3 — Text post-processing: Full-text cleanup: restore placeholders, collapse digits, format time patterns, normalize whitespace
This ordering is a hard constraint. Some steps depend on earlier steps having run (e.g. a placeholder protecting a decimal point in Stage 1 must be restored in Stage 3, so that remove_symbols doesn't destroy it in between).
Pipelines are defined declaratively in YAML presets. Each preset lists the steps that run in each stage and the order they run in. See the full step reference for every available step.
Supported languages
| Code | Language |
|---|---|
ar |
Arabic |
da |
Danish |
de |
German |
en |
English |
es |
Spanish |
fi |
Finnish |
fr |
French |
it |
Italian |
nl |
Dutch |
no |
Norwegian |
pt |
Portuguese |
ro |
Romanian |
sv |
Swedish |
Unsupported language codes fall back to a safe default that applies language-independent normalization only.
Adding a new language is self-contained — create a folder, register it with a decorator, done. See Contributing.
Custom presets
A preset is a YAML file that declares which steps run in each stage and in what order.
name: my-preset-v1
stages:
text_pre:
- protect_email_symbols
- expand_contractions
- casefold_text
- remove_symbols
- remove_diacritics
word:
- apply_word_replacements
text_post:
- restore_email_at_symbol_with_word
- restore_email_dot_symbol_with_word
Load from your custom configuration:
from normalization import load_pipeline
pipeline = load_pipeline("path/to/my-custom-configuration.yaml", language="en")
result = pipeline.normalize("some transcription text")
Inspect a loaded pipeline:
pipeline.describe()
# {'name': 'my-preset-v1', 'language': 'en', 'text_pre_steps': [...], ...}
Preset rules:
- Step names must match the
nameattribute of a registered step class. - Every
protect_*step intext_prerequires a matchingrestore_*intext_post. The pipeline validates this at load time. - List order is execution order.
- Published presets are immutable — new behavior means a new file.
Contributing
Bug reports, new steps, and new language support are all welcome. See CONTRIBUTING.md for the full guide — including how to add steps, add languages, write tests, and the commit style we follow.
Development
uv run pre-commit install --install-hooks # install hooks once after cloning
uv run pytest # run tests
uv run ruff check . # lint
uv run ruff format . # format
uv run ty check # type-check
About
gladia-normalization grew out of internal tooling at Gladia, where we are building an audio intelligence platform powered by speech recognition. When benchmarking ASR systems, we kept hitting the same problem: computing WER from raw transcript penalizes formatting differences that have nothing to do with the quality itself. We built this library to solve it for ourselves, then open-sourced it so the broader speech community doesn't have to solve it again.
Sharing it felt like the right next step: the problem is universal, and community contributions are the best way to make reliable normalization available for every language, not just the ones we support today.
License
Metadata
Release files for gladia-normalization 0.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| gladia_normalization-0.4.0.tar.gz | 257.0 kB | Without provenance | Details for gladia_normalization-0.4.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| gladia_normalization-0.4.0-py3-none-any.whl | Python 3 | none | any | Details for gladia_normalization-0.4.0-py3-none-any.whl |
Total release size: 398.5 kB
Release files / gladia_normalization-0.4.0.tar.gz
| Download URL | gladia_normalization-0.4.0.tar.gz |
|---|---|
| Size | 257.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ee112899f5dc1a6315ddb58aa8a547543f4cb8a99460b4e9b9951d183a99d3b7
|
|
BLAKE2b-256 checksum How to use checksums |
58961cd36a55aea7a3f3c2dcf294b301c23c684301cd3b16bf8ad05caa4f7582
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / gladia_normalization-0.4.0-py3-none-any.whl
| Download URL | gladia_normalization-0.4.0-py3-none-any.whl |
|---|---|
| Size | 141.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c9783f960827fec68c61b6dc225043dd74b0b7e3c32c22cb2463c6bebab79290
|
|
BLAKE2b-256 checksum How to use checksums |
b78967d53b6f02d8bf0b6ceb07dd20300a7982fe12dcb4534d10da74305a87fe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for gladia-normalization 0.4.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.