Skip to main content

Normalization

A lightweight library for normalizing speech transcripts before computing WER.
Quick Start · Documentation · Contributing

PyPI version Python versions License


Why normalization matters

Word Error Rate (WER) is the standard metric for evaluating speech-to-text systems. But WER operates on raw strings — it has no notion of meaning. Two transcriptions that say the same thing in different surface forms get penalized as errors:

Ground truth STT output Match without normalization
It's $50 it is fifty dollars 0/3 words match
3:00 PM 3 pm 0/2 words match
Mr. Smith mister smith 0/2 words match

These aren't transcription errors — they're formatting differences. Without normalization, WER scores become unreliable and comparisons across engines are meaningless.

gladia-normalization solves this by reducing both the ground truth and the STT output to a shared canonical form before WER is computed, so that only genuine recognition errors affect the score.

Used in Gladia's benchmarks

This library is the normalization step behind the WER figures Gladia publishes. Every reference and every provider's output goes through the gladia-3 preset before WER is computed, so engines are compared on recognition errors, not formatting.

What it does

The library runs your text through a configurable pipeline of normalization steps — expanding contractions, converting symbols to words, removing fillers, casefolding, and more — to produce a clean, canonical output.

Input:  "It's $50.9 at 3:00PM — y'know, roughly."
Output: "it is 50 point 9 dollars at 3 pm you know roughly"

The pipeline is deterministic, language-aware, and fully defined in YAML — run the same preset and get the same output every time.

Quick start

Installation

pip install gladia-normalization
Install from source
git clone https://github.com/gladiaio/normalization.git
cd normalization
uv sync

Usage

from normalization import load_pipeline

# Load a built-in preset by name
pipeline = load_pipeline("gladia-3", language="en")

pipeline.normalize("It's $50 at 3:00PM")
# => "it is 50 dollars at 3 pm"

CLI

# Normalize a single text
gladia-normalization "A tea cost £2 at 3:30PM" --language en

# Normalize a text file
gladia-normalization --file transcript.txt --language en

# Pipe from stdin
echo "A tea cost £2 at 3:30PM" | gladia-normalization --language fr

# Use a custom preset
gladia-normalization "A tea cost £2 at 3:30PM" --preset path/to/my-preset.yaml --language en

# Inspect the pipeline
gladia-normalization --describe --language en

If you don't want a permanent installation, run it directly with uvx:

uvx gladia-normalization "A tea cost £2 at 3:30PM" --language en

Compute WER

Normalize both the reference and the hypothesis with the same pipeline, then score them. This example uses jiwer:

pip install gladia-normalization jiwer
import jiwer
from normalization import load_pipeline

pipeline = load_pipeline("gladia-3", language="en")

reference = "It's $50 at 3:00PM, Mr. Smith."
hypothesis = "it is fifty dollars at 3 pm mister smith"

jiwer.wer(reference, hypothesis)
# => 1.33 (formatting differences counted as errors)

jiwer.wer(pipeline.normalize(reference), pipeline.normalize(hypothesis))
# => 0.0

How it works

Every pipeline runs exactly three stages, always in this order:

  • Stage 1 — Text pre-processing: Full-text transforms: protect symbols, expand contractions, convert numbers, casefold, remove symbols
  • Stage 2 — Word processing: Per-token transforms: word replacements, filler removal
  • Stage 3 — Text post-processing: Full-text cleanup: restore placeholders, collapse digits, format time patterns, normalize whitespace

This ordering is a hard constraint. Some steps depend on earlier steps having run (e.g. a placeholder protecting a decimal point in Stage 1 must be restored in Stage 3, so that remove_symbols doesn't destroy it in between).

Pipelines are defined declaratively in YAML presets. Each preset lists the steps that run in each stage and the order they run in. See the full step reference for every available step.

Supported languages

Code Language
ar Arabic
da Danish
de German
en English
es Spanish
fi Finnish
fr French
it Italian
nl Dutch
no Norwegian
pt Portuguese
ro Romanian
sv Swedish

Unsupported language codes fall back to a safe default that applies language-independent normalization only.

Adding a new language is self-contained — create a folder, register it with a decorator, done. See Contributing.

Custom presets

A preset is a YAML file that declares which steps run in each stage and in what order.

name: my-preset-v1

stages:
  text_pre:
    - protect_email_symbols
    - expand_contractions
    - casefold_text
    - remove_symbols
    - remove_diacritics

  word:
    - apply_word_replacements

  text_post:
    - restore_email_at_symbol_with_word
    - restore_email_dot_symbol_with_word

Load from your custom configuration:

from normalization import load_pipeline

pipeline = load_pipeline("path/to/my-custom-configuration.yaml", language="en")
result = pipeline.normalize("some transcription text")

Inspect a loaded pipeline:

pipeline.describe()
# {'name': 'my-preset-v1', 'language': 'en', 'text_pre_steps': [...], ...}

Preset rules:

  • Step names must match the name attribute of a registered step class.
  • Every protect_* step in text_pre requires a matching restore_* in text_post. The pipeline validates this at load time.
  • List order is execution order.
  • Published presets are immutable — new behavior means a new file.

Contributing

Bug reports, new steps, and new language support are all welcome. See CONTRIBUTING.md for the full guide — including how to add steps, add languages, write tests, and the commit style we follow.

Development

uv run pre-commit install --install-hooks   # install hooks once after cloning
uv run pytest              # run tests
uv run ruff check .        # lint
uv run ruff format .       # format
uv run ty check            # type-check

About

gladia-normalization grew out of internal tooling at Gladia, where we are building an audio intelligence platform powered by speech recognition. When benchmarking ASR systems, we kept hitting the same problem: computing WER from raw transcript penalizes formatting differences that have nothing to do with the quality itself. We built this library to solve it for ourselves, then open-sourced it so the broader speech community doesn't have to solve it again.

Sharing it felt like the right next step: the problem is universal, and community contributions are the best way to make reliable normalization available for every language, not just the ones we support today.

License

MIT

Metadata

Release files for gladia-normalization 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for gladia-normalization 0.4.0
File Size Uploaded Provenance Actions
gladia_normalization-0.4.0.tar.gz 257.0 kB Without provenance Details for gladia_normalization-0.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for gladia-normalization 0.4.0
File Interpreter ABI Platform Actions
gladia_normalization-0.4.0-py3-none-any.whl Python 3 none any Details for gladia_normalization-0.4.0-py3-none-any.whl

Total release size: 398.5 kB

Release files / gladia_normalization-0.4.0.tar.gz

Download URL gladia_normalization-0.4.0.tar.gz
Size 257.0 kB
Tags Source
SHA-256 checksum
How to use checksums
ee112899f5dc1a6315ddb58aa8a547543f4cb8a99460b4e9b9951d183a99d3b7
BLAKE2b-256 checksum
How to use checksums
58961cd36a55aea7a3f3c2dcf294b301c23c684301cd3b16bf8ad05caa4f7582
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / gladia_normalization-0.4.0-py3-none-any.whl

Download URL gladia_normalization-0.4.0-py3-none-any.whl
Size 141.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c9783f960827fec68c61b6dc225043dd74b0b7e3c32c22cb2463c6bebab79290
BLAKE2b-256 checksum
How to use checksums
b78967d53b6f02d8bf0b6ceb07dd20300a7982fe12dcb4534d10da74305a87fe
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for gladia-normalization 0.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page