khmerspeech

KhmerSpeech is a text-normalization toolkit tailored for Khmer speech applications. It provides a set of focused processors that clean raw text and verbalize numbers, currencies, dates, URLs, and other tokens that regularly appear in transcripts.

This project is heavily inspired by, and builds on, the open-source tha repository created by Seanghay Yat; the modules here adapt his original work for Khmer-specific speech use cases.

Requirements

• Python 3.8+
• Bundled processors rely on regex, phonenumbers, urlextract, and ftfy.

Installation

PyPI

pip install khmerspeech

From source

You can work directly from the repository today:

git clone https://github.com/MetythornPenn/khmerspeech
cd khmerspeech
pip install -e .

This will install the package in editable mode along with its core dependencies.

Quick start

All processors live under the khmerspeech.text_normalization namespace. Compose them to fit your pipeline:

from khmerspeech.text_normalization import (
  normalize,
  datetime as km_datetime,
  phone_numbers,
  currency,
  cardinals,
  decimals,
  urls,
  dict_verbalize,
)

raw = "010123123 គិតថ្លៃ $100.25 នៅថ្ងៃទី 2024-01-02 វេលា 10:23AM ចូលតាម https://google.com.kh"

# 1. Normalize Khmer punctuation / Unicode issues.
clean = normalize.processor(raw)

# 2. Handle structured tokens.
clean = phone_numbers.processor(clean, chunk_size=3)
clean = km_datetime.date_processor(clean)
clean = km_datetime.time_processor(clean)
clean = currency.processor(clean)
clean = urls.processor(clean)

# 3. Expand dictionary-driven spellings / units.
clean = dict_verbalize(clean)

print(clean)
# 0▁10▁123▁123 គិតថ្លៃ មួយរយដុល្លារ▁ម្ភៃប្រាំសេន នៅថ្ងៃទី 2024 01 02 វេលា 10 23▁A▁M ចូលតាម google dot com dot k▁h

# Standalone helpers
print(cardinals.processor("1234"))   # មួយពាន់▁ពីររយ▁សាមសិបបួន
print(decimals.processor("-123.45")) # ដក▁មួយរយ▁ម្ភៃបី▁ចុច▁សែសិបប្រាំ

Processors at a glance

Each module is intentionally tiny—inspect the source under khmerspeech/text_normalization/ if you need to tweak behaviour or build bespoke processors.

Dictionary resources

The text_normalization/dict/ folder ships pragmatic TSVs containing common spellings, pronunciations, and measurement units. dict_verbalize loads them at runtime:

from khmerspeech.text_normalization import dict_verbalize

text = "100 km ឬ 10 kg"
print(dict_verbalize(text))
# 100▁គីឡូម៉ែត្រ ឬ 10▁គីឡូក្រាម

Add new terms by editing the TSV files and re-running your pipeline—no extra build step is required.

Development

After installing in editable mode you can run the included assertions to spot regressions:

python tests/tests.py

Code contributions are welcome. Please format new code with ruff or black, keep functions focused, and add tests for new processors.

Acknowledgements

• Seanghay Yat for the original tha project, released under the MIT license.
• Nvidia NeMo text processing

License

khmerspeech is distributed under the Apache License 2.0. See the LICENSE file for the full text.