khmer-nlp-toolkits 0.1.0

Khmer NLP toolkits

khmer_nlp_toolkits

Khmer NLP Toolkits is an open-source Python library that provides practical utilities, text processing functions, and NLP components for the Khmer language.

The project is designed to make Khmer NLP easier for:

• Researchers
• Students
• Developers
• Production systems
• AI applications

It aims to provide a unified, lightweight, and extensible toolkit for common Khmer language processing tasks.

Installation

Requirements

• Python 3.10+

Install from PyPI

pip install khmer_nlp_toolkits

Quick Start

from khmer_nlp_toolkits.text import *
from khmer_nlp_toolkits.number import num2text

print(num2text(2026))
# ពីរពាន់ម្ភៃប្រាំមួយ

print(anonymizer(("You can contact us by phone at [phone], email us at [e-mail], "
"or visit our website at https://www.example.com anytime.")))
# You can contact us by phone at [TEL], email us at [EML], or visit our website at [URL] anytime.

print(text_cleaner(
    "\u200b Hello\u2014\u2014World!!!??      ខ្មែរ   123\u200b "
))
# "Hello - World !? ខ្មែរ 123"

Features

Khmer NLP Toolkits provides modular utilities and NLP components for Khmer language processing.

Commoncrawl 🔗

• Content Quality Warning
• Document Quality Filtering
• Adult Content Filtering

Text Processing 🔗

• Text Anonymization
• Text Normalization
• Text Cleaning
• Segmentation

Number Processing 🔗

• number conversion to text

Deduplication 🔗

• URL deduplication
• Content Deduplication

Utilities 🔗

• End2End Pipeline
• File Utilities Features

Vision

The long-term goal of Khmer NLP Toolkits is to help build a comprehensive open-source ecosystem for Khmer Natural Language Processing and accelerate Khmer AI/NLP research and development.

We hope this project can contribute to:

• Better Khmer language tooling
• Educational resources
• NLP research accessibility
• Open-source AI infrastructure for Khmer

Contributing

Contributions, issues, and feature requests are welcome.

Feel free to open a pull request or discussion.

Contributors

• Sopagna HEANG
• Natt KORAT
• Chily RAN

License

This project is licensed under the MIT License.