khmer-nlp-kcc 0.2.0

Khmer word segmentation, POS tagging, NER, and more — powered by a custom Khmer language model

khmer-nlp-kcc

Khmer NLP toolkit — word segmentation, POS tagging, named entity recognition (NER), and sentiment polarity.

Powered by a custom Khmer language model. The checkpoint is downloaded automatically from HuggingFace Hub on first use.

Installation

pip install khmer-nlp-kcc

Quick start

from khmer_nlp import KhmerNLP

nlp = KhmerNLP()  # checkpoint downloads on first call

# Word segmentation
nlp.segment("ខ្ញុំចូលចិត្តញាំបាយ")
# → ["ខ្ញុំ", "ចូលចិត្ត", "ញាំ", "បាយ"]

# POS tagging
nlp.pos("ខ្ញុំចូលចិត្តញាំបាយ")
# → [{"word": "ខ្ញុំ", "label": "PRO"}, {"word": "ចូលចិត្ត", "label": "VB"}, ...]

# Named entity recognition
nlp.ner("គ្រូពេទ្យធ្វើការនៅមន្ទីរពេទ្យភ្នំពេញ")
# → [{"text": "ភ្នំពេញ", "label": "LOC"}]

# Sentiment polarity
nlp.polarity("ខ្ញុំចូលចិត្តប្រទេសខ្មែរណាស់")
# → {"label": "positive", "confidence": 0.94, "scores": {...}}

# All tasks at once
nlp.analyze("ខ្ញុំចូលចិត្តញាំបាយ")

Custom checkpoint

nlp = KhmerNLP(checkpoint_path="/path/to/your/model.pt")

Device selection

import torch
nlp = KhmerNLP(device=torch.device("cuda:0"))

Available methods

Method	Returns	Description
segment(text)	list[str]	Segmented word list
pos(text)	list[dict]	Word + POS label
ner(text)	list[dict]	Named entities (PER, LOC)
polarity(text)	dict	Sentiment polarity
analyze(text)	dict	All tasks combined

Raw token-level outputs: seg_tokens(), pos_tokens(), ner_tokens().

License

MIT