khmer-nlp-kcc 0.2.2

Khmer word segmentation, POS tagging, NER, and more — powered by a custom Khmer language model

khmer-nlp-kcc

Khmer NLP toolkit — word segmentation, POS tagging, named entity recognition (NER), and sentiment polarity.

Powered by a custom Khmer language model. The checkpoint is downloaded automatically from HuggingFace Hub on first use.

Installation

pip install khmer-nlp-kcc

Quick start

from khmer_nlp import KhmerNLP

nlp = KhmerNLP()  # checkpoint downloads on first call

# Word segmentation
nlp.segment("ខ្ញុំចូលចិត្តញាំបាយ")
# → ["ខ្ញុំ", "ចូលចិត្ត", "ញាំបាយ"]

# POS tagging
nlp.pos("ខ្ញុំចូលចិត្តញាំបាយ")
# → [{"word": "ខ្ញុំ", "label": "PRO"}, {"word": "ចូលចិត្ត", "label": "VB"}, {"word": "ញាំបាយ", "label": "VB"}]

# Named entity recognition
nlp.ner("លោក សុខ សុភ័ក្រ្ត ធ្វើការនៅភ្នំពេញ")
# → [{"text": "សុខសុភ័ក្រ្ត", "label": "PER"}]

# Sentiment polarity
nlp.polarity("ខ្ញុំចូលចិត្តប្រទេសខ្មែរណាស់")
# → {"label": "positive", "confidence": 0.9944, "scores": {"negative": 0.0023, "neutral": 0.0032, "positive": 0.9944}}

# All tasks at once
nlp.analyze("ខ្ញុំចូលចិត្តញាំបាយ")

Custom checkpoint

nlp = KhmerNLP(checkpoint_path="/path/to/your/model.pt")

Device selection

import torch
nlp = KhmerNLP(device=torch.device("cuda:0"))

Available methods

Method	Returns	Description
segment(text)	list[str]	Segmented word list
pos(text)	list[dict]	Word + POS label
ner(text)	list[dict]	Named entities (PER, LOC)
polarity(text)	dict	Sentiment polarity
analyze(text)	dict	All tasks combined

Raw token-level outputs: seg_tokens(), pos_tokens(), ner_tokens().

License

MIT