CLAWS Tokenizer: Cluster Level Aggregation Khmer Word Segmentation

CLAWS Tokenizer is a lightweight, fast Khmer word segmentation library for Python. It is based on a Graph Neural Network. The model is trained using Tensorflow and the weights are converted to TFLite.

Installation

Install via pip:

pip install claws-tokenizer

Quick Start

from claws_tokenizer import segment

tokens = segment("ខ្ញុំស្រឡាញ់ភាសាខ្មែរ")
print(tokens)
# Output: ['ខ្ញុំ', 'ស្រឡាញ់', 'ភាសា', 'ខ្មែរ']

Citation

If you use Claws in your research, please cite it as follows:

BibTeX

@software{claws_tokenizer,
  author = {Socret Lee},
  title = {CLAWS Tokenizer: Cluster Level Aggregation Khmer Word Segmentation},
  year = {2026},
  url = {[https://github.com/Socret360/claws](https://github.com/Socret360/claws)},
  version = {0.1.1}
}

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

claws_tokenizer-0.1.2.tar.gz (1.5 MB view details)

Uploaded Aug 19, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

claws_tokenizer-0.1.2-py3-none-any.whl (1.5 MB view details)

Uploaded Aug 19, 2026 Python 3

File details

Details for the file claws_tokenizer-0.1.2.tar.gz.

File metadata

Download URL: claws_tokenizer-0.1.2.tar.gz
Upload date: Aug 19, 2026
Size: 1.5 MB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/7.0.0 CPython/3.11.9

File hashes

Hashes for claws_tokenizer-0.1.2.tar.gz
Algorithm	Hash digest
SHA256	`0b2e9d8968fdc0e429c12c97f56631abbf3d6a99928ce7b888725204161e40d1`
MD5	`1ce992d0791a9108d90843fbaca10f5d`
BLAKE2b-256	`d37a424281f54e16c164026316529ce867303cfe762983582a73ef5edc5e0a30`

See more details on using hashes here.

File details

Details for the file claws_tokenizer-0.1.2-py3-none-any.whl.

File metadata

Download URL: claws_tokenizer-0.1.2-py3-none-any.whl
Upload date: Aug 19, 2026
Size: 1.5 MB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/7.0.0 CPython/3.11.9

File hashes

Hashes for claws_tokenizer-0.1.2-py3-none-any.whl
Algorithm	Hash digest
SHA256	`202da717f2ec3cb886250a6b89bfae28a3a8185bd1ddcff1a5f6eedc1677980b`
MD5	`de63bed72377781b6bd732ee18d44712`
BLAKE2b-256	`376d0e7d44660b8e2f4792c97b8fd05ec07bf6aa2b2f2ed05164aef59b3dd72c`