nlpo3 1.1.3

Python binding for nlpO3 Thai language processing library

Python binding for nlpO3, a Thai natural language processing library in Rust.

• Thai word tokenizer
• use maximal-matching dictionary-based tokenization algorithm and honor Thai Character Cluster boundaries
• 2.5x faster than similar pure Python implementation
• built-in dictionary included (62,000 words, a copy from PyThaiNLP)
• support custom dictionary

pip install nlpo3

Tokenization using default dictionary:

from nlpo3 import segment

segment("สวัสดีครับ")  # returns ["สวัสดี", "ครับ"]

Load file path/to/dict.file to memory and assigned it with name custom_dict. Then tokenize a text with custom_dict dictionary:

from nlpo3 import load_dict, segment

load_dict("path/to/dict.file", "custom_dict")
segment("สวัสดีครับ", "custom_dict")

For more documentation, go https://github.com/PyThaiNLP/nlpo3