Samdei Type
Samdei Type converts romanized Khmer, the Latin spelling many people use in chat, to Khmer script:
$ samdei-type "knhom jong tov phsar"
ខ្ញុំចង់ទៅផ្សារ
This package runs the released Samdei Type models: a command-line converter and a Python API, with beam search, int8 ONNX decoding and the reranking used in the paper. The models are on the Hugging Face Hub and are downloaded on first use:
| Model | Parameters | Decoded as in the paper with | First download |
|---|---|---|---|
| Samdei Type Mini | 7.1M | int8 ONNX graphs + word LM | about 8 MB (ONNX) |
| Samdei Type Base | 18.2M | PyTorch weights + character LM, word LM and channel model | about 120 MB |
Both are character-level Transformer encoder-decoders. Mini is small enough for a phone keyboard; Base is more accurate. Their scores on Samdei Type Bench, training data and limitations are in the model cards. All releases: the Samdei Type collection. Paper: technical report (PDF).
Samdei (សំដី, "what you say") is our team, Samdei Labs.
Install
pip install "samdei-type[deploy]"
Python 3.10 or later. The deploy extra adds onnxruntime for the int8 ONNX graphs, which Mini is decoded with.
Without it, Mini falls back to its PyTorch weights. PyTorch is needed either way: on macOS the install
takes about 1 GB. On Linux, pip installs PyTorch with its CUDA libraries, several GB; for a CPU-only install, add
PyTorch's CPU build first: pip install torch --index-url https://download.pytorch.org/whl/cpu.
The latest code from GitHub: pip install "samdei-type[deploy] @ git+https://github.com/Samdei-Labs/samdei-type.git".
Command line
$ samdei-type "knhom jong tov phsar" # Samdei Type Mini
ខ្ញុំចង់ទៅផ្សារ
$ samdei-type -m samdei/samdei-type-base "bong nham bay nov?"
បងញ៉ាំបាយនៅ?
$ echo "saek nh tv rean" | samdei-type - # one message per line from stdin
ស្អែកខ្ញុំទៅរៀន
$ samdei-type --nbest "jam knhom bontech" # also each piece's candidates and scores
ចាំខ្ញុំបន្តិច
-8.16 ចាំខ្ញុំបន្តិច
-11.31 ចាំ ខ្ញុំបន្តិច
...
samdei-type --help lists the options: --backend torch|onnx, --beam, --no-rerank (beam search only),
--no-chunk, --tsv, --device, --threads. python -m samdei_type runs the same command.
The first run downloads the model into the Hugging Face cache (~/.cache/huggingface). After that it works
offline; set HF_HUB_OFFLINE=1 to skip the check for updates. -m also takes a local folder:
hf download samdei/samdei-type-mini --local-dir samdei-type-mini
samdei-type -m ./samdei-type-mini "ot mean wifi te"
Python
from samdei_type.pipeline import Converter
conv = Converter("samdei/samdei-type-base") # a Hub repo id or a local model folder
conv.convert("knhom jong tov phsar") # 'ខ្ញុំចង់ទៅផ្សារ'
conv.convert_batch(["bong nham bay nov?", "ot mean wifi te"])
# ['បងញ៉ាំបាយនៅ?', 'អត់មាន wifi ទេ']
mini = Converter("samdei/samdei-type-mini") # int8 ONNX graphs, as rerank.json says
mini.nbest(mini.pieces("jam knhom bontech")) # [[('ចាំខ្ញុំបន្តិច', -8.16), ...]]
Converter reads the model's rerank.json, so it decodes as the published scores were computed: the backend,
the beam size, the pieces and the rerank weights. backend="torch" or "onnx" overrides the backend. The same steps
with the lower-level pieces:
import json
from huggingface_hub import snapshot_download
from samdei_type.evaluation import chunk_source, translate
from samdei_type.model import load
from samdei_type.rerank import Reranker
d = snapshot_download("samdei/samdei-type-base")
model, sv, tv = load(d) # config.json + model.safetensors
weights = json.load(open(f"{d}/rerank.json"))["rerank_weights"]
rr = Reranker(weights, char_lm=f"{d}/lm/char5_nowiki.npz", word_lm=f"{d}/lm/wordfull_nowiki.json.gz",
channel=f"{d}/channel")
pieces = chunk_source("knhom jong tov phsar", 8)
nbest = translate(model, sv, tv, pieces, beam=5)
print(" ".join(rr.rerank(p, hyps)[0][0] for p, hyps in zip(pieces, nbest)))
For the int8 graphs, OnnxConverter(f"{d}/onnx").beam(piece, k=5) from samdei_type.onnx_infer gives a piece's
n-best list. That module uses only numpy and onnxruntime, and is the reference to port to a phone runtime.
How decoding works
- The input is cut into pieces of at most eight words, with a break after any punctuation mark
(
samdei_type.evaluation.chunk_source). The models were trained on short pieces. - Each piece gets a beam search with k = 5 (
samdei_type.model.beam_cached, orOnnxConverter.beam). A piece longer than 249 characters is cut. - Each hypothesis y of input x is scored as
log P(y|x) + a·log P_char(y) + b·log P_word(y) − c·#OOV(y) + d·log P_ch(x|y)(samdei_type.rerank). The channel model P_ch reads Khmer and predicts the romanized input, so it checks that a hypothesis explains all of the input. The weights a–d come from the model'srerank.json. - The best hypotheses of the pieces are joined with a space.
Limitations
- The models convert what people type in chat. They are not a dictionary: an unusual spelling, a name or a rare word can come out as a common word that sounds alike. The model cards give the error rates and examples.
- English words are often spelled out in Khmer script (
see you→ ស៊ីយូ); some, likewifi, are kept. - Emoji, URLs and @mentions are not passed through: emoji are dropped and other symbols can change. Split them off before converting if you need to keep them.
Package layout
| Module | What it does |
|---|---|
samdei_type.pipeline |
Converter: a model folder or Hub repo id → convert(text) |
samdei_type.model |
the encoder-decoder, load, KV-cached greedy and beam search |
samdei_type.onnx_infer |
int8 ONNX decoding with numpy and onnxruntime |
samdei_type.rerank, samdei_type.lm |
n-best reranking; hashed character n-gram LM and word bigram LM |
samdei_type.evaluation |
input chunking, batched decoding, CER and exact match (score, as for Samdei Type Bench) |
samdei_type.text, samdei_type.khnormal |
Khmer normalization |
samdei_type.vocab |
character vocabularies |
samdei_type.cli |
the samdei-type command |
Training code and the data generator are not part of this package. The training data are on the Hub: samdei/samdei-type-pretraining-data and samdei/samdei-type-finetuning-data.
Development
git clone https://github.com/Samdei-Labs/samdei-type && cd samdei-type
pip install -e ".[deploy]"
python -m unittest discover -s tests
SAMDEI_TYPE_MODEL=samdei/samdei-type-mini python -m unittest discover -s tests # also an end-to-end conversion
License
The code is licensed under the Apache License 2.0 (see LICENSE and NOTICE). src/samdei_type/khnormal.py is
vendored from SIL Global's khmer-character-specification
under the MIT License. The models and datasets on the Hub have their own licenses, given in their cards.
Citation
The technical report is paper/samdei-type.pdf.
@misc{samdei-type-2026,
title = {Samdei Type: Separating What Is Written from How It Is Spelled for Romanized Khmer Transliteration},
author = {Mengsay Loem},
year = {2026},
note = {Technical report},
url = {https://github.com/Samdei-Labs/samdei-type/blob/main/paper/samdei-type.pdf}
}
Metadata
Release files for samdei-type 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| samdei_type-0.1.0.tar.gz | 27.4 kB | With provenance | Details for samdei_type-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| samdei_type-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for samdei_type-0.1.0-py3-none-any.whl |
Total release size: 57.9 kB
Release files / samdei_type-0.1.0.tar.gz
| Download URL | samdei_type-0.1.0.tar.gz |
|---|---|
| Size | 27.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
19a9ff29d2ab6bfdc7df56daf7906b7e3cf923e5a55e07a16ce06250102ea9fe
|
|
BLAKE2b-256 checksum How to use checksums |
733d5860e713cec67afa959f680915e7b2acdfa417916cbbfd721e119baaf8f2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logRelease files / samdei_type-0.1.0-py3-none-any.whl
| Download URL | samdei_type-0.1.0-py3-none-any.whl |
|---|---|
| Size | 30.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0f7c69a75f07edc072472f22d691f7ed80b3a59c90ccf58b040ca6567727b135
|
|
BLAKE2b-256 checksum How to use checksums |
811a148a8fa7306e6b498e25d101a7d2fad78a2af05e201ca33c483b9056ca60
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logSecurity for samdei-type 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for samdei_type-0.1.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.