kaldingram 0.1.0

Train and entropy-prune ARPA n-gram language models

kaldingram

kaldingram provides Python and CLI tools to:

• train Kneser-Ney back-off n-gram language models in ARPA format
• entropy-prune ARPA language models

The implementation is based on Kaldi WSJ scripts and matches SRILM-style behavior.

Install

pip install kaldingram

CLI Usage

Train an n-gram LM

kaldingram train --ngram-order 4 --text corpus.txt --lm 4gram.arpa

Or stream text from stdin and write ARPA to stdout:

cat corpus.txt | kaldingram train --ngram-order 3 > 3gram.arpa

Prune an n-gram LM

kaldingram prune --threshold 1e-8 --lm 4gram.arpa --write-lm 4gram_pruned.arpa

Development

Build package locally:

python -m pip install --upgrade build
python -m build