ogonki

Restore Polish diacritics in text typed without them. A tiny character-level model that runs offline with numpy only.

pip install ogonki

>>> from ogonki import restore
>>> restore("Wczoraj bylem w Lodzi")
'Wczoraj byłem w Łodzi'
>>> restore("Zolw zjadl salate")
'Żółw zjadł sałatę'
>>> df["title"] = df["title"].map(restore)   # pandas

ogonki notes.txt > fixed.txt
echo "Sprzedam rower gorski, stan bardzo dobry, odbior osobisty w Gdansku" | ogonki

• Tiny and offline: a 0.8 MB model, with numpy as the only dependency. No GPU, API key or download on first use.
• Fast: about 125,000 characters per second on a laptop CPU.
• Safe: only a c e l n o s z can change, and the output always has the same length as the input. Words that already have Polish letters, URLs and e-mail addresses are left untouched.

Benchmark

Word accuracy on 300,000 characters of held-out Polish web text (FineWeb-2), counting only words that could take a diacritic. Reproduce with python train.py.

ogonki makes about half as many mistakes as a dictionary lookup (1.5% of words instead of 3.2%).

Limitations

• Rare words can be missed, for example in the pangram Zażółć gęślą jaźń.
• Words whose spelling depends on meaning usually get the more common form: sad (orchard) vs sąd (court), czuje (he feels) vs czuję (I feel), laska (cane) vs łaska (grace).
• The model was trained and tested on web text only. Accuracy on OCR output, chat messages or old documents hasn't been measured yet.

How it works

For every a c e l n o s z, a small character-level CNN (8 dilated convolution layers that see 38 characters on each side) picks one of three classes: leave it, add the usual diacritic (a→ą, l→ł, z→ż…), or z→ź. The training data is free: take Polish text, strip its diacritics, and learn to put them back.

ogonki/__init__.py   restore(), strip(), CLI    (~75 lines, numpy only)
ogonki/weights.npz   trained model              (produced by train.py)
train.py             data -> train -> export -> benchmark

Train it yourself

Training takes under an hour on a free Google Colab T4 GPU, and much longer on a CPU.

pip install -e ".[train]"
python train.py

Data

Trained on the Polish part of FineWeb-2 by Hugging Face, released under the ODC-By 1.0 license.

License

MIT