text2tashkeel

text2tashkeel restores the missing vowel marks (tashkeel) in Arabic text. It is not one model but a model picker: a single small API over interchangeable diacritization models, all running on onnxruntime. It needs no PyTorch and no API keys, and it works offline by default. Pick the model that fits your accuracy, speed, or size budget. The only runtime dependencies are numpy and onnxruntime.

from text2tashkeel import Diacritizer
Diacritizer().diacritize("بسم الله الرحمن الرحيم")              # default model - 2.04% DER
Diacritizer("rawi-v2-int8").diacritize("بسم الله الرحمن الرحيم")  # lean single model

Most diacritizers add only the short-vowel marks, and only to text that is already spelled correctly. The default rawi models also restore the hamza (ء) and the silent dagger-alef, so they fix real, inconsistently spelled input, such as a bare ا typed for أ, not just clean text. See what makes rawi different for the reasons and the method.

Install

pip install text2tashkeel

The wheel is small (about 10 MB). It bundles the best models, and these work fully offline, with no downloads and no torch. The full-precision (fp32) variants are fetched from Hugging Face on first use, if you opt in:

pip install text2tashkeel        # int8 + flagship, offline
pip install text2tashkeel[hf]    # + auto-download fp32 models on demand

Without [hf], asking for a non-bundled model raises a clear message with its Hugging Face link. You can also point at your own model, such as one trained on a different corpus, with register_model(...). See below. For development, run pip install -e ".[test]", then run pytest.

Models

Two models cover almost every use. Both ship in the wheel and run offline.

22 model configurations are available for comparison, research, or special cases: the rawi family (V1/V2/V3 + INT8), two independent diacritizers (bilstm and libtashkeel), and gated ensembles of them.

from text2tashkeel import available_models, Diacritizer
available_models()                 # all models
available_models(bundled_only=True)  # the models that ship in the wheel (offline)
Diacritizer("rawi-v2-int8").diacritize("بسم الله الرحمن الرحيم")

available_models(bundled_only=True) lists the models that ship in the wheel. Everything else downloads from Hugging Face on first use, with [hf] installed. Each model's weights live in its own repo (rawi, rawi-v2, rawi-v3, rawi-ensemble, bilstm, libtashkeel), grouped in the Arabic Diacritizers collection.

If you trained a diacritizer on a different corpus, point at it:

from text2tashkeel import register_model, Diacritizer
register_model("my-rawi", "my_model.onnx", "my_vocab.json", arch="rawi")  # or arch="rawi-v3"
Diacritizer("my-rawi").diacritize("نص عربي")

Diacritizer is callable (d("...")) and lazily builds one onnxruntime session, which it reuses. Construct it once, then call it many times. For full credits and licenses for every model, see docs/07-credits-and-license.md.

CLI

text2tashkeel "الحمد لله رب العالمين"          # flagship default
echo "محمد رسول الله" | text2tashkeel
text2tashkeel -m rawi-v2-int8 < input.txt > output.txt

Benchmarks

Measured DER/WER for every model across the corpus's train/test/val splits is in benchmarks/.

Related projects

• phoonnx — offline text-to-speech, which reads diacritized text for pronunciation.

License

Apache License 2.0. See LICENSE and docs/07-credits-and-license.md for the license of each bundled model.