Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

text2tashkeel

text2tashkeel restores the missing vowel marks (tashkeel) in Arabic text. It is not one model but a model picker: a single small API over interchangeable diacritization models, all running on onnxruntime. It needs no PyTorch and no API keys, and it works offline by default. Pick the model that fits your accuracy, speed, or size budget. The only runtime dependencies are numpy and onnxruntime.

from text2tashkeel import Diacritizer
Diacritizer().diacritize("بسم الله الرحمن الرحيم")              # default model - 2.04% DER
Diacritizer("rawi-v2-int8").diacritize("بسم الله الرحمن الرحيم")  # lean single model

Most diacritizers add only the short-vowel marks, and only to text that is already spelled correctly. The default rawi models also restore the hamza (ء) and the silent dagger-alef, so they fix real, inconsistently spelled input, such as a bare ا typed for أ, not just clean text. See what makes rawi different for the reasons and the method.

Install

pip install --pre text2tashkeel

The --pre flag installs the latest alpha. The stable release on PyPI is older and lacks the orthography constraints and waqf.

The wheel is small (about 10 MB). It bundles the best models, and these work fully offline, with no downloads and no torch. The full-precision (fp32) variants are fetched from Hugging Face on first use, if you opt in:

pip install --pre text2tashkeel        # int8 + flagship, offline
pip install --pre "text2tashkeel[hf]"  # + auto-download fp32 models on demand

Without [hf], asking for a non-bundled model raises a clear message with its Hugging Face link. You can also point at your own model, such as one trained on a different corpus, with register_model(...). See below. For development, run pip install -e ".[test]", then run pytest.

Models

Two models cover almost every use. Both ship in the wheel and run offline.

Use case Model DER ↓ latency size
best accuracy (default) ⭐ rawi-ensemble 2.04% ~2 ms 4.9 MB
fastest & smallest rawi-v2-int8 2.30% ~1 ms 2.5 MB

22 model configurations are available for comparison, research, or special cases: the rawi family (V1/V2/V3 + INT8), two independent diacritizers (bilstm and libtashkeel), and gated ensembles of them.

from text2tashkeel import available_models, Diacritizer
available_models()                 # all models
available_models(bundled_only=True)  # the models that ship in the wheel (offline)
Diacritizer("rawi-v2-int8").diacritize("بسم الله الرحمن الرحيم")

available_models(bundled_only=True) lists the models that ship in the wheel. Everything else downloads from Hugging Face on first use, with [hf] installed. Each model's weights live in its own repo (rawi, rawi-v2, rawi-v3, rawi-ensemble, bilstm, libtashkeel), grouped in the Arabic Diacritizers collection.

If you trained a diacritizer on a different corpus, point at it:

from text2tashkeel import register_model, Diacritizer
register_model("my-rawi", "my_model.onnx", "my_vocab.json", arch="rawi")  # or arch="rawi-v3"
Diacritizer("my-rawi").diacritize("نص عربي")

Diacritizer is callable (d("...")) and lazily builds one onnxruntime session, which it reuses. Construct it once, then call it many times. For full credits and licenses for every model, see docs/07-credits-and-license.md.

CLI

text2tashkeel "الحمد لله رب العالمين"          # flagship default
echo "محمد رسول الله" | text2tashkeel
text2tashkeel -m rawi-v2-int8 < input.txt > output.txt

Benchmarks

Measured DER/WER for every model across the corpus's train/test/val splits is in benchmarks/.

  • phoonnx — offline text-to-speech, which reads diacritized text for pronunciation.

License

Apache License 2.0. See LICENSE and docs/07-credits-and-license.md for the license of each bundled model.

Metadata

Release files for text2tashkeel 0.3.1a1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for text2tashkeel 0.3.1a1
File Size Uploaded Provenance Actions
text2tashkeel-0.3.1a1.tar.gz 10.5 MB Without provenance Details for text2tashkeel-0.3.1a1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for text2tashkeel 0.3.1a1
File Interpreter ABI Platform Actions
text2tashkeel-0.3.1a1-py3-none-any.whl Python 3 none any Details for text2tashkeel-0.3.1a1-py3-none-any.whl

Total release size: 21.0 MB

Release files / text2tashkeel-0.3.1a1.tar.gz

Download URL text2tashkeel-0.3.1a1.tar.gz
Size 10.5 MB
Tags Source
SHA-256 checksum
How to use checksums
29d7c41e010a2448dcbd45dfc9a9c3a73e60c7d6f0c3d1f3453bb7a3ee6506e8
BLAKE2b-256 checksum
How to use checksums
3bd3a001137bd4032e4e14c71550fac3476481452b905156239ba1da746d8693
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / text2tashkeel-0.3.1a1-py3-none-any.whl

Download URL text2tashkeel-0.3.1a1-py3-none-any.whl
Size 10.5 MB
Tags Python 3
SHA-256 checksum
How to use checksums
2d233898f4118cd30b6ce3ca5cfcbf97b068e53d8e52050b169081f7ddfe049e
BLAKE2b-256 checksum
How to use checksums
63a7ca9d7fc0e8abf3d88d841969e68f7549ffa21072c64b12eef30935e11e44
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for text2tashkeel 0.3.1a1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page