Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

text2tashkeel

text2tashkeel restores the missing vowel marks (tashkeel) in Arabic text. It is not one model but a model picker: a single small API over interchangeable diacritization models, all running on onnxruntime. It needs no PyTorch and no API keys, and it works offline by default. Pick the model that fits your accuracy, speed, or size budget. The only runtime dependencies are numpy and onnxruntime.

from text2tashkeel import Diacritizer
Diacritizer().diacritize("بسم الله الرحمن الرحيم")              # default model - 2.03% DER
Diacritizer("rawi-v2-int8").diacritize("بسم الله الرحمن الرحيم")  # lean single model

Most diacritizers add only the short-vowel marks, and only to text that is already spelled correctly. The default rawi models also restore the hamza (ء) and the silent dagger-alef, so they fix real, inconsistently spelled input, such as a bare ا typed for أ, not just clean text. See what makes rawi different for the reasons and the method.

Install

pip install --pre text2tashkeel

The --pre flag installs the latest alpha. The stable release on PyPI is older and lacks the orthography constraints and waqf.

The wheel is small (about 10 MB). It bundles the best models, and these work fully offline, with no downloads and no torch. The full-precision (fp32) variants are fetched from Hugging Face on first use, if you opt in:

pip install --pre text2tashkeel        # int8 + flagship, offline
pip install --pre "text2tashkeel[hf]"  # + auto-download fp32 models on demand

Without [hf], asking for a non-bundled model raises a clear message with its Hugging Face link. You can also point at your own model, such as one trained on a different corpus, with register_model(...). See below. For development, run pip install -e ".[test]", then run pytest.

Models

Two models cover almost every use. Both ship in the wheel and run offline.

Use case Model DER ↓ latency size
best accuracy (default) ⭐ rawi-ensemble 2.03% ~2 ms 4.9 MB
fastest & smallest rawi-v2-int8 2.30% ~1 ms 2.5 MB

22 model configurations are available for comparison, research, or special cases: the rawi family (V1/V2/V3 + INT8), two independent diacritizers (bilstm and libtashkeel), and gated ensembles of them.

from text2tashkeel import available_models, Diacritizer
available_models()                 # all models
available_models(bundled_only=True)  # the models that ship in the wheel (offline)
Diacritizer("rawi-v2-int8").diacritize("بسم الله الرحمن الرحيم")

available_models(bundled_only=True) lists the models that ship in the wheel. Everything else downloads from Hugging Face on first use, with [hf] installed. Each model's weights live in its own repo (rawi, rawi-v2, rawi-v3, rawi-ensemble, bilstm, libtashkeel), grouped in the Arabic Diacritizers collection.

If you trained a diacritizer on a different corpus, point at it:

from text2tashkeel import register_model, Diacritizer
register_model("my-rawi", "my_model.onnx", "my_vocab.json", arch="rawi")  # or arch="rawi-v3"
Diacritizer("my-rawi").diacritize("نص عربي")

Diacritizer is callable (d("...")) and lazily builds one onnxruntime session, which it reuses. Construct it once, then call it many times. For full credits and licenses for every model, see docs/07-credits-and-license.md.

CLI

text2tashkeel "الحمد لله رب العالمين"          # flagship default
echo "محمد رسول الله" | text2tashkeel
text2tashkeel -m rawi-v2-int8 < input.txt > output.txt

Benchmarks

The rawi-ensemble DER is scored on the full held-out test split. The command that prints it is in benchmarks/README.md. The same page has the numbers for the other models.

  • phoonnx — offline text-to-speech, which reads diacritized text for pronunciation.

License

Apache License 2.0. See LICENSE and docs/07-credits-and-license.md for the license of each bundled model.

Metadata

Release files for text2tashkeel 0.3.1a2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for text2tashkeel 0.3.1a2
File Size Uploaded Provenance Actions
text2tashkeel-0.3.1a2.tar.gz 10.5 MB Without provenance Details for text2tashkeel-0.3.1a2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for text2tashkeel 0.3.1a2
File Interpreter ABI Platform Actions
text2tashkeel-0.3.1a2-py3-none-any.whl Python 3 none any Details for text2tashkeel-0.3.1a2-py3-none-any.whl

Total release size: 21.0 MB

Release files / text2tashkeel-0.3.1a2.tar.gz

Download URL text2tashkeel-0.3.1a2.tar.gz
Size 10.5 MB
Tags Source
SHA-256 checksum
How to use checksums
3c45f47dd23cc4fcab89993d05f522c3e06e1ba8fbab737893111c8125925352
BLAKE2b-256 checksum
How to use checksums
82ef2a7c19688e0bcb4fbd76d747d02bbf93b6514c5a1a184f9557d3d90cba78
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / text2tashkeel-0.3.1a2-py3-none-any.whl

Download URL text2tashkeel-0.3.1a2-py3-none-any.whl
Size 10.5 MB
Tags Python 3
SHA-256 checksum
How to use checksums
071ff2c26610ea0b06d491a94b06dd58bc54da3f0a82838334b3dbf28849dc7e
BLAKE2b-256 checksum
How to use checksums
68b77db562ce6cb42ada239abacea7a34974fca1eaeba5fca4ccf67f6a715e92
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for text2tashkeel 0.3.1a2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page