This release is a pre-release and may not be stable for production use.
text2tashkeel
text2tashkeel restores the missing vowel marks (tashkeel) in Arabic text. It is
not one model but a model picker: a single small API over interchangeable
diacritization models, all running on onnxruntime. It needs no PyTorch and
no API keys, and it works offline by default. Pick the model that fits your
accuracy, speed, or size budget. The only runtime dependencies are numpy
and onnxruntime.
from text2tashkeel import Diacritizer
Diacritizer().diacritize("بسم الله الرحمن الرحيم") # default model - 2.03% DER
Diacritizer("rawi-v2-int8").diacritize("بسم الله الرحمن الرحيم") # lean single model
Most diacritizers add only the short-vowel marks, and only to text that is
already spelled correctly. The default rawi models also restore the hamza
(ء) and the silent dagger-alef, so they fix real, inconsistently spelled
input, such as a bare ا typed for أ, not just clean text. See
what makes rawi different
for the reasons and the method.
Install
pip install --pre text2tashkeel
The --pre flag installs the latest alpha. The stable release on PyPI is
older and lacks the orthography constraints and waqf.
The wheel is small (about 10 MB). It bundles the best models, and these work fully offline, with no downloads and no torch. The full-precision (fp32) variants are fetched from Hugging Face on first use, if you opt in:
pip install --pre text2tashkeel # int8 + flagship, offline
pip install --pre "text2tashkeel[hf]" # + auto-download fp32 models on demand
Without [hf], asking for a non-bundled model raises a clear message with its
Hugging Face link. You can also point at your own model, such as one trained
on a different corpus, with register_model(...). See below. For
development, run pip install -e ".[test]", then run pytest.
Models
Two models cover almost every use. Both ship in the wheel and run offline.
| Use case | Model | DER ↓ | latency | size |
|---|---|---|---|---|
| best accuracy (default) ⭐ | rawi-ensemble |
2.03% | ~2 ms | 4.9 MB |
| fastest & smallest | rawi-v2-int8 |
2.30% | ~1 ms | 2.5 MB |
22 model configurations are available for comparison, research, or special
cases: the rawi family (V1/V2/V3 + INT8), two independent diacritizers
(bilstm and libtashkeel), and gated ensembles of them.
from text2tashkeel import available_models, Diacritizer
available_models() # all models
available_models(bundled_only=True) # the models that ship in the wheel (offline)
Diacritizer("rawi-v2-int8").diacritize("بسم الله الرحمن الرحيم")
available_models(bundled_only=True) lists the models that ship in the
wheel. Everything else downloads from Hugging Face on first use, with [hf]
installed. Each model's weights live in its own repo
(rawi,
rawi-v2,
rawi-v3,
rawi-ensemble,
bilstm,
libtashkeel),
grouped in the Arabic Diacritizers collection.
If you trained a diacritizer on a different corpus, point at it:
from text2tashkeel import register_model, Diacritizer
register_model("my-rawi", "my_model.onnx", "my_vocab.json", arch="rawi") # or arch="rawi-v3"
Diacritizer("my-rawi").diacritize("نص عربي")
Diacritizer is callable (d("...")) and lazily builds one onnxruntime
session, which it reuses. Construct it once, then call it many times. For
full credits and licenses for every model, see
docs/07-credits-and-license.md.
CLI
text2tashkeel "الحمد لله رب العالمين" # flagship default
echo "محمد رسول الله" | text2tashkeel
text2tashkeel -m rawi-v2-int8 < input.txt > output.txt
Benchmarks
The rawi-ensemble DER is scored on the full held-out test split. The command
that prints it is in benchmarks/README.md.
The same page has the numbers for the other models.
Related projects
phoonnx— offline text-to-speech, which reads diacritized text for pronunciation.
License
Apache License 2.0. See LICENSE and
docs/07-credits-and-license.md for the
license of each bundled model.
Metadata
Release files for text2tashkeel 0.3.1a2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| text2tashkeel-0.3.1a2.tar.gz | 10.5 MB | Without provenance | Details for text2tashkeel-0.3.1a2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| text2tashkeel-0.3.1a2-py3-none-any.whl | Python 3 | none | any | Details for text2tashkeel-0.3.1a2-py3-none-any.whl |
Total release size: 21.0 MB
Release files / text2tashkeel-0.3.1a2.tar.gz
| Download URL | text2tashkeel-0.3.1a2.tar.gz |
|---|---|
| Size | 10.5 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3c45f47dd23cc4fcab89993d05f522c3e06e1ba8fbab737893111c8125925352
|
|
BLAKE2b-256 checksum How to use checksums |
82ef2a7c19688e0bcb4fbd76d747d02bbf93b6514c5a1a184f9557d3d90cba78
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Release files / text2tashkeel-0.3.1a2-py3-none-any.whl
| Download URL | text2tashkeel-0.3.1a2-py3-none-any.whl |
|---|---|
| Size | 10.5 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
071ff2c26610ea0b06d491a94b06dd58bc54da3f0a82838334b3dbf28849dc7e
|
|
BLAKE2b-256 checksum How to use checksums |
68b77db562ce6cb42ada239abacea7a34974fca1eaeba5fca4ccf67f6a715e92
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Security for text2tashkeel 0.3.1a2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.