araclean 0.2.0

Arabic text normalization and cleaning — pure-Python, composable, reproducible.

araclean

Arabic text normalization and cleaning — pure-Python, composable, reproducible.

Status: pre-release (0.x). The v1 normalization core is complete and fully tested; the API may still shift before 1.0.

araclean is non-destructive by default: the bare call is lossless encoding repair (Unicode form, presentation forms, tatweel, bidi/zero-width characters, look-alike letters, whitespace) and never silently strips tashkeel or folds letters. Everything lossy is opt-in through named, serializable profiles (SEARCH, ML, SOCIAL, CLASSICAL), so the exact preprocessing a corpus went through can be published and reproduced.

>>> from araclean import normalize
>>> normalize("العـــربية")                          # lossless encoding repair (default)
'العربية'
>>> normalize("اَلسّلامُ عليكم", profile="search")   # opt-in lossy folds for recall
'السلام عليكم'

Documentation

Full documentation lives at https://mhdmartini.github.io/araclean/:

• Getting started — install, first call, choosing a profile.
• Profiles — every step each profile applies, lossless vs lossy (generated from the code).
• Guides — the CLI, pandas & polars, tuning and composing pipelines, custom steps, reproducibility, and stopwords.
• Why araclean — the rationale and what sets it apart.
• API reference and CLI reference.

Every Python example in the docs is executed as a doctest in CI, and the generated pages (profiles, glossary, CLI reference) are drift-checked against the code.

Install

pip install araclean

Optional extras (declared now, populated by later slices):

pip install "araclean[cli]"     # command-line interface
pip install "araclean[pandas]"  # pandas Series accessor
pip install "araclean[polars]"  # polars accessor
pip install "araclean[all]"     # everything

The core install is lean: it requires only pydantic v2 — no compiler, Java, or data download.

Development

This project uses uv for environments and pre-commit for the quality gate.

uv sync                       # create the dev environment
uv run pre-commit install     # wire up the pre-commit hooks
uv run pre-commit run --all-files

The gate runs ruff, mypy --strict, pyright, pytest, and cspell (canonical Arabic terminology per GLOSSARY.md).

Commits & versioning

The version and changelog are derived from commit messages by Commitizen — see ADR-0008. Every commit must be a Conventional Commit; the format is enforced by a commit-msg hook (wired by uv run pre-commit install) and in CI on every PR.

feat(steps): add RemoveTashkeel step     # → minor bump
fix(pipeline): preserve step order        # → patch bump
feat(api)!: rename normalize() argument   # breaking → minor (we are pre-1.0)

feat bumps the minor, fix/others the patch, and a breaking change (! or a BREAKING CHANGE: footer) the minor — the project stays in 0.x until 1.0 is declared. To cut a release:

uv run cz bump        # compute the bump, update pyproject.toml + uv.lock + CHANGELOG.md, tag vX.Y.Z
uv run cz changelog   # preview release notes without bumping

Never hand-edit [project].version — Commitizen owns it.

License

MIT