Arabic Morph Segmenter

A lightweight neural Arabic morphological segmenter based on BiLSTM + CRF for Modern Standard Arabic (MSA) and multiple Arabic dialects.

---

Overview

Arabic words frequently contain attached conjunctions, prepositions, articles, pronouns, and other clitics.

For example,

وبكتابهم

should be segmented into

وب + كتاب + هم

instead of being treated as a single token.

This project provides a lightweight neural segmenter that predicts the character-level boundaries of:

• Prefix
• Stem
• Suffix

using a BiLSTM-CRF architecture.

The output can be used as a preprocessing step for:

• POS Tagging
• Named Entity Recognition (NER)
• Lemmatization
• Dependency Parsing
• Arabic Search
• Information Retrieval
• Large Language Models (LLMs)
• Arabic NLP pipelines

---

Features

• Lightweight (~9 MB)
• Fast inference
• Character-level neural segmentation
• BiLSTM + CRF architecture
• Supports Modern Standard Arabic
• Supports multiple Arabic dialects
• Easy Python API
• Ready for PyPI

---

Supported Arabic

Current training includes:

• Modern Standard Arabic (MSA)
• Iraqi Arabic
• Syrian Arabic
• Moroccan Arabic
• Najdi Arabic
• Sanaani Arabic
• Taizi Arabic

The architecture is language-independent and can easily be extended with additional dialects.

---

Model Architecture

Characters
      │
      ▼
Character Embedding
      │
      ▼
2-layer BiLSTM
      │
      ▼
Linear Layer
      │
      ▼
CRF
      │
      ▼
P / S / X

Where

• P = Prefix
• S = Stem
• X = Suffix

---

Installation

pip install arabic-morph-segmenter

---

Quick Start

from arabic_morph_segmenter import ArabicMorphSegmenter

segmenter = ArabicMorphSegmenter()

result = segmenter.segment_word("وبكتابهم")

print(result)

Output

{
    "word": "وبكتابهم",
    "prefix": "وب",
    "stem": "كتاب",
    "suffix": "هم",
    "segmented": "وب + كتاب + هم"
}

---

Sentence Segmentation

sentence = "والناس تفرقت من المدرسة"

for token in segmenter.segment_sentence(sentence):
    print(token["segmented"])

Output

وال + ناس
تفرق + ت
من
ال + مدرس + ة

---

More Examples

Word	Segmentation
وندد	و + ندد
الفلسطينيون	ال + فلسطين + يون
وبكتابهم	وب + كتاب + هم
فبالمدرسة	فبال + مدرس + ة
والناس	وال + ناس
تفرقت	تفرق + ت
وسيكتبون	وسي + كتب + ون
وللمهندسين	ولل + مهندس + ين

---

Evaluation

Evaluation on the held-out validation set:

Metric	Score
Character Accuracy	99.04%
Word Exact Accuracy	96.90%

---

Training Data

The model was trained on a combination of:

• Modern Standard Arabic morphological corpus
• Iraqi dialect
• Syrian dialect
• Moroccan dialect
• Najdi dialect
• Sanaani dialect
• Taizi dialect

using character-level prefix/stem/suffix annotations.

---

Project Structure

arabic_morph_segmenter/
│
├── assets/
│   └── arabic_morph_segmenter_bilstm_crf.pt
│
├── segmenter.py
├── __init__.py
│
examples/
│   └── demo.py
│
README.md
LICENSE

---

Citation

If you use this project in your research, please cite:

@software{alshargi2026arabicsegmenter,
  author = {Faisal Alshargi},
  title = {Arabic Morph Segmenter},
  year = {2026},
  url = {https://github.com/alshargi/arabic-morph-segmenter}
}

---

Roadmap

• ONNX export
• Hugging Face model
• REST API
• CLI interface
• Transformer version
• Additional Arabic dialects
• Confidence scores
• Batch inference

---

License

MIT License.

---

Author

Dr. Faisal Alshargi

AI • NLP • Arabic Language Technologies • Large Language Models

GitHub: https://github.com/alshargi

---

⭐ If you find this project useful, please consider giving it a star.