ArabiNLP

A lightweight, dependency-free Python toolkit for Arabic Natural Language Processing (NLP), providing preprocessing, normalization, morphology, stemming, lemmatization, spell checking, and information retrieval.

Highlights

• Pure Python implementation
• No heavy external dependencies
• Arabic-first design
• Production-ready APIs
• Python 3.9+
• MIT License

---

Version 2.1.2

Features

• Text cleaning
• Arabic normalization
• Tokenization
• Stopword removal
• Sentence splitting
• Text statistics
• Word and character n-grams
• Keyword extraction
• Language detection
• Jaccard similarity
• Arabic Light Stemmer
• Root Extraction
• Lemmatization
• Morphological Segmentation
• Morphological Analysis
• Arabic Spell Checker
• TF-IDF Vectorizer
• BM25 Ranking & Retrieval

---

Installation

pip install arabinlp

---

Quick Start

from arabinlp import (
    normalize_arabic,
    remove_stopwords,
    tokenize,
    ArabicLightStemmer,
)

text = "أنا أحب تعلم الذكاء الاصطناعي في الجامعات."

# Normalize
text = normalize_arabic(text)

# Remove stopwords (expects text, returns filtered tokens)
filtered_tokens = remove_stopwords(text)

# Tokenize after converting tokens back to text
tokens = tokenize(" ".join(filtered_tokens))

# Stem
stemmer = ArabicLightStemmer()
stems = [stemmer.stem(token) for token in tokens]

print(stems)

Expected output

['احب', 'تعلم', 'ذكاء', 'اصطناع', 'جامع']

---

End-to-End NLP Pipeline

from arabinlp import (
    normalize_arabic,
    remove_stopwords,
    tokenize,
    ArabicLightStemmer,
    ArabicRootExtractor,
    ArabicLemmatizer,
)

text = "والمكتبات جميلة"

text = normalize_arabic(text)
filtered_tokens = remove_stopwords(text)
tokens = tokenize(" ".join(filtered_tokens))

stemmer = ArabicLightStemmer()
extractor = ArabicRootExtractor()
lemmatizer = ArabicLemmatizer()

for token in tokens:
    print(
        token,
        stemmer.stem(token),
        extractor.extract(token),
        lemmatizer.lemmatize(token),
    )

---

Python Support

• Python >= 3.9

Supported versions

• Python 3.9
• Python 3.10
• Python 3.11
• Python 3.12

---

Links

• Repository: https://github.com/IbtihalMakki/arabic-nlp-toolkit
• PyPI: https://pypi.org/project/arabinlp/
• Issues: https://github.com/IbtihalMakki/arabic-nlp-toolkit/issues

---

API Overview

API	Description
preprocess	Complete preprocessing pipeline
normalize_arabic	Normalize Arabic text
tokenize	Tokenization
remove_stopwords	Stopword removal
split_sentences	Sentence segmentation
word_frequency	Word frequency
ngrams	Word n-grams
character_ngrams	Character n-grams
detect_language	Language detection
text_statistics	Text statistics
extract_keywords	Keyword extraction
jaccard_similarity	Text similarity
ArabicLightStemmer	Light stemming
ArabicRootExtractor	Root extraction
ArabicLemmatizer	Lemmatization
ArabicSegmenter	Morphological segmentation
ArabicMorphAnalyzer	Morphological analysis
ArabicSpellChecker	Spell checker
ArabicTfidfVectorizer	TF-IDF vectorizer
ArabicBM25	BM25 Retrieval & Ranking

---

Core Preprocessing

from arabinlp import (
    preprocess,
    normalize_arabic,
    tokenize,
    remove_stopwords,
    split_sentences,
    word_frequency,
    ngrams,
    character_ngrams,
    detect_language,
    text_statistics,
    extract_keywords,
    jaccard_similarity,
)

text = "السَّلَامُ عَلَيْكُمْ 😊 زوروا https://example.com"

cleaned = preprocess(text)

normalized = normalize_arabic(
    "إسلام وآثار",
    level="light",
)

tokens = tokenize(
    "السلام عليكم ورحمة الله",
)

filtered = remove_stopwords(
    "أنا أحب تعلم الذكاء الاصطناعي في الجامعة",
)

sentences = split_sentences(
    "السلام عليكم. كيف حالك؟ أنا بخير!",
)

freq = word_frequency(
    "الذكاء الذكاء الاصطناعي رائع",
)

word_bigrams = ngrams(
    "السلام عليكم ورحمة الله",
    2,
)

char_trigrams = character_ngrams(
    "السلام",
    3,
)

lang = detect_language(
    "Hello مرحبا",
)

stats = text_statistics(
    "السلام عليكم. كيف حالك؟",
)

keywords = extract_keywords(
    "الذكاء الاصطناعي رائع. الذكاء يتطور بسرعة.",
    top_k=2,
)

sim = jaccard_similarity(
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي رائع",
)

---

Cleaning Utilities

from arabinlp import (
    remove_urls,
    remove_mentions,
    remove_hashtags,
    remove_extra_spaces,
    remove_diacritics,
    remove_tatweel,
    remove_emojis,
)

print(remove_urls("Visit https://example.com"))

print(remove_mentions("@ibtihal hello"))

print(remove_hashtags("#AI is awesome"))

print(remove_extra_spaces("Hello     World"))

print(remove_diacritics("السَّلَامُ عَلَيْكُمْ"))

print(remove_tatweel("الســــلام"))

print(remove_emojis("Hello 😊"))

---

Stemming

from arabinlp import ArabicLightStemmer

stemmer = ArabicLightStemmer()

print(stemmer.stem("والمكتبات"))

Expected output

مكتب

---

Root Extraction

from arabinlp import ArabicRootExtractor

extractor = ArabicRootExtractor()

print(extractor.extract("المكتبات"))

Expected output

كتب

---

Lemmatization

from arabinlp import ArabicLemmatizer

lemmatizer = ArabicLemmatizer()

print(lemmatizer.lemmatize("المهندسون"))

print(
    lemmatizer.lemmatize_sentence(
        "المهندسون في الجامعات",
    )
)

print(
    lemmatizer.analyze(
        "المهندسون",
    )
)

---

Morphological Analysis

from arabinlp import (
    ArabicSegmenter,
    ArabicMorphAnalyzer,
)

segmenter = ArabicSegmenter()

print(segmenter.segment("وبالمكتبات"))

print(
    segmenter.segment_sentence(
        "وبالمكتبات كتابهما",
    )
)

analyzer = ArabicMorphAnalyzer()

print(analyzer.analyze("المكتبات"))

print(
    analyzer.analyze_sentence(
        "وبالمكتبات كتابهما",
    )
)

---

Spell Checking

from arabinlp import ArabicSpellChecker

checker = ArabicSpellChecker()

print(checker.is_correct("رائع"))

print(
    checker.distance(
        "راءع",
        "رائع",
    )
)

print(checker.suggest("راءع"))

print(
    checker.correct(
        "اللغه العربيه جميله",
    )
)

Expected output

اللغة العربية جميلة

---

TF-IDF

from arabinlp import ArabicTfidfVectorizer

docs = [
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي يعالج اللغة العربية",
    "معالجة اللغة الطبيعية مجال مهم",
]

vectorizer = ArabicTfidfVectorizer(
    ngram_range=(1, 2),
    min_df=1,
)

matrix = vectorizer.fit_transform(docs)

print(vectorizer.vocabulary_)
print(matrix)

---

BM25

from arabinlp import ArabicBM25

docs = [
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي يعالج اللغة العربية",
    "معالجة اللغة الطبيعية مجال مهم",
]

bm25 = ArabicBM25()

bm25.fit(docs)

print(
    bm25.get_scores(
        "الذكاء الاصطناعي",
    )
)

print(
    bm25.search(
        "الذكاء الاصطناعي",
        top_k=2,
    )
)

---

Examples

Additional runnable examples are available in the GitHub repository:

• examples/showcase.py
• examples/sample.txt
• examples/sample_errors.txt

Repository:

https://github.com/IbtihalMakki/arabic-nlp-toolkit

---

Running Tests

The automated test suite is available in the GitHub repository.

git clone https://github.com/IbtihalMakki/arabic-nlp-toolkit
cd arabic-nlp-toolkit
python -m pytest

---

Roadmap

Current Release (v2.1.2)

Implemented:

• Complete preprocessing pipeline
• Arabic normalization
• Cleaning utilities
• Tokenization
• Stopword removal
• Sentence splitting
• Keyword extraction
• Text statistics
• Word & character n-grams
• Language detection
• Jaccard similarity
• Arabic Light Stemmer
• Arabic Root Extractor
• Arabic Lemmatizer
• Morphological Segmentation
• Morphological Analysis
• Arabic Spell Checker
• TF-IDF Vectorizer
• BM25 Ranking & Retrieval

---

Planned

• Named Entity Recognition (NER)
• AraBERT integration
• Additional pretrained Arabic NLP pipelines
• Larger benchmarking datasets

---

Contributing

Contributions, bug reports, feature requests, and pull requests are welcome.

---

License

This project is licensed under the MIT License.

---

Author

Ibtihal Makki

AI Engineer • NLP Researcher • Generative AI

GitHub: https://github.com/IbtihalMakki