ArabiNLP

A lightweight, dependency-free Python toolkit for Arabic Natural Language Processing (NLP), providing preprocessing, normalization, morphology, stemming, lemmatization, spell checking, and information retrieval utilities.

---

Version 2.1.0

Features

✔ Text cleaning

✔ Arabic normalization

✔ Tokenization

✔ Stopword removal

✔ Sentence splitting

✔ Text statistics

✔ Word and character n-grams

✔ Keyword extraction

✔ Language detection

✔ Jaccard similarity

✔ Arabic Light Stemmer

✔ Root Extraction

✔ Lemmatization

✔ Morphological Segmentation

✔ Morphological Analysis

✔ Arabic Spell Checker

✔ TF-IDF Vectorizer

✔ BM25 Ranking

---

Installation

pip install arabinlp

---

Quick Start

from arabinlp import (
    normalize_arabic,
    tokenize,
    remove_stopwords,
    ArabicLightStemmer,
)

text = "أنا أحب تعلم الذكاء الاصطناعي في الجامعات."

text = normalize_arabic(text)

tokens = tokenize(text)

tokens = remove_stopwords(tokens)

stemmer = ArabicLightStemmer()

stems = [stemmer.stem(token) for token in tokens]

print(stems)

Example output

['احب', 'تعلم', 'ذكاء', 'اصطناع', 'جامع']

---

End-to-End NLP Pipeline

from arabinlp import (
    normalize_arabic,
    tokenize,
    remove_stopwords,
    ArabicLightStemmer,
    ArabicRootExtractor,
    ArabicLemmatizer,
)

text = "والمكتبات جميلة"

text = normalize_arabic(text)

tokens = tokenize(text)

tokens = remove_stopwords(tokens)

stemmer = ArabicLightStemmer()
extractor = ArabicRootExtractor()
lemmatizer = ArabicLemmatizer()

for token in tokens:

    print(
        token,
        stemmer.stem(token),
        extractor.extract(token),
        lemmatizer.lemmatize(token),
    )

---

Python Support

• Python >= 3.9

Supported versions

• Python 3.9
• Python 3.10
• Python 3.11
• Python 3.12

---

Project Links

Homepage

https://github.com/IbtihalMakki/arabic-nlp-toolkit

Repository

https://github.com/IbtihalMakki/arabic-nlp-toolkit

Issues

https://github.com/IbtihalMakki/arabic-nlp-toolkit/issues

---

API Overview

API	Description
preprocess	Complete preprocessing pipeline
normalize_arabic	Normalize Arabic text
tokenize	Tokenization
remove_stopwords	Stopword removal
split_sentences	Sentence segmentation
word_frequency	Word frequency
ngrams	Word n-grams
character_ngrams	Character n-grams
detect_language	Language detection
text_statistics	Text statistics
extract_keywords	Keyword extraction
jaccard_similarity	Text similarity
ArabicLightStemmer	Light stemming
ArabicRootExtractor	Root extraction
ArabicLemmatizer	Lemmatization
ArabicSegmenter	Morphological segmentation
ArabicMorphAnalyzer	Morphological analysis
ArabicSpellChecker	Spell checker
ArabicTfidfVectorizer	TF-IDF vectorizer
ArabicBM25	BM25 ranking

---

Core Preprocessing

from arabinlp import (
    preprocess,
    normalize_arabic,
    tokenize,
    remove_stopwords,
    split_sentences,
    word_frequency,
    ngrams,
    character_ngrams,
    detect_language,
    text_statistics,
    extract_keywords,
    jaccard_similarity,
)

text = "السَّلَامُ عَلَيْكُمْ 😊 زوروا https://example.com"

cleaned = preprocess(text)

normalized = normalize_arabic(
    "إسلام وآثار",
    level="light",
)

tokens = tokenize(
    "السلام عليكم ورحمة الله",
)

filtered = remove_stopwords(
    "أنا أحب تعلم الذكاء الاصطناعي في الجامعة",
)

sentences = split_sentences(
    "السلام عليكم. كيف حالك؟ أنا بخير!",
)

freq = word_frequency(
    "الذكاء الذكاء الاصطناعي رائع",
)

word_bigrams = ngrams(
    "السلام عليكم ورحمة الله",
    2,
)

char_trigrams = character_ngrams(
    "السلام",
    3,
)

lang = detect_language(
    "Hello مرحبا",
)

stats = text_statistics(
    "السلام عليكم. كيف حالك؟",
)

keywords = extract_keywords(
    "الذكاء الاصطناعي رائع. الذكاء يتطور بسرعة.",
    top_k=2,
)

sim = jaccard_similarity(
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي رائع",
)

---

Cleaning Utilities

from arabinlp import (
    remove_urls,
    remove_mentions,
    remove_hashtags,
    remove_extra_spaces,
    remove_diacritics,
    remove_tatweel,
    remove_emojis,
)

remove_urls("Visit https://example.com")

remove_mentions("@ibtihal hello")

remove_hashtags("#AI is awesome")

remove_extra_spaces("Hello     World")

remove_diacritics("السَّلَامُ عَلَيْكُمْ")

remove_tatweel("الســــلام")

remove_emojis("Hello 😊")

---

Stemming

from arabinlp import ArabicLightStemmer

stemmer = ArabicLightStemmer()

stem = stemmer.stem("والمكتبات")

print(stem)

Output

مكتب

---

Root Extraction

from arabinlp import ArabicRootExtractor

extractor = ArabicRootExtractor()

root = extractor.extract("المكتبات")

print(root)

Output

كتب

---

Lemmatization

from arabinlp import ArabicLemmatizer

lemmatizer = ArabicLemmatizer()

lemma = lemmatizer.lemmatize("المهندسون")

lemmas = lemmatizer.lemmatize_sentence(
    "المهندسون في الجامعات",
)

analysis = lemmatizer.analyze(
    "المهندسون",
)

---

Morphological Analysis

from arabinlp import (
    ArabicSegmenter,
    ArabicMorphAnalyzer,
)

segmenter = ArabicSegmenter()

segmenter.segment("وبالمكتبات")

segmenter.segment_sentence(
    "وبالمكتبات كتابهما",
)

analyzer = ArabicMorphAnalyzer()

analyzer.analyze("المكتبات")

analyzer.analyze_sentence(
    "وبالمكتبات كتابهما",
)

---

Spell Checking

from arabinlp import ArabicSpellChecker

checker = ArabicSpellChecker()

checker.is_correct("رائع")

checker.distance(
    "راءع",
    "رائع",
)

checker.suggest(
    "راءع",
)

checker.correct(
    "اللغه العربيه جميله",
)

Output

اللغة العربية جميلة

---

TF-IDF

from arabinlp import ArabicTfidfVectorizer

docs = [
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي يعالج اللغة العربية",
    "معالجة اللغة الطبيعية مجال مهم",
]

vectorizer = ArabicTfidfVectorizer(
    ngram_range=(1, 2),
    min_df=1,
)

matrix = vectorizer.fit_transform(docs)

print(vectorizer.vocabulary_)

---

BM25

from arabinlp import ArabicBM25

docs = [
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي يعالج اللغة العربية",
    "معالجة اللغة الطبيعية مجال مهم",
]

bm25 = ArabicBM25()

bm25.fit(docs)

scores = bm25.get_scores(
    "الذكاء الاصطناعي",
)

ranking = bm25.search(
    "الذكاء الاصطناعي",
    top_k=2,
)

---

Examples

The examples/showcase.py script demonstrates all major public APIs in a single executable example.

Run:

python examples/showcase.py

Sample files

examples/sample.txt
examples/sample_errors.txt

---

Running Tests

python -m pytest

---

Roadmap

Implemented (v2.1.0)

• Complete preprocessing pipeline
• Arabic normalization
• Cleaning utilities
• Tokenization
• Stopword removal
• Morphological segmentation
• Morphological analysis
• Arabic Light Stemmer
• Root Extraction
• Lemmatization
• Spell Checker
• TF-IDF
• BM25

All public APIs exported from arabinlp.__init__ are implemented and covered by automated tests.

---

Planned

• Named Entity Recognition (NER)
• AraBERT integration
• Additional pretrained Arabic NLP pipelines
• Larger benchmarking datasets

---

Contributing

Contributions, bug reports, feature requests, and pull requests are welcome.

---

License

MIT License.