arabinlp 2.0.0

A modern Python toolkit for Arabic Natural Language Processing (NLP).

Arabic NLP Toolkit

A lightweight, modular, and easy-to-use Python library for Arabic Natural Language Processing (NLP).

The toolkit provides common Arabic text preprocessing utilities including cleaning, normalization, tokenization, stopword removal, sentence segmentation, keyword extraction, similarity measurement, and more.

---

Features

Text Cleaning

• Remove URLs
• Remove Mentions (@username)
• Remove Hashtags (#hashtag)
• Remove Emojis
• Remove Arabic Diacritics (Tashkeel)
• Remove Tatweel (ـ)
• Remove Extra Spaces

Arabic NLP

• Arabic Text Normalization
• Arabic Tokenization
• Arabic Stopword Removal
• Sentence Splitting
• Word Frequency Analysis
• Word N-Grams
• Character N-Grams
• Keyword Extraction
• Language Detection
• Text Statistics
• Jaccard Text Similarity

---

Installation

pip install arabic-nlp-toolkit

Note

The package is currently under active development.

---

Quick Start

from arabinlp import preprocess

text = """
السَّلَامُ عَلَيْكُمْ 😊

زوروا موقعنا:
https://example.com

#الذكاء_الاصطناعي
"""

print(preprocess(text))

Output

السلام عليكم زوروا موقعنا: الذكاء_الاصطناعي

---

API Overview

from arabinlp import (
    preprocess,
    normalize_arabic,
    tokenize,
    remove_stopwords,
    split_sentences,
    word_frequency,
    ngrams,
    character_ngrams,
    detect_language,
    text_statistics,
    extract_keywords,
    jaccard_similarity,
)

---

Examples

Tokenization

from arabinlp import tokenize

tokenize("السلام عليكم ورحمة الله")

Output

['السلام', 'عليكم', 'ورحمة', 'الله']

---

Stopword Removal

from arabinlp import remove_stopwords

remove_stopwords("أنا أحب تعلم الذكاء الاصطناعي")

Output

['أحب', 'تعلم', 'الذكاء', 'الاصطناعي']

---

Sentence Splitting

from arabinlp import split_sentences

split_sentences(
    "السلام عليكم. كيف حالك؟ أنا بخير!"
)

Output

[
    'السلام عليكم.',
    'كيف حالك؟',
    'أنا بخير!'
]

---

Word Frequency

from arabinlp import word_frequency

word_frequency(
    "الذكاء الذكاء الاصطناعي رائع"
)

Output

{
    'الذكاء': 2,
    'الاصطناعي': 1,
    'رائع': 1
}

---

Word N-Grams

from arabinlp import ngrams

ngrams(
    "السلام عليكم ورحمة الله",
    n=2
)

Output

[
    ('السلام', 'عليكم'),
    ('عليكم', 'ورحمة'),
    ('ورحمة', 'الله')
]

---

Character N-Grams

from arabinlp import character_ngrams

character_ngrams(
    "السلام",
    n=3
)

Output

[
    'الس',
    'لسل',
    'سلا',
    'لام'
]

---

Language Detection

from arabinlp import detect_language

detect_language("Hello مرحبا")

Output

mixed

---

Text Statistics

from arabinlp import text_statistics

text_statistics(
    "السلام عليكم. كيف حالك؟"
)

Output

{
    'characters': 24,
    'characters_without_spaces': 20,
    'words': 4,
    'unique_words': 4,
    'sentences': 2
}

---

Keyword Extraction

from arabinlp import extract_keywords

extract_keywords(
    "الذكاء الاصطناعي رائع. الذكاء يتطور بسرعة."
)

Output

[
    'الذكاء',
    'الاصطناعي',
    'رائع'
]

---

Text Similarity

from arabinlp import jaccard_similarity

score = jaccard_similarity(
    "أنا أحب الذكاء الاصطناعي",
    "الذكاء الاصطناعي رائع"
)

print(score)

Output

0.4

---

Project Structure

arabic-nlp-toolkit/

├── arabinlp/
├── tests/
├── .github/
├── README.md
├── LICENSE
├── pyproject.toml
└── requirements.txt

---

Running Tests

python -m pytest

Current Status

40 tests passed

---

Roadmap

Version 1.1

• Arabic Light Stemmer
• Root Extraction
• TF-IDF
• BM25 Ranking
• Arabic Spell Checker
• Named Entity Recognition (NER)
• AraBERT Integration

---

Contributing

Contributions, bug reports, feature requests, and pull requests are always welcome.

If you have ideas for improving Arabic NLP support, feel free to open an Issue.

---

License

This project is licensed under the MIT License.

---

Author

Ibtihal Makki

AI Engineer • NLP Researcher • Generative AI