anltk 1.0.8

Arabic Natural Language Toolkit (ANLTK)

Arabic Natural Language Toolkit (ANLTK)

ANLTK is a set of Arabic natural language processing tools. developed with focus on simplicity and performance.

ANLTK is a C++ library, with python bindings

Installation

for python :

pip install anltk

Building

Note: Currently only tested on Linux, prebuilt python wheels are available for Linux, Windows, Macos on pypi

Dependencies

• utfcpp, automatically downloaded.
• utf8proc, automatically downloaded.
• C++ Compiler that supports c++17.
• Python3, meson, ninja
• Task (optional, for simplified build commands)

Building C++ Library

git clone https://github.com/Abdullah-AlAttar/anltk.git
cd anltk/

# Using taskfile (recommended)
task configure
task build
task test

# Or manually with meson
meson build --buildtype=release -Dbuild_tests=false
cd build
ninja

Building Python Bindings

# Complete setup (creates venv, installs deps, builds package)
task py:setup

# Or step by step:
task py:venv              # Create virtual environment
task py:deps              # Install build dependencies
task py:install           # Install in development mode

# Test the installation
task py:test              # Run quick tests

# Build wheel for distribution
task py:wheel             # Build wheel package

# Clean build artifacts
task clean                # Clean all build artifacts

Manual Python Build (without taskfile)

python3 -m venv .venv
.venv/bin/pip install --upgrade pip meson-python build pybind11 ninja patchelf
.venv/bin/pip install -e .

Usage Examples

C++ API

#include "anltk/anltk.hpp"
#include <iostream>
#include <string>

int main()
{

    std::string ar_text = "أبجد هوز حطي كلمن سعفص قرشت ثخذ ضظغ";

    std::cout << anltk::transliterate(ar_text, anltk::CharMapping::AR2BW) << '\n';
    // >bjd hwz HTy klmn sEfS qr$t vx* DZg

    std::string text = "فَرَاشَةٌ مُلَوَّنَةٌ تَطِيْرُ في البُسْتَانِ، حُلْوَةٌ مُهَنْدَمَةٌ تُدْهِشُ الإِنْسَانَ.";

    std::cout << anltk::remove_tashkeel(text) << '\n';
    // فراشة ملونة تطير في البستان، حلوة مهندمة تدهش الإنسان.

    // Third paramters is a stop_list, charactres in this list won't be removed
    std::cout << anltk::remove_non_alpha(text, " ") << '\n';
    // فراشة ملونة تطير في البستان حلوة مهندمة تدهش الإنسان

    anltk::TafqitOptions opts;
    std::cout<< anltk::tafqit(15000120, opts) <<'\n';
    // خمسة عشر مليونًا ومائة وعشرون
}

Python API

import anltk


ar = "أبجد هوز حطي كلمن سعفص قرشت ثخذ ضظغ"
bw = anltk.transliterate(ar, anltk.AR2BW)
print(bw)
# >bjd hwz HTy klmn sEfS qr$t vx* DZg

print(anltk.remove_tashkeel("فَرَاشَةٌ مُلَوَّنَةٌ تَطِيْرُ في البُسْتَانِ، حُلْوَةٌ مُهَنْدَمَةٌ تُدْهِشُ الإِنْسَانَ."))

# فراشة ملونة تطير في البستان، حلوة مهندمة تدهش الإنسان.

print(anltk.tafqit(15000120))
# خمسة عشر مليونًا ومائة وعشرون

For list of features see Features.md

Benchmarks

Processing a file containing 500000 Line, 6787731 Word, 112704541 Character. the task is to remove diacritics / transliterate to buckwalter

Buckwatler transliteration

Method	Time
anltk python-api	1.379 seconds
python camel_tools	11.46 seconds

Remove Diacritics

Method	Time
anltk python-api	0.989 seconds
python camel_tools	4.892 seconds