darija-router 0.2.0

Moroccan Darija (Arabic) phrase analyzer — classify each word by source language (French/English/Darija), detect semantic entities (money, time, date, phone...) and strip morphological prefixes. Built in Rust for O(1) lookup.

darija-router

Moroccan Darija language router — O(1) phrase analysis with morphological stripping and semantic tagging. Built in Rust, wrapped for Python.

Classifies Arabic-script Darija words by source language (fr, en, darija), Latin spelling, category, probability, and semantic role.

Install

pip install darija-router

Quick Start

import darija_router

# "Brother, I want to meet you tomorrow at 3 AM in the square,
#  do you want us to buy a new phone for 500 dirhams?"
result = darija_router.analyze(
    "خويا باغي نلقاك غدوة الساعة 3 صباح فالساحة، "
    "واش باغي نشريو شي تيليفون جديد ب 500 درهم؟"
)

Response:

[
  {
    "token": "خويا",
    "index": 0,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "family_relationships" }
    ],
    "stem": null,
    "semantic": null
  },
  {
    "token": "باغي",
    "index": 1,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "نلقاك",
    "index": 2,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "غدوة",
    "index": 3,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "time_weather" }
    ],
    "stem": null,
    "semantic": "DATE"
  },
  {
    "token": "الساعة",
    "index": 4,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "time_weather" }
    ],
    "stem": {
      "matched_stem": "ساعة",
      "stripped_prefix": "ال",
      "stripped_suffix": null,
      "normalized": false
    },
    "semantic": "TIME"
  },
  {
    "token": "3",
    "index": 5,
    "entries": [],
    "stem": null,
    "semantic": "TIME"
  },
  {
    "token": "صباح",
    "index": 6,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "time_weather" }
    ],
    "stem": null,
    "semantic": "TIME"
  },
  {
    "token": "فالساحة",
    "index": 7,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "،",
    "index": 8,
    "entries": [],
    "stem": null,
    "semantic": "PUNCT"
  },
  {
    "token": "واش",
    "index": 9,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "slang_fillers" }
    ],
    "stem": null,
    "semantic": null
  },
  {
    "token": "باغي",
    "index": 10,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "نشريو",
    "index": 11,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "شي",
    "index": 12,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "تيليفون",
    "index": 13,
    "entries": [
      { "lang": "fr", "latin_word": "téléphone", "prob": 100.0, "category": "technology_digital" }
    ],
    "stem": null,
    "semantic": null
  },
  {
    "token": "جديد",
    "index": 14,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "ب",
    "index": 15,
    "entries": [],
    "stem": null,
    "semantic": null
  },
  {
    "token": "500",
    "index": 16,
    "entries": [],
    "stem": null,
    "semantic": "MONEY"
  },
  {
    "token": "درهم",
    "index": 17,
    "entries": [
      { "lang": "darija", "latin_word": null, "prob": 100.0, "category": "administration_official" }
    ],
    "stem": null,
    "semantic": "MONEY"
  },
  {
    "token": "؟",
    "index": 18,
    "entries": [],
    "stem": null,
    "semantic": "PUNCT"
  }
]

Breakdown:

Token	Lang	Category	Semantic	Notes
خويا	darija	family	—	exact match — "my brother"
باغي	—	—	—	Darija verb (I want)
نلقاك	—	—	—	Darija verb (I meet you)
غدوة	darija	time	DATE	exact match — "tomorrow"
الساعة	darija	time	TIME	stripped ال → ساعة (hour)
3	—	—	TIME	context: follows الساعة
صباح	darija	time	TIME	exact match — "morning"
فالساحة	—	—	—	Darija (in the square)
،	—	—	PUNCT	Arabic comma
واش	darija	slang	—	exact match — question particle
باغي	—	—	—	Darija verb (you want)
نشريو	—	—	—	Darija verb (we buy)
شي	—	—	—	Darija (a/some)
تيليفون	fr	tech	—	exact match — téléphone
جديد	—	—	—	Darija adjective (new)
ب	—	—	—	Darija preposition (for/with)
500	—	—	MONEY	context: followed by درهم
درهم	darija	admin	MONEY	exact match — "dirham"
؟	—	—	PUNCT	Arabic question mark

Semantic Tags

Each token gets a semantic field with rule-based detection:

Tag	Detection	Example
PUNCT	Punctuation characters	؟ ، . !
CARDINAL	Digits or Arabic number words	50 عشرين مية
DECIMAL	Decimal numbers	3.5 ١٢.٥
MONEY	Cardinal + currency word	50 درهم → both tokens tagged
MEASURE	Cardinal + unit word	2 كيلو → both tokens tagged
TIME	الساعة + cardinal, or time words	الساعة 3 صباح مساء
DATE	Date words or cardinal + month	غدوة 15 مارس
ORDINAL	Arabic ordinal words	تاني تالت أول
FRACTION	Arabic fraction words	نص ربع تلت
PERCENT	% suffix or بالمئة	20%
PHONE	Phone number patterns	[phone] 0612345678
EMAIL	Contains @ and .	[e-mail]
URL	Starts with http or www.	http://google.com
ADDRESS	Address keyword + following word	زنقة الحسن

Single Word Lookup

darija_router.lookup("بون")

{
  "token": "بون",
  "index": 0,
  "entries": [
    { "lang": "fr", "latin_word": "bon", "prob": 20.01, "category": "adjectives_expressions" },
    { "lang": "fr", "latin_word": "bonne", "prob": 13.33, "category": "adjectives_expressions" },
    { "lang": "fr", "latin_word": "bon", "prob": 33.33, "category": "money_commerce" },
    { "lang": "fr", "latin_word": "pont", "prob": 33.33, "category": "transportation_vehicles" }
  ],
  "stem": null,
  "semantic": null
}

API Reference

darija_router.analyze(phrase: str) -> list[dict]

Tokenizes a phrase and classifies each token. Returns a list of result objects:

Field	Type	Description
token	str	The original token
index	int	Position in the phrase (0-based)
entries	list[dict]	Classification results (empty if not found)
stem	dict or null	Morphological info if a prefix/suffix was stripped
semantic	str or null	Semantic tag (PUNCT, CARDINAL, MONEY, TIME, DATE, etc.)

darija_router.lookup(word: str) -> dict

Looks up a single word. Same result format as one element from analyze().

darija_router.__version__

Package version string.

How It Works

1. Embedded lexicon — 1,595 Arabic-script keys compiled into the binary via include_str!(). Zero file I/O, zero startup cost.

2. O(1) HashMap lookup — exact match against a HashMap<String, Vec<Entry>>.

3. Morphological fallback — when an exact match fails, the engine strips prefixes and suffixes:

   Prefixes: ال و ف ب ل م ك

   Suffixes: هم ها ش ك ي

   Normalization: ة → ه

   Tries: exact → normalize → single strip → double strip. Minimum stem length of 2 prevents over-stripping.

4. Semantic tagger — rule-based second pass detects 14 semantic categories via regex patterns and context-aware grouping (e.g. cardinal + currency word → MONEY).

5. 17 thematic categories — transportation, technology, food, health, clothing, home, education, sports, work, money, adjectives, family, colors, time, nature, administration, slang.

Build from Source

git clone https://github.com/LaamiriOuail/darija-router.git
cd darija-router
pip install maturin
maturin develop --release

License

MIT