Skip to main content

Arabic multi-label toxic content classifier

Project description

arabic-toxic

Arabic multi-label toxic content classifier for content moderation.

Install

pip install arabic-toxic

Usage

from arabic_toxic import classify, predict_proba, classify_many

print(classify("يلعن امك يا ابن الحمار"))

texts = [
    "أنت إنسان محترم",
    "يلعن امك يا ابن الحمار"
]
print(classify_many(texts))

Output example

{
  "text": "يلعن امك يا ابن الحمار",
  "labels": ["Cussing"],
  "top_label": "Cussing",
  "is_toxic": True,
  "scores": {
    "Cussing": 99.1,
    "Violence": 0.4
  }
}

Labels

  • Appearance
  • Cussing
  • Hatred
  • NOT
  • Racial
  • Sexual
  • Violence

Model

This package uses Arabic normalization, TF-IDF word/character n-grams, and a multi-label One-vs-Rest SGD logistic classifier.

Test-set results from the provided split:

Metric Score
Macro F1 0.9109
Micro F1 0.9205
Subset Accuracy 0.8433
Hamming Loss 0.0307

Notes

This model is intended for Arabic content-moderation research and production prototyping. Always validate on your own domain data before relying on automated moderation decisions.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

arabic_toxic-0.2.0.tar.gz (3.4 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

arabic_toxic-0.2.0-py3-none-any.whl (3.4 MB view details)

Uploaded Python 3

File details

Details for the file arabic_toxic-0.2.0.tar.gz.

File metadata

  • Download URL: arabic_toxic-0.2.0.tar.gz
  • Upload date:
  • Size: 3.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.7

File hashes

Hashes for arabic_toxic-0.2.0.tar.gz
Algorithm Hash digest
SHA256 aabad6d6cee1ce391f3e88fe4a3b8411018e57a16818b65161c4eec5ba0b431b
MD5 5fa74339aa6153c754222c42b1e929bf
BLAKE2b-256 4e9360cbe8afcf44bd4bdfa83ee9547cfe2eae97611ebb4ea01408efbde35e6f

See more details on using hashes here.

File details

Details for the file arabic_toxic-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: arabic_toxic-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 3.4 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.7

File hashes

Hashes for arabic_toxic-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 a436f3944f4822004843be1ee220b3b2826dd86565a269004556f98971b8a309
MD5 2efad4625e5b7e6308a5499bee8532f8
BLAKE2b-256 9f2382e9f207092a3bb23a315a347fcf5097dc48c0d09789b7f1415ee27e8909

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page