AH-Translit-Bench 1.0.0

A benchmark dataset for Arabic to Hindi transliteration.

AH-Translit_Bench: Arabic to Hindi Transliteration Benchmark Dataset

Repository Description

AH-Translit_Bench is a 2000-entry Arabic to Hindi transliteration benchmark dataset covering Al-Quran, bibliographical, and Modern Standard Arabic (MSA) domains. Provided as a pip-installable Python package for model testing and research.

Dataset Overview

A benchmark dataset for Arabic to Hindi transliteration, providing parallel transliterated text pairs across diverse domains.

Dataset Curators

This dataset was proudly curated by:

• Vilal Ali
  Research Scholar at Data Sciences And Analytics Centre, IIIT Hyderabad,
  Email: [e-mail]
• Mohd Hozaifa Khan
  Research Scholar at Center for Visual Information Technology (CVIT) IIIT Hyderabad Email: [e-mail]

Dataset Usage

This dataset is ideal for training, validating, and testing models for Arabic to Hindi transliteration tasks.

Content Type

Text, specifically Arabic source text and its corresponding Hindi transliterated target text.

Dataset Description

The AH-Translit_Bench dataset provides a collection of parallel text pairs for evaluating and developing Arabic to Hindi transliteration systems. It comprises text from three distinct domains: Al-Quran, bibliographical entries, and Modern Standard Arabic (MSA), ensuring a broad coverage for robust model development. Each entry consists of an Arabic string and its manually curated transliteration into Hindi (Devanagari script). The dataset's structure allows for straightforward loading and integration into various machine learning frameworks.

Version Name

AH-Translit_Bench_v1.0

File Type

CSV (Comma Separated Values)

Version Overview

This is the initial release of the AH-Translit_Bench dataset, offering a foundational benchmark for Arabic to Hindi transliteration research.

Version Description

This inaugural version of the AH-Translit_Bench dataset, version 1.0, provides a comprehensive collection of 2000 Arabic-Hindi transliteration pairs. The dataset is segmented into three domain-specific files to cater to varied research needs: al-quran_test_bench_mark_500.csv containing 500 entries from the Al-Quran domain, biblo_test_bench_mark_1000.csv with 1000 entries from the bibliographical domain, and mse_test_bench_mark_500.csv offering 500 entries from the Modern Standard Arabic (MSA) domain. Each entry consists of an Arabic string and its corresponding, carefully transliterated Hindi equivalent, making it suitable for training and evaluating transliteration models.

Dataset Structure

The dataset is provided as a zip file (though usually downloaded directly via pip) containing three separate .csv files, each corresponding to a different domain. The structure within each CSV file is simple: the first column contains the Arabic text, and the second column contains its transliterated Hindi equivalent.

Author's

Vilal Ali

Mohd Hozaifa Khan