langid-indian 0.1.4

A language identifier for Indian languages using a pretrained Random Forest model.

langid_indian

A language identifier for Indian languages using a pretrained Machine Learning model.

---

📌 Overview

langid_indian is a Python package for language identification of Indian languages written in native scripts. It is based on a pretrained Machine Learning model and supports multiple languages listed in the Eighth Schedule of the Indian Constitution.

The library is designed to be:

• Lightweight and easy to use
• Suitable for NLP pipelines, chatbots, multilingual datasets, and research
• Minimal in preprocessing requirements

Recommended citation title (as suggested): "Language Identification using langid_indian library."

---

🔗 Project Links

• GitHub Repository: https://github.com/yashingle-ai/Indian-Language-Identification-based-on-ML
• PyPI Package: https://pypi.org/project/langid-indian/

---

📦 Installation

Using Terminal / Command Prompt

pip install langid_indian

Using Jupyter Notebook / Google Colab / Kaggle

!pip install langid_indian

---

🚀 Quick Start

import langid_indian

# Initialize the language identifier
lid = langid_indian.LanguageIdentifier()

# Predict the language of a single sentence
print(lid.predict("जब मैं छोटा था, मैं हर रोज़ पार्क जाता था।"))
# Output: hin

---

🔍 Predicting Multiple Sentences

test_texts = [
    "Hello, how are you?",
    "जब मैं छोटा था, मैं हर रोज़ पार्क जाता था।",
    "आनी हो एक गंभीर मूर्खपणा.",
    "ਮਨੁੱਖੀ ਦਿਮਾਗ਼ ਦੀ ਕਾਢ ਨੇ ਭਾਵੇਂ ਸਭ ਕੁਝ ਸੌਖਾ ਕਰ ਦਿੱਤਾ ਹੈ ਪਰ ਫਿਰ ਵੀ ਸਭ ਕੁਝ ਸਮਝਣਾ ਜਾਂ ਕਰਨਾ ਨਿਯਮਾਂ ਵਿੱਚ ਬੱਝਾ ਪਿਆ ਹੈ।",
    "માં વરસાદનું પાણી મોટા જથ્થામાં જમીનની નીચે જ ઉતરી જાય છે।",
    "କିନ୍ତୁ ପୁଅ, ତୁମେ ଛୋଟ।"
]

for text in test_texts:
    print(f"Text: {text} | Detected language: {lid.predict(text)}")

Sample Output

Text: Hello, how are you? | Detected language: eng
Text: जब मैं छोटा था, मैं हर रोज़ पार्क जाता था। | Detected language: hin
Text: आनी हो एक गंभीर मूर्खपणा. | Detected language: gom
Text: ਮਨੁੱਖੀ ਦਿਮਾਗ਼ ਦੀ ਕਾਢ ਨੇ ਭਾਵੇਂ ਸਭ ਕੁਝ ਸੌਖਾ ਕਰ ਦਿੱਤਾ ਹੈ ਪਰ ਫਿਰ ਵੀ ਸਭ ਕੁਝ ਸਮਝਣਾ ਜਾਂ ਕਰਨਾ ਨਿਯਮਾਂ ਵਿੱਚ ਬੱਝਾ ਪਿਆ ਹੈ। | Detected language: pan
Text: માં વરસાદનું પાણી મોટા જથ્થામાં જમીનની નીચે જ ઉતરી જાય છે। | Detected language: guj
Text: କିନ୍ତୁ ପୁଅ, ତୁମେ ଛୋଟ। | Detected language: ory

---

🧪 Testing the Package

The repository includes a basic test file located at:

tests/test_basic.py

test_basic.py

from langid_indian import LanguageIdentifier

def test_prediction():
    identifier = LanguageIdentifier()
    text = "यह एक परीक्षण है"
    lang = identifier.predict(text)
    print(f"Predicted language: {lang}")

if __name__ == "__main__":
    test_prediction()

Run the Test

From the project root directory:

python -m tests.test_basic

---

🌐 Supported Languages

The model supports 22 Indian languages:

• Assamese (অসমীয়া)
• Bengali (বাংলা)
• Bodo
• Dogri
• English
• Gujarati (ગુજરાતી)
• Hindi (हिन्दी)
• Kannada (ಕನ್ನಡ)
• Kashmiri
• Konkani
• Maithili
• Malayalam (മലയാളം)
• Manipuri
• Marathi (मराठी)
• Nepali
• Odia (ଓଡ଼ିଆ)
• Punjabi (ਪੰਜਾਬੀ)
• Santali
• Sindhi
• Tamil (தமிழ்)
• Telugu (తెలుగు)
• Urdu (اردو)

---

📄 Citation

If you use langid_indian in academic or research work, please cite the following paper:

BibTeX

@misc{ingle2025ilidnativescriptlanguage,
  title={ILID: Native Script Language Identification for Indian Languages},
  author={Yash Ingle and Pruthwik Mishra},
  year={2025},
  eprint={2507.11832},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2507.11832}
}

---

👤 Author

Yash Ingle B.Tech (AI), SVNIT Surat

---

📜 License

This project is licensed under the MIT License.

---

⭐ Acknowledgement

This work is part of ongoing research on Indian Language Identification using Machine Learning and Native Script Text, developed under academic guidance.