A Comprehensive Toolkit for Scene Text Recognition in Indian Languages

IndicPhotoOCR is a scene text recognition toolkit designed for detecting, identifying, and recognizing text across Indian languages, including Assamese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu, and English, with support for Urdu and Meitei in the pipeline. It is built to handle the unique scripts and complex structures of Indian languages, offering robust detection and recognition capabilities. The package can be installed with just few lines of code, and a straightforward wrapper function makes it easy to use.

Table of Content

Updates
Installation
How to use
Bharat Scene Text Dataset
Contributors
Acknowledgement
Contact us

Updates

[Feburary 2025]: Added option to choose between tri-lingual and 12 class script identifiction models.
[Feburary 2025]: Added recoginition models for Malayalam and Kannada.
[January 2025]: Added ViT based script identification models.
[January 2025]: Demo available in huggingface space.
Currently demo supports scene images containing bi-lingual Hindi and English text.
[December 2024]: Detection Module: TextBPN++ added.
[November 2024]: Code available at Google Colab.
[November 2024]: Added support for 10 languages in the recognition module.
[September 2024]: Repository created.

Installation

Currently we need to manually create virtual environemnt.

conda create -n indicphotoocr python=3.9 -y
conda activate indicphotoocr


git clone https://github.com/Bhashini-IITJ/IndicPhotoOCR.git
cd IndicPhotoOCR

python setup.py sdist bdist_wheel
pip install dist/IndicPhotoOCR-1.3.0-py3-none-any.whl[cpu]

python setup.py sdist bdist_wheel
pip install ./dist/IndicPhotoOCR-1.3.0-py3-none-any.whl[cu118] --extra-index-url https://download.pytorch.org/whl/cu118

python setup.py sdist bdist_wheel
pip install ./dist/IndicPhotoOCR-1.3.0-py3-none-any.whl[cu121] --extra-index-url https://download.pytorch.org/whl/cu121

If you find any trouble with the above installation use the setup.sh script. Make sure you have miniconda installed.

git clone https://github.com/Bhashini-IITJ/IndicPhotoOCR.git
cd IndicPhotoOCR
chmod +x setup.sh
./setup.sh

Config

Currently this model works for hindi v/s english script identification and thereby hindi and english recognition.

Detection Model: TextBPN++
ScripIndetification Model: Hindi v/s English
Recognition Model: Hindi, English, Assamese, Bengali, Gujarati, Marathi, Odia, Punjabi, Tamil, Telugu, Malayalam, kannada.

How to use

Detection

>>> from IndicPhotoOCR.ocr import OCR
# Create an object of OCR
>>> ocr_system = OCR(verbose=True) # for CPU --> OCR(device="cpu")

# Get detections
>>> detections = ocr_system.detect("test_images/image_141.jpg")

# Running text detection...
# 4334 text boxes before nms
# 1.027989387512207

# Save and visualize the detection results
>>> ocr_system.visualize_detection("test_images/image_141.jpg", detections)
# Image saved at: test.png

Cropped Word Script Identification

>>> from IndicPhotoOCR.ocr import OCR
# Create an object of OCR
>>> ocr_system = OCR(verbose=True,  identifier_lang ="auto") # for CPU --> OCR(device="cpu")
# Get script id
>>> ocr_system.identify("test_images/cropped_image/image_141_0.jpg")
# 'hindi'

Cropped Word Recognition

>>> from IndicPhotoOCR.ocr import OCR
# Create an object of OCR
>>> ocr_system = OCR(verbose=True) # for CPU --> OCR(device="cpu")
# Get recognitions
>>> ocr_system.recognise("test_images/cropped_image/image_141_0.jpg", "hindi")
# Recognizing text in detected area...
# 'मण्डी'

End-to-end Scene Text Recognition

>>> from IndicPhotoOCR.ocr import OCR
# Create an object of OCR
>>> ocr_system = OCR(verbose=True, identifier_lang ="auto") # for CPU --> OCR(device="cpu")
# Complete pipeline
>>> ocr_system.ocr("test_images/image_141.jpg")
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Recognized word: रोड
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Recognized word: बाराखम्बा
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Recognized word: राजीव
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Recognized word: चौक
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Recognized word: मण्डी
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Recognized word: हाऊस
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Using cache found in /root/.cache/torch/hub/baudm_parseq_main
# Recognized word: rajiv
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Using cache found in /root/.cache/torch/hub/baudm_parseq_main
# Recognized word: chowk
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Using cache found in /root/.cache/torch/hub/baudm_parseq_main
# Recognized word: mandi
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Using cache found in /root/.cache/torch/hub/baudm_parseq_main
# Recognized word: house
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Using cache found in /root/.cache/torch/hub/baudm_parseq_main
# Recognized word: barakhamba
# Identifying script for the cropped area...
# Recognizing text in detected area...
# Using cache found in /root/.cache/torch/hub/baudm_parseq_main
# Recognized word: road

Related Large-Scale Indian Language Scene Text Dataset

Bharat Scene Text Dataset - BSTD

Contributors

Anik De
Tech Lead and Main Contributor

Abhirama	Aditya Rathore	Harshiv Shah
Contributor	Contributor	Contributor

Sagar Agarwal	Rajeev Yadav	Pravin Kumar
Contributor	Contributor	Contributor

Anand Mishra
Project Investigator

Citation

@misc{ipo,
  author = {Anik De et al.}
  title      = {{I}ndic{P}hoto{O}CR: A comprehensive toolkit for {I}ndian language scene text understanding},
  howpublished = {\url{https://github.com/Bhashini-IITJ/IndicPhotoOCR/}},
  year         = 2024,
}

Acknowledgement

Text Recognition - PARseq
EAST re-implemenation repository.
TextBPN++ repository.
National Language Translation Mission Bhashini.

Contact us

For any queries, please contact us at:

• Anik De

Changelog

[1.3.1] - 2025-17-25

Added

• Seperate function for script identification
• Added device opts for recognition

[1.3.0] - 2025-17-25

Added

• ViT models for script detection
• Word order based on horizontal and vertical lines

[1.2.0] - 2024-11-24

Added

• Textbpn++ detection model added

[1.1.0] - 2024-11-24

Added

• Updated package naming convention

[1.0.3] - 2024-11-06

Added

• Python package requirements sorted with setup.py

[1.0.2] - 2024-11-01

Added

• Added language support for 10 additional models in the recognition module.

[1.0.1] - 2024-10-28

Added

• Added support for detecting polygonal bounding boxes in visualize_detection.
• Introduced the show argument in visualize_detection to control image display.