UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition
Project description
Welcome to the official repository of UniMERNet, a solution that converts images of mathematical expressions into LaTeX, suitable for a wide range of real-world scenarios.
News
2024.4.24 🎉🎉 Paper now available on ArXiv.
2024.4.24 🎉🎉 Inference code and checkpoints have been released.
Quick Start
Clone the repo and download the model
git clone https://github.com/opendatalab/UniMERNet.git
cd UniMERNet/models
# Download the model and tokenizer individually or use git-lfs
git lfs install
git clone https://huggingface.co/wanderkid/unimernet
Installation
conda create -n unimernet python=3.10
conda activate unimernet
pip install unimernet
Run the demo
Input an image and predict the LaTeX code
python demo.py
Input an image and obtain the LaTeX code and corresponding rendered image
jupyter-lab ./demo.ipynb
Performance Comparison (BLEU) with SOTA Methods.
UniMERNet significantly outperforms mainstream models in recognizing real-world mathematical expressions, demonstrating superior performance across Simple Printed Expressions (SPE), Complex Printed Expressions (CPE), Screen-Captured Expressions (SCE), and Handwritten Expressions (HWE), as evidenced by the comparative BLEU Score evaluation.
Visualization Result with Different Methods.
UniMERNet excels in visual recognition of challenging samples, outperforming other methods.
TODO
- Release inference code and checkpoints of UniMERNet.
- Release UniMER-1M and UniMER-Test.
- Release the training code for UniMERNet.
Citation
If you find our models / code / papers useful in your research, please consider giving us a star ⭐ and citing our work 📝, thank you :)
@misc{wang2024unimernet,
title={UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition},
author={Bin Wang and Zhuangcheng Gu and Chao Xu and Bo Zhang and Botian Shi and Conghui He},
year={2024},
eprint={2404.15254},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Acknowledgements
- VIGC. The model framework is dependent on VIGC.
- Texify. A mainstream MER algorithm, UniMERNet data processing refers to Texify.
- Latex-OCR. Another mainstream MER algorithm.
- Donut. The UniMERNet's Transformer Encoder-Decoder are referenced from Donut.
- Nougat. The tokenizer uses Nougat.
Contact Us
If you have any questions, comments, or suggestions, please do not hesitate to contact us at wangbin@pjlab.org.cn.
License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for unimernet-0.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a244751d8fdaba607192b053caa29fc6b4e3683c6e46659af825556f7653a380 |
|
MD5 | 440c4a0f8618d149be7338be707a3e64 |
|
BLAKE2b-256 | 43d494d8257314c4d0beb39c905865f137efbe27740c3bf02c30c8afbda75371 |