sinapsis-glm-ocr 0.1.1

Templates for optical character recognition using the GLM-OCR model

Sinapsis GLM OCR

GLM-OCR-based Optical Character Recognition (OCR) for images

🐍 Installation • 🚀 Features • 📚 Usage example • 📙 Documentation • 🔍 License

Sinapsis GLM OCR provides a powerful implementation for extracting text from images using Zhipu AI's GLM-OCR model. Built on the GLM-V encoder-decoder architecture, it supports document parsing (text, formula, table recognition) and structured information extraction via JSON schema prompts. It also supports batch inference for faster processing of multiple images.

🐍 Installation

Install using your package manager of choice. We encourage the use of uv

Example with uv:

  uv pip install sinapsis-glm-ocr --extra-index-url https://pypi.sinapsis.tech

or with raw pip:

  pip install sinapsis-glm-ocr --extra-index-url https://pypi.sinapsis.tech

[!IMPORTANT] Templates may require extra dependencies. For development, we recommend installing the package with all the optional dependencies:

with uv:

  uv pip install sinapsis-glm-ocr[all] --extra-index-url https://pypi.sinapsis.tech

or with raw pip:

  pip install sinapsis-glm-ocr[all] --extra-index-url https://pypi.sinapsis.tech

[!TIP] Use CLI command sinapsis info --all-template-names to show a list with all the available Template names installed with Sinapsis OCR.

[!TIP] Use CLI command sinapsis info --example-template-config GLMOCRInference to produce an example Agent config for the GLMOCRInference template.

🚀 Features

Templates Supported

This module includes templates tailored for the GLM-OCR engine:

• GLMOCRInference: Uses GLM-OCR model to extract text from images. Supports document parsing (text, formula, table) and structured information extraction.
• GLMOCRBatchInference: Batch inference version for processing multiple images efficiently.

GLMOCRInference Attributes

• prompt (str): The prompt to send to the model. Defaults to "Text Recognition:". Other options include "Formula Recognition:" and "Table Recognition:".
• init_args (GLMOCRInitArgs): Initialization arguments for the model including:
  • pretrained_model_name_or_path: Model identifier. Defaults to "zai-org/GLM-OCR".
  • torch_dtype: Model precision ("float16", "bfloat16", "auto"). Defaults to "auto".
  • attn_implementation: Attention implementation ("kernels-community/flash-attn2", "kernels-community/paged-attention"). Defaults to "kernels-community/flash-attn2".
  • device_map: Device mapping ("auto", "balanced", "balanced_low_0", "sequential", or specific device like "cuda:0"). Defaults to "auto".
  • Note: This model requires CUDA. CPU inference is not supported.
• generation_config (GLMOCRGenerationConfig): Generation configuration including:
  • max_new_tokens: Maximum tokens to generate. Defaults to 8192.
  • min_new_tokens: Minimum tokens to generate. Defaults to 1.
  • do_sample: Whether to use sampling. Defaults to False.
  • repetition_penalty: Penalty for repeating tokens. Defaults to 1.0.
  • length_penalty: Penalty for sequence length. Defaults to 1.0.

📚 Usage example

Text Recognition

agent:
  name: glm_ocr_agent
  description: Agent to run inference with GLM OCR for text recognition

templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}

- template_name: FolderImageDatasetCV2
  class_name: FolderImageDatasetCV2
  template_input: InputTemplate
  attributes:
    data_dir: dataset/input

- template_name: GLMOCRInference
  class_name: GLMOCRInference
  template_input: FolderImageDatasetCV2
  attributes:
    prompt: "Text Recognition:"
    init_args:
      pretrained_model_name_or_path: zai-org/GLM-OCR
      torch_dtype: auto
      attn_implementation: kernels-community/flash-attn2
      device_map: auto
    generation_config:
      max_new_tokens: 8192
      do_sample: false

Table Recognition

agent:
  name: glm_ocr_table_agent
  description: Agent to run inference with GLM OCR for table recognition

templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}

- template_name: FolderImageDatasetCV2
  class_name: FolderImageDatasetCV2
  template_input: InputTemplate
  attributes:
    data_dir: dataset/input

- template_name: GLMOCRInference
  class_name: GLMOCRInference
  template_input: FolderImageDatasetCV2
  attributes:
    prompt: "Table Recognition:"
    init_args:
      pretrained_model_name_or_path: zai-org/GLM-OCR
      torch_dtype: auto
      attn_implementation: kernels-community/flash-attn2
      device_map: auto
    generation_config:
      max_new_tokens: 8192
      do_sample: false

Information Extraction (JSON Schema)

agent:
  name: glm_ocr_json_agent
  description: Agent to extract structured information using JSON schema

templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}

- template_name: FolderImageDatasetCV2
  class_name: FolderImageDatasetCV2
  template_input: InputTemplate
  attributes:
    data_dir: dataset/input

- template_name: GLMOCRInference
  class_name: GLMOCRInference
  template_input: FolderImageDatasetCV2
  attributes:
    prompt: |
      Extract the information from the provided image and output
      it strictly as a valid JSON object matching the following schema.
      {
        "name": "string",
        "date": "string",
        "amount": "number"
      }
    init_args:
      pretrained_model_name_or_path: zai-org/GLM-OCR
      torch_dtype: auto
      attn_implementation: kernels-community/flash-attn2
      device_map: auto
    generation_config:
      max_new_tokens: 8192
      do_sample: false

Batch Inference

agent:
  name: glm_ocr_batch_agent
  description: Agent to run batch inference with GLM OCR for faster processing

templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}

- template_name: FolderImageDatasetCV2
  class_name: FolderImageDatasetCV2
  template_input: InputTemplate
  attributes:
    data_dir: dataset/input
    batch_size: 4

- template_name: GLMOCRBatchInference
  class_name: GLMOCRBatchInference
  template_input: FolderImageDatasetCV2
  attributes:
    prompt: "Text Recognition:"
    init_args:
      pretrained_model_name_or_path: zai-org/GLM-OCR
      torch_dtype: auto
      attn_implementation: kernels-community/flash-attn2
      device_map: auto
    generation_config:
      max_new_tokens: 8192
      do_sample: false

To run, simply use:

sinapsis run name_of_the_config.yml

📙 Documentation

Documentation for this and other sinapsis packages is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.