Skip to main content

ComiQ: Comic-Focused Hybrid OCR Library

ComiQ is a Python library built specifically for reading comics and manga. It pairs state-of-the-art OCR detection with an AI vision model (MLLM) to solve the hardest problems in comic text extraction: grouping fragmented word boxes into coherent speech bubbles, classifying bubble types (dialogue, thought, narration, SFX), and cleaning up recognition errors.

For visual capability demonstrations, check the examples directory.


How It Works

┌─────────────────┐       ┌────────────────────────┐       ┌───────────────────────────────┐       ┌──────────────────────┐
│   Comic Image   │ ────► │       OCR Engine       │ ────► │       MLLM + Instructor       │ ────► │  Structured Bubbles  │
│  (file / array) │       │ (PP-OCRv6 / EasyOCR)   │       │ (Bubble Grouping & Cleaning)  │       │ (text, boxes, meta)  │
└─────────────────┘       └────────────────────────┘       └───────────────────────────────┘       └──────────────────────┘
  1. High-Precision OCR: Detects raw word bounding boxes across panels using PP-OCRv6 (unified 50-language SOTA model) or EasyOCR.
  2. AI Layout & Bubble Grouping: An MLLM (via Gemini or any OpenAI-compatible vision endpoint) analyzes the visual page layout and groups scattered word boxes into complete bubbles.
  3. Structured Extraction via Instructor: Schema enforcement with automatic retries guarantees valid JSON and allows extracting custom metadata (character speakers, emotional tone, translations).

Features

  • 🎯 SOTA Comic OCR: Powered by PP-OCRv6 with 3 scalable model tiers (tiny, small, medium) supporting 50 languages in one model.
  • 💬 Intelligent Bubble Grouping: Combines individual word boxes into coherent dialogue, thought bubbles, and narration panels.
  • ✨ OCR Error Correction: The vision model cleans split words, misrecognized punctuation, and manga-specific font quirks.
  • 🏷️ Custom Pydantic Schemas: Powered by Instructor — easily extract speakers, emotion, translation, or narrative tags alongside text.
  • ⚡ GPU & MKL-DNN Accelerated: Fast inference on NVIDIA GPUs (CUDA) or multi-threaded CPU.
  • 🔌 Extensible: Register custom OCR engines (Tesseract, RapidOCR, cloud APIs) with a simple decorator.
  • 🖼️ Flexible Input: Works directly with file paths (.png, .jpg) or in-memory OpenCV / NumPy arrays.

Installation

Install ComiQ with pip:

pip install comiq

This automatically installs:

  • ✅ PaddleOCR 3.x with PP-OCRv6 — SOTA accuracy, 50 languages in a unified model (Python 3.8+)
  • ✅ EasyOCR — Multi-engine fallback supporting CUDA 11.x–13.x
  • ✅ Instructor & OpenAI SDK — Structured MLLM extraction with validation retries

GPU Acceleration (Optional)

ComiQ runs on CPU by default, but NVIDIA GPU acceleration is 10–50× faster.

PP-OCRv6 GPU Support:

  1. Install the PaddlePaddle GPU build matching your CUDA version (CUDA 11.8+ or 12.x) from the PaddlePaddle install guide, e.g.:
    pip install paddlepaddle-gpu -i https://www.paddlepaddle.org.cn/packages/stable/cu126/
    
  2. Pass device="gpu" in your configuration. Windows GPU is supported (including RTX 30, 40, and 50 series).

EasyOCR GPU Support:

Install PyTorch with CUDA:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118

Minimal Installation (EasyOCR Only)

To install without PaddleOCR:

pip install --no-deps comiq
pip install openai instructor python-dotenv pydantic easyocr

Quick Start

1. Set your API Key

ComiQ requires an MLLM API key (Gemini by default). You can pass it directly to ComiQ() or create a .env file in your project root:

MLLM_API_KEY="your-api-key-here"

2. Extract Text from a Comic

import cv2
from comiq import ComiQ

# Initialize ComiQ (loads MLLM_API_KEY from .env)
comiq = ComiQ()

# Process from an image path
data = comiq.extract("path/to/comic_page.png")

# Or process directly from a NumPy array
image_array = cv2.imread("path/to/comic_page.png")
data = comiq.extract(image_array)

for bubble in data:
    print(f"[{bubble['type']}] Panel {bubble['panel_id']}: {bubble['text']}")

OCR Engines & PP-OCRv6

ComiQ supports multiple built-in OCR engines:

# Use default PP-OCRv6 (best accuracy, 50 unified languages)
data = comiq.extract(image_path, ocr="paddleocr")

# Use EasyOCR
data = comiq.extract(image_path, ocr="easyocr")

# Use both engines for maximum coverage
data = comiq.extract(image_path, ocr=["paddleocr", "easyocr"])

PP-OCRv6 Model Tiers

PP-OCRv6 offers three model tiers configurable via the tier parameter:

Tier Parameters Speed (GPU) Best For
tiny 1.5M ~0.2s Edge devices, real-time preview, high-speed batching
small 7.7M ~0.5s Balanced speed and accuracy
medium (default) 34.5M ~1.5s Production quality, complex typography, stylized text
config = {
    "ocr": {
        "paddleocr": {
            "tier": "medium",  # tiny | small | medium
            "device": "gpu",   # cpu | gpu
        }
    }
}
comiq = ComiQ(**config)

Performance & Caching

OCR engine instances are cached per configuration, so the initial model download and initialization only happen once.

Measured inference time on a sample comic page (RTX 4050 GPU / 8-core CPU):

Setup Warm Latency Detections
GPU, medium ~0.2–1.5s High (SOTA)
CPU, tiny (MKL-DNN) ~0.5s Good
CPU, small (MKL-DNN) ~1.4s Great
CPU, medium ~13s High (SOTA)

Tuning tips:

  • tier: The most effective speed dial on CPU (tiny is 7× faster, small is 3× faster than medium).
  • enable_mkldnn: True: Opt-in CPU acceleration (0.5s tiny / 1.4s small). Kept False by default for stability because paddlepaddle 3.3.x has an upstream oneDNN bug. Users on paddlepaddle<=3.2.2 can safely pass enable_mkldnn: True.
  • enable_hpi: True: PaddleOCR's High-Performance Inference (auto TensorRT/OpenVINO). Linux only.

Custom Response Models

Powered by Instructor, ComiQ allows you to extract structured data beyond text by passing your own Pydantic model.

Simply extend comiq.Group (or define a model with a groups list). Any extra fields you define are filled by the vision model from context and included in the output:

from pydantic import BaseModel, Field
from comiq import ComiQ, Group

# 1. Extend the Group model with custom fields
class RichGroup(Group):
    speaker: str = Field("unknown", description="Name of the speaking character, or 'narrator'.")
    emotion: str = Field("neutral", description="Emotional tone: angry, shouting, whisper, calm, etc.")
    translation: str = Field("", description="English translation if the original text is Japanese/foreign.")

class RichAnalysis(BaseModel):
    groups: list[RichGroup]

# 2. Pass your schema to ComiQ
comiq = ComiQ(
    model_name="gemini-3.5-flash-lite",
    response_model=RichAnalysis
)

results = comiq.extract("manga_page.png")

for bubble in results:
    print(f"{bubble['speaker']} ({bubble['emotion']}): {bubble['text']}")
    # Output: Orihime (concerned): BE CAREFUL, CHAD...

Custom Configuration

You can customize AI parameters, OCR settings, and retries:

config = {
    "ocr": {
        "paddleocr": {
            "tier": "medium",
            "device": "gpu",
        },
        "easyocr": {
            "reader": {"gpu": True},
        }
    },
    "ai": {
        "temperature": 0.2,       # Lower = more deterministic
        "max_retries": 3,          # Instructor retry count on schema mismatch
        "instructor_mode": "JSON", # JSON (default), TOOLS, or MD_JSON
    }
}

comiq = ComiQ(
    model_name="gemini-3.5-flash-lite",
    base_url="https://generativelanguage.googleapis.com/v1beta/",
    **config
)

Registering a Custom OCR Engine

You can plug in any third-party OCR library (e.g., Tesseract, RapidOCR, cloud APIs):

import cv2
import numpy as np
import pytesseract
import comiq

# 1. Define the engine function (accepts BGR image + **kwargs)
def pytesseract_engine(image: np.ndarray, **kwargs) -> list:
    rgb_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    data = pytesseract.image_to_data(
        rgb_image,
        output_type=pytesseract.Output.DICT,
        config=kwargs.get("config", "--psm 6")
    )
    
    results = []
    for i in range(len(data['text'])):
        if int(data['conf'][i]) > 60 and data['text'][i].strip():
            x, y, w, h = data['left'][i], data['top'][i], data['width'][i], data['height'][i]
            results.append({
                "text_box": [y, x, y + h, x + w],  # [ymin, xmin, ymax, xmax]
                "text": data['text'][i]
            })
    return results

# 2. Register with ComiQ
comiq.register_ocr_engine("pytesseract", pytesseract_engine)

# 3. Use it in extract()
my_comiq = comiq.ComiQ()
data = my_comiq.extract("image.png", ocr="pytesseract")

API Reference

ComiQ

ComiQ(
    api_key: Optional[str] = None,
    model_name: str = "gemini-3.5-flash-lite",
    base_url: str = "https://generativelanguage.googleapis.com/v1beta/",
    response_model: Optional[Type[BaseModel]] = None,
    **kwargs
)
  • api_key: MLLM API key. If omitted, loaded from MLLM_API_KEY in environment / .env.
  • model_name: Vision model identifier (defaults to "gemini-3.5-flash-lite").
  • base_url: Endpoint URL for the OpenAI-compatible vision service.
  • response_model: Optional custom Pydantic response schema (see Custom Response Models).
  • **kwargs: Nested configuration dictionaries (ocr, ai).

comiq.extract()

extract(
    image: Union[str, np.ndarray],
    ocr: Union[str, List[str]] = "paddleocr"
) -> List[Dict[str, Any]]
  • image: File path (str) or loaded image as a NumPy array (np.ndarray in BGR format).
  • ocr: Engine name ("paddleocr", "easyocr", or custom registered name) or list of names (["paddleocr", "easyocr"]).

Return Schema

Returns a list of dictionaries, each representing an extracted speech bubble:

[
  {
    "panel_id": "1",                                # Panel number
    "text_bubble_id": "1-1",                        # Bubble identifier within panel
    "text_box": [31, 25, 87, 97],                   # Pixel coordinates [ymin, xmin, ymax, xmax]
    "text": "BE CAREFUL, CHAD...",                  # Cleaned & reconstructed text
    "type": "dialogue",                             # dialogue | thought | narration | sound_effect | background
    "style": "normal",                              # normal | emphasized | angled | split
    "notes": "none",                                # AI notes, uncertainties, or SFX justification
    "original_text": "BE CAREFUL, CHAD...",         # Raw OCR text before AI correction
    # + Any extra fields defined in your custom response_model
  },
  ...
]

OCR Registry Functions

register_ocr_engine(name: str, engine: Callable)

Registers a custom OCR function. The engine must accept (image: np.ndarray, **kwargs) and return a list of {"text_box": [ymin, xmin, ymax, xmax], "text": str} dicts.

get_available_ocr_engines() -> List[str]

Returns the list of currently registered OCR engines (default: ['paddleocr', 'paddleocr6', 'ppocrv6', 'easyocr']).


Contributing

Contributions are welcome! Please check our Contributing Guide and Changelog for details on development workflows and guidelines.


License

ComiQ is licensed under the MIT License.

Metadata

Release files for comiq 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for comiq 1.0.0
File Size Uploaded Provenance Actions
comiq-1.0.0.tar.gz 7.7 MB Without provenance Details for comiq-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for comiq 1.0.0
File Interpreter ABI Platform Actions
comiq-1.0.0-py3-none-any.whl Python 3 none any Details for comiq-1.0.0-py3-none-any.whl

Total release size: 7.7 MB

Release files / comiq-1.0.0.tar.gz

Download URL comiq-1.0.0.tar.gz
Size 7.7 MB
Tags Source
SHA-256 checksum
How to use checksums
8782ff426af69c9e7d91a7d588f3d1ecd028f5b4a4b26ec357c3e66cf0909c97
BLAKE2b-256 checksum
How to use checksums
cd5c6e8f31509ecf4d4d860d72bd0f190bd66923afbb6b76d903344970540a1f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.11.6

Release files / comiq-1.0.0-py3-none-any.whl

Download URL comiq-1.0.0-py3-none-any.whl
Size 16.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8adc83629087da303c9acbf934d1c5e503fd33bc3852d03022c6b0200ee42bda
BLAKE2b-256 checksum
How to use checksums
d50f1da06885c51397f60c9adbe05fff8ce4f94fe39b1128942a0d837abdc785
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.11.6

Security for comiq 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page