OCR Foundry

Synthetic OCR training data generator with legacy Khmer/English behavior and opt-in multilingual whole-line rendering. Use ocr-foundry; existing khocr-gen commands run the same CLI. The v0.2 distribution ocr-foundry installs both executables. Python imports remain khocr_gen, and Rust and KHOCR_GEN_* environment contracts are unchanged.

See branding and compatibility for migration details. Repository links to https://github.com/0x1d1e/ocr-foundry are valid now.

Installation

pip install ocr-foundry

pip install ocr-foundry is the v0.2 distribution. It provides both the ocr-foundry and khocr-gen commands.

The PyPI project khocr-gen is deprecated. Release 0.2.0 contains no code and depends on this package. khocr-gen==0.1.13 is the last standalone release. If khocr-gen==0.1.13 is already installed, upgrade or remove it before installing ocr-foundry, because both distributions would otherwise provide the khocr-gen command.

The rust extra is not installable from PyPI because khocr-gen-core is not published. Native use is a source checkout with uv sync --extra dev --extra rust.

Requirements

• Python >= 3.12
• Rust toolchain (cargo/rustc, edition 2021) - only needed to build the optional acceleration extension

Quick Start

1. Prepare fonts

fonts/
├── khmer/     <- .ttf / .otf Khmer fonts
└── english/   <- .ttf / .otf English fonts

Fonts placed directly in fonts/ (not in a subdirectory) are added to both pools as fallbacks.

2. Prepare corpus

A plain UTF-8 text file, one string per line:

សួស្តី
Hello World
ស្វាគមន៍ Welcome

3. Generate a dataset

ocr-foundry generate --corpus corpus/corpus.txt --fonts fonts --output data --copies 3 --storage lmdb

4. View generated images

# Summary of an LMDB database
ocr-foundry view --lmdb data/train/lmdb

# Extract images
ocr-foundry view --lmdb data/train/lmdb --output-dir extracted

# Show labels only
ocr-foundry view --lmdb data/train/lmdb --labels-only

Multilingual mode and evaluation

--multilingual requires Pillow RAQM and renders horizontal logical Unicode lines using one covering font by default, with opt-in cluster-aware font fallback (--cluster-font-fallback) segmenting grapheme clusters, preserving combining marks and Khmer coeng sequences, and aligning multi-run segments to a shared baseline. Labels retain normalized logical text without visual reordering. Arbitrary recursive font pools, explicit NFC/no-normalization policies, and auto/ltr/rtl direction are supported. JSONL manifests ingest rich metadata, emit provenance.jsonl sidecars, and record font hashes in metadata.jsonl.

The source recipe configs/multilingual-realistic.yml includes clean samples, disables rare oversampling and foreground crops, and enables a measured line-ink height guard, with opt-in binary glyph-mask guards (--min-glyph-height-for-augmentation) and script/font intensity scaling. Page layout composition (ocr-foundry layout) adds opt-in whitespace word annotations, native RAQM vertical writing (words and vertical modes requiring shared libraqm with linked FreeType), and paginated table hierarchy. Rendering checks are not a universal language-quality claim; see the capability matrix and page layout contract.

ocr-foundry evaluate --reference reference.jsonl --predictions predictions.jsonl --normalization NFC

The offline evaluator reports corpus CER, exact-line accuracy and reference-language groups from ID-keyed JSONL. It downloads no model. See the schema and frozen real-document protocol and provenance/migration roadmap.

Configuration

All generation parameters can be specified via CLI flags, a YAML config file, or both (CLI overrides YAML). See CONFIG.md for details. Example: generate.yml

Config loading order (lowest to highest priority):

1. argparse built-in defaults
2. YAML config file values
3. Explicit CLI flags

CLI Reference

See CLI_REFERENCE.md for complete command documentation.

Documentation

• Getting Started for a first dataset.
• GitHub wiki for usage guides and troubleshooting.
• In-repository references for the CLI, configuration, augmentation, and Rust extension.

Acknowledgments

OCR Foundry builds on these open-source libraries:

• khmernormalizer (MIT) - Khmer text normalization
• OpenCV (Apache 2.0) - Image processing and augmentation
• Pillow (HPND) - Font rendering and image creation
• Albumentations (MIT) - Augmentation primitives
• NumPy (BSD-3-Clause) - Numerical operations
• LMDB (OpenLDAP 2.8) - Embedded database for training pipelines

License

MIT