unlimited-ocr-max

baidu/Unlimited-OCR served through MAX on Apple Silicon. The vision tower, the MoE decoder and its sliding-window attention are rebuilt as a MAX custom architecture with a Mojo custom op (ngram_block, the no-repeat-n-gram guard), and exposed as an OpenAI-compatible endpoint on the Metal GPU or the CPU. The weights are baidu's, unchanged, served from kthierbach/unlimited-ocr-max.

Install

uv tool install --extra-index-url https://whl.modular.com/nightly/simple/ unlimited-ocr-max
# or, into a venv
pip install --extra-index-url https://whl.modular.com/nightly/simple/ unlimited-ocr-max

The extra index is required: the package pins one exact MAX nightly build (max[all]==26.6.0.dev2026082707) because the port depends on fixes no stable MAX release carries yet, and that build is published only on Modular's nightly index. The pin and the flag go away with the next stable Modular release.

Serve

unlimited-ocr-max serve --devices gpu   # Metal
unlimited-ocr-max serve --devices cpu   # supported, slow

This downloads the model repository once (6.2 GiB) and runs max serve with this port's flags, on http://127.0.0.1:8010 under the model id unlimited-ocr-max.

• --revision defaults to v0.1.0, the model-repo tag this package version was validated against, so a fixed package version serves fixed weights; the tag must exist or the download fails before MAX starts. Ignored for a local directory.
• --model <dir> serves a local copy with the repository's layout (config.json, the tokenizer files, model.safetensors).
• --weights bf16 (the only variant today) selects the unquantised model.safetensors and passes it as max serve --weight-path; future quantised variants are model-<variant>.safetensors. The unquantised file has no encoding token in its name because MAX reads hints such as bf16 out of weight filenames and would refuse the CPU path.
• --ngram-size sets the no-repeat-n-gram guard, default 35; 0 switches it off, which reproduces the PyTorch reference byte for byte.

One page per request, base mode, image first:

curl -s http://127.0.0.1:8010/v1/chat/completions -H 'Content-Type: application/json' -d @- <<EOF
{"model": "unlimited-ocr-max", "temperature": 0, "max_tokens": 1766,
 "messages": [{"role": "user", "content": [
   {"type": "text", "text": "<|grounding|>Convert the document to markdown."},
   {"type": "image_url", "image_url": {"url": "data:image/png;base64,$(base64 < page.png)"}}]}]}
EOF

Tested on

Apple M4, 24 GB unified memory, macOS 26.5.2, Python 3.12, max==26.6.0.dev2026082707. The GPU path needs full Xcode plus the Metal Toolchain (xcodebuild -downloadComponent MetalToolchain); the Command Line Tools do not ship the Metal compiler MAX shells out to. Greedy output with the guard off is byte-identical to the fp32 PyTorch reference on both devices. The server is large for a 24 GB machine; run one at a time and leave it the memory.

Not supported

gundam (tiled) mode — the tiling code is in the package and usable in-process, but the served batch holds one resolution; batch sizes above 1 (the prefill graph's sequence length is static); multi-GPU.

References

The model is described in Unlimited OCR Works (Yin et al., 2026), arXiv:2606.23050; this package changes the serving runtime only.

License and authorship

MIT for this code; the weights, tokenizer and config.json are baidu's, redistributed unchanged under baidu's MIT (both notices in LICENSE).