Mobilint Model Zoo

mblt-model-zoo is a curated collection of AI models optimized by Mobilint’s Neural Processing Units (NPUs).

Designed to help developers accelerate deployment, Mobilint's Model Zoo offers access to public, pre-trained, and pre-quantized models for vision, language, and multimodal tasks. Along with performance results, we provide pre- and post-processing tools to help developers evaluate, fine-tune, and integrate the models with ease.

Installation

• Prepare environment equipped with Mobilint's NPU. In case you are not a Mobilint customer, please contact us.
• Install mblt-model-zoo using pip:

pip install mblt-model-zoo

• If you want to install the latest version from the source, clone the repository and install it:

git clone https://github.com/mobilint/mblt-model-zoo.git
cd mblt-model-zoo
pip install -e .

Release notes are tracked in CHANGELOG.md.

Vision

Vision models, model metadata, preprocessing, postprocessing, datasets, evaluation, and Python API documentation are maintained in mblt-vision-python. Install it and import from mblt_vision for new applications. mblt_model_zoo.vision remains available as a forwarding-only compatibility facade for existing code.

Model Zoo retains compatibility bridges for the Vision CLI and compilation. Vision benchmarks and dataset-management workflows are maintained in mblt-vision-python.

Optional Extras

When working with tasks other than vision, extra dependencies may be required. Those options can be installed via pip install mblt-model-zoo[NAME] or pip install -e .[NAME].

Currently, these optional functions are only available on environment equipped with Mobilint's ARIES.

The qbcompiler extra is strictly isolated from ordinary package use. qbcompiler is loaded only when compile_vision_model() or mblt-model-zoo compile actually starts compilation. The base package, vision APIs, compilation module import, and non-compile CLI commands continue to work without qbcompiler installed; only a compilation request reports the installation error.

For the transformers extra, the repository also includes:

• functional test instructions in tests/transformers/TEST.md
• benchmark script usage in benchmark/transformers/README.md

Note: The MeloTTS extra includes unidic, which requires an additional dictionary download step. Python packaging (PEP 517/518) does not support running arbitrary post-install commands automatically, so run mblt-unidic-download (or python -m unidic download) after installing the extra when needed.

Command Line Interface

Installing this package exposes the mblt-model-zoo console command:

mblt-model-zoo -h

The CLI provides Mobilint-specific helper commands and delegates selected upstream Hugging Face Transformers commands to the installed transformers package.

The built-in command surface shown by mblt-model-zoo -h is:

• predict — run classification, depth estimation, object or face detection, instance or semantic segmentation, OBB, and pose inference.
• val — validate a vision model on its benchmark dataset.
• compile — compile a configured vision ONNX model to MXQ.
• tps measure and tps sweep — run Transformers token-per-second benchmarks.
• melo — run the MeloTTS CLI; melotts is an alias.
• melo-ui — launch the MeloTTS Gradio WebUI.

Run mblt-model-zoo <command> -h for argparse-based commands. melo and melotts are Click-based and use --help.

Compile a configured vision ONNX model with the optional compiler dependency:

mblt-model-zoo compile --model-cls alexnet

The matching Python API is mblt_model_zoo.compile.vision.compile_vision_model. New applications should use mblt_vision.compile.compile_vision_model; see the standalone Vision compilation guide for calibration datasets and options. Invoking this API or command is the only point where qbcompiler is imported.

When paths are omitted, compilation stores downloaded ONNX models and compiled MXQ outputs under ~/.mblt_model_zoo, and uses registry datasets under ~/.mblt_model_zoo/datasets. It does not derive these defaults from the current checkout or working directory.

Compilation accepts one of three data entry levels: --data-path for a full organized image dataset, --subset-path for already-sampled images, or --calib-data-path for ready preprocessed .npy tensors. Later-stage input skips all earlier preparation stages. --model-path also accepts the --onnx-path compatibility alias, and --calib-data-path accepts the --calib-data-dir alias.

Vision Prediction And Validation

The vision CLI runs the same preprocess, NPU inference, postprocess, and plotting pipeline used by the Python API. Use predict with a source image and a model name; the task is inferred from the model configuration. It supports image classification, depth estimation, object and face detection, instance and semantic segmentation, oriented bounding boxes (OBB), and pose estimation. classify, detect, pose, and segment are also accepted as aliases.

mblt-model-zoo predict --source ./cat.png --model resnet50
mblt-model-zoo predict --source ./street.jpg --model yolo11m --output ./result_detect.jpg

Vision commands accept a shared --model-path for local MXQ and local ONNX files. When --framework is omitted, the CLI infers .mxq and .onnx suffixes and otherwise falls back to MXQ. If the explicit framework conflicts with the local file suffix, the command fails with a clear error. The compatibility flags --mxq-path and --onnx-path stay separate from --model-path, so framework-specific resolution still works when both a local MXQ artifact and an explicit ONNX runtime are involved.

ONNX inference uses ONNX Runtime's CPUExecutionProvider by default. This avoids probing TensorRT, CUDA, or other accelerators during normal package use. Python callers that intentionally need another provider can pass its ordered provider list through MBLT_Engine(onnx_providers=...).

mblt-model-zoo predict --source ./cat.png --model resnet50 --model-path ./resnet50.mxq
mblt-model-zoo predict --source ./cat.png --model resnet50 --model-path ./resnet50.onnx
mblt-model-zoo predict --source ./cat.png --model resnet50 --framework onnx
mblt-model-zoo predict --source ./cat.png --model resnet50 --framework onnx --mxq-path ./resnet50.mxq
mblt-model-zoo predict --source ./cat.png --model resnet50 --framework onnx --onnx-path ./resnet50.onnx

Prediction results are saved under runs/vision/predict/ by default. Pass --output or --save-path to choose a specific output file. Classification models accept --topk; object detection, instance segmentation, and pose estimation models accept --conf-thres and --iou-thres. Depth and semantic segmentation save colorized overlays without detection thresholds. Both predict and val accept --e2e to enable end-to-end YOLO postprocessing; provide true or false, or pass the bare flag to enable it. predict --raw-output PATH saves the export-style model output when end-to-end postprocessing is disabled.

mblt-model-zoo predict --source ./cat.png --model resnet50 --topk 5
mblt-model-zoo predict --source ./street.jpg --model yolo11m --conf-thres 0.5 --iou-thres 0.5

Use val to validate a supported vision model on its benchmark dataset. Classification models use ImageNet, object detection, instance segmentation, and pose estimation models use COCO. YOLO26 *-sem models use Cityscapes, while *-sem-ade20k models keep their independent ADE20K pipeline. Validation also supports --framework onnx, the shared --model-path override, and the framework-specific compatibility aliases. Pass --data-path for an already organized validation dataset; otherwise the CLI uses the default cache location. --force-organize (also --force or --reload) rebuilds an organized dataset, while --image-dir, --xml-dir, and --annotation-dir override the local archive paths or download URLs used by automatic organization.

mblt-model-zoo val --model resnet50
mblt-model-zoo val --model yolo11m --batch-size 8 --conf-thres 0.001 --iou-thres 0.7
mblt-model-zoo val --model resnet50 --model-path ./resnet50.mxq
mblt-model-zoo val --model resnet50 --model-path ./resnet50.onnx
mblt-model-zoo val --model resnet50 --framework onnx
mblt-model-zoo val --model resnet50 --framework onnx --mxq-path ./resnet50.mxq
mblt-model-zoo val --model resnet50 --framework onnx --onnx-path ./resnet50.onnx
mblt-model-zoo val --model yolo26n-sem-ade20k --framework onnx \
  --data-path ~/.mblt_model_zoo/datasets/ADEChallengeData2016
mblt-model-zoo val --model yolo26n-sem --framework onnx \
  --data-path ~/.mblt_model_zoo/datasets/cityscapes

Common NPU and artifact options are shared by the vision commands:

mblt-model-zoo predict \
  --source ./cat.png \
  --model resnet50 \
  --model-type DEFAULT \
  --model-path /path/to/model.mxq \
  --core-mode global8 \
  --dev-no 0

Use --core-mode single, multi, global4, or global8 to select the NPU execution mode. For manual placement, pass semicolon-separated values with --target-cores, such as 0:0;0:1, or --target-clusters, such as 0;1. Full vision CLI details and supported model names are available in mblt_model_zoo/vision/README.md.

The canonical wire form for NPU targets is fully-qualified: --target-cores 0:0:0;0:0:1;1:0:0 (device : cluster : core) and --target-clusters 0:0;1:0 (device : cluster). Legacy 2-part c:k cores and bare integers still work — they are migrated to the canonical form using --dev-no as the device prefix. Passing --dev-no as a scalar keeps single-device behavior; passing it as a list (e.g. [0, 1] when embedded in a config) spreads slots across those devices without listing every core by hand.

TPS Benchmark Helpers

The tps command measures token-per-second performance for Transformers-based text-generation and image-text-to-text pipelines. It requires the transformers extra.

pip install "mblt-model-zoo[transformers]"
mblt-model-zoo tps measure --help
mblt-model-zoo tps sweep --help

tps measure accepts --temperature FLOAT (default 0.0) to sample instead of greedy-decoding. A value of 0.0 keeps the current greedy behavior; any value greater than zero enables do_sample=True with that temperature. tps sweep remains greedy so its numbers stay comparable.

On VLM (--task image-text-to-text) pipelines whose language model uses the fake-prefill decode path, tps measure decode TPS is measured with a greedy torch.argmax and the CLI rejects --temperature > 0 with a clear error. Use --temperature 0 (default) for VLM decode TPS.

--decode N forces exactly N new tokens on non-speculative models. For EAGLE-3 speculative decode, N is an upper bound: generation stops at the configured EOS and reported TPS is computed over the tokens actually produced.

--batch-size B sets the aggregate batch capacity. B maps to the model's max_batch_size, which the runtime resolves to N qbruntime.Model slots so that N * K >= B, where K is the compiled MXQ batch axis. A non-batch MXQ (K == 1) with B > 1 therefore fans out into N = B Model slots that dispatch in parallel across the target device set (see --dev-no, --target-cores, --target-clusters); a batched MXQ (K > 1) reuses hardware batching until N * K >= B. Beam search paths remain N = 1. Legacy configs that store the older 2-part target_cores / bare-int target_clusters are silently upgraded to the canonical form on load, so no explicit migration step is required.

Batched MXQ execution (K > 1) is only supported under --core-mode single; other core modes are rejected at runtime. The text-generation and VLM benchmark scripts enforce this by exiting with SystemExit("batch benchmark only supports --core-mode single") when a batch run is paired with any other explicit --core-mode, and the batch text-generation test suite is likewise pinned to single (see mblt_model_zoo/hf_transformers/README.md). (In batch mode the benchmark scripts also skip their non-batch default --target-cores 0:0 injection, so batched runs rely on the config's default target_cores or an explicit --target-cores.)

The text-generation benchmark script also accepts --batch-size and --dev-no on both measure and sweep. --batch-size N overrides config.max_batch_size for the effective input batch dim, and, on Mobilint targets only, forwards the same value as the backend max_batch_size kwarg; on upstream/original Hugging Face targets it stays a measurement-only override. Passing --batch --original-models --batch-size N with N > 1 therefore admits an upstream target whose config reports max_batch_size == 1. --dev-no on non-Mobilint targets is a silent no-op, so a mixed Mobilint-vs-GPU sweep can share one CLI. See benchmark/transformers/README.md for the full example.

tps measure --print-output is a diagnostic flag that decodes and prints the tokens actually generated by the last measured run in two versions (special tokens preserved, then cleaned). Use it to visually confirm whether an EOS token terminated decoding before the --decode budget. The trailing footer separates the TTFT sample from decode tokens using the same convention as decode_tps: it reports X decode tokens (+ 1 TTFT sample = Y emitted; --decode N max) so the count matches the measured throughput.

For thinking-capable models (e.g., Qwen3), tps measure exposes the mutually exclusive --enable-thinking and --disable-thinking flags to override the enable_thinking argument passed to tokenizer.apply_chat_template. When neither is set the tokenizer default is used, so existing runs are unaffected. Use --disable-thinking to prevent a small --decode budget from being consumed entirely by the <think> block; use --enable-thinking to force the block on.

Detailed TPS benchmark examples are available in benchmark/transformers/README.md.

MeloTTS Helpers

The melo command, also available as melotts, forwards arguments to the MeloTTS Click CLI. The melo-ui command launches the MeloTTS Gradio WebUI. These commands require the MeloTTS extra. melo-ui accepts --share, --host, and --port; use melo --help to see the MeloTTS text, language, speaker, speed, device, and local-file options.

pip install "mblt-model-zoo[MeloTTS]"
mblt-model-zoo melo --help
mblt-model-zoo melotts --help
mblt-model-zoo melo-ui --help

Delegated Transformers Commands

When the first argument is one of add-fast-image-processor, add-new-model-like, chat, convert, download, env, run, serve, or version, mblt-model-zoo delegates execution to the installed Transformers CLI. For chat and serve, the CLI installs Mobilint model registration hooks when the delegated Transformers backend loads models through the local serve command path.

Verbose Option

By default, model initialization stays quiet. To print the model file size and MD5 hash whenever an MXQ model loads, set the environment variable MBLT_MODEL_ZOO_VERBOSE to a truthy value before running your script:

export MBLT_MODEL_ZOO_VERBOSE=true  # accepted values: true/1/yes/on (case-insensitive)
python your_script.py

Example Verbose Output

Model Initialized
Model Size: 216.94 MB
Model Hash: 23c262c43b4c1c453dd0326e249480a0
Device Number: 0
Core Mode: single
Target Cores: [CoreId(cluster=Cluster.Cluster0, core=Core.Core0)]
Model Variant 0
        Input Shape: [(1, 200, 96), (1, 200, 96), (2, 200, 200)]
        Output Shape: [(1, 102400, 1)]
Model Variant 1
        Input Shape: [(1, 300, 96), (1, 300, 96), (2, 300, 300)]
        Output Shape: [(1, 153600, 1)]
Model Variant 2
        Input Shape: [(1, 400, 96), (1, 400, 96), (2, 400, 400)]
        Output Shape: [(1, 204800, 1)]
Model Variant 3
        Input Shape: [(1, 500, 96), (1, 500, 96), (2, 500, 500)]
        Output Shape: [(1, 256000, 1)]
Model Variant 4
        Input Shape: [(1, 600, 96), (1, 600, 96), (2, 600, 600)]
        Output Shape: [(1, 307200, 1)]
Model Variant 5
        Input Shape: [(1, 900, 96), (1, 900, 96), (2, 900, 900)]
        Output Shape: [(1, 460800, 1)]

Unset or set the variable to any other value to suppress these messages.

License

The Mobilint Model Zoo is released under BSD 3-Clause License. Please see the LICENSE file for more details.

Additionally, the license for each model provided in this package follows the terms specified in the source link provided with it.

Support & Issues

If you encounter any problems with this package, please feel free to contact us.