mblt-model-zoo 1.3.0

A codebase for pre-quantized AI models for Mobilint NPUs.

Mobilint Model Zoo

mblt-model-zoo is a curated collection of AI models optimized by Mobilint’s Neural Processing Units (NPUs).

Designed to help developers accelerate deployment, Mobilint's Model Zoo offers access to public, pre-trained, and pre-quantized models for vision, language, and multimodal tasks. Along with performance results, we provide pre- and post-processing tools to help developers evaluate, fine-tune, and integrate the models with ease.

Installation

• Prepare environment equipped with Mobilint's NPU. In case you are not a Mobilint customer, please contact us.
• Install mblt-model-zoo using pip:

pip install mblt-model-zoo

• If you want to install the latest version from the source, clone the repository and install it:

git clone https://github.com/mobilint/mblt-model-zoo.git
cd mblt-model-zoo
pip install -e .

Quick Start Guide

Initializing Quantized Model Class

mblt-model-zoo provides a quantized model with associated pre- and post-processing tools. The following code snippet shows how to use the pre-trained model for inference.

from mblt_model_zoo.vision import ResNet50

# Load the pre-trained model. 
# Automatically download the model if not found in the local cache.
resnet50 = ResNet50() 

# Load the model trained with a different recipe
# Currently, the default is "DEFAULT", or "IMAGENET1K_V1".
resnet50 = ResNet50(model_type = "IMAGENET1K_V2")

# Download the model to local directory and load it
resnet50 = ResNet50(local_path = "path/to/local/") # the file will be downloaded to "path/to/local/model.mxq"

# Load the model from a local path or download as filename and file path you want
resnet50 = ResNet50(local_path = "path/to/local/model.mxq")

# Set inference mode for better performance
# ARIES supports "single", "multi", "global4", and "global8" inference mode. Default is "global8"
resnet50 = ResNet50(infer_mode = "global8")

# (Beta) If you are holding a model compiled for REGULUS, enable inference on the REGULUS device.
resnet50 = ResNet50(product = "regulus")

# In summary, the model can be loaded with the following arguments. 
# You may customize those arguments to work with Mobilint's NPU.
resnet50 = ResNet50(
    local_path = None,
    model_type = "DEFAULT",
    infer_mode = "global8",
    product = "aries",
)

Working with Quantized Model

With the image given as a path, PIL image, numpy array, or torch tensor, you can perform inference with the quantized model. The following code snippet shows how to use the quantized model for inference:

image_path = "path/to/image.jpg"

input_img = resnet50.preprocess(image_path) # Preprocess the input image
output = resnet50(input_img) # Perform inference with the quantized model
result = resnet50.postprocess(output) # Postprocess the output

result.plot(
    source_path=image_path,
    save_path="path/to/save/result.jpg",
)

Listing Available Models

mblt-model-zoo offers a function to list all available models. You can use the following code snippet to list the models for a specific task (e.g., image classification, object detection, etc.):

from mblt_model_zoo.vision import list_models
from pprint import pprint

available_models = list_models()
pprint(available_models)

Model List

We provide the models that are quantized with our advanced quantization techniques. A list of available vision models is here.

Optional Extras

When working with tasks other than vision, extra dependencies may be required. Those options can be installed via pip install mblt-model-zoo[NAME] or pip install -e .[NAME].

Currently, these optional functions are only available on environment equipped with Mobilint's ARIES.

Name	Use	Details
transformers	For using HuggingFace transformers related models	README.md
MeloTTS	For using MeloTTS models	README.md

For the transformers extra, the repository also includes:

• functional test instructions in tests/transformers/TEST.md
• benchmark script usage in benchmark/transformers/README.md

Note: The MeloTTS extra includes unidic, which requires an additional dictionary download step. Python packaging (PEP 517/518) does not support running arbitrary post-install commands automatically, so run mblt-unidic-download (or python -m unidic download) after installing the extra when needed.

Verbose Option

By default, model initialization stays quiet. To print the model file size and MD5 hash whenever an MXQ model loads, set the environment variable MBLT_MODEL_ZOO_VERBOSE to a truthy value before running your script:

export MBLT_MODEL_ZOO_VERBOSE=true  # accepted values: true/1/yes/on (case-insensitive)
python your_script.py

Example Verbose Output

Model Initialized
Model Size: 216.94 MB
Model Hash: 23c262c43b4c1c453dd0326e249480a0
Device Number: 0
Core Mode: single
Target Cores: [CoreId(cluster=Cluster.Cluster0, core=Core.Core0)]
Model Variant 0
        Input Shape: [(1, 200, 96), (1, 200, 96), (2, 200, 200)]
        Output Shape: [(1, 102400, 1)]
Model Variant 1
        Input Shape: [(1, 300, 96), (1, 300, 96), (2, 300, 300)]
        Output Shape: [(1, 153600, 1)]
Model Variant 2
        Input Shape: [(1, 400, 96), (1, 400, 96), (2, 400, 400)]
        Output Shape: [(1, 204800, 1)]
Model Variant 3
        Input Shape: [(1, 500, 96), (1, 500, 96), (2, 500, 500)]
        Output Shape: [(1, 256000, 1)]
Model Variant 4
        Input Shape: [(1, 600, 96), (1, 600, 96), (2, 600, 600)]
        Output Shape: [(1, 307200, 1)]
Model Variant 5
        Input Shape: [(1, 900, 96), (1, 900, 96), (2, 900, 900)]
        Output Shape: [(1, 460800, 1)]

Unset or set the variable to any other value to suppress these messages.

License

The Mobilint Model Zoo is released under BSD 3-Clause License. Please see the LICENSE file for more details.

Additionally, the license for each model provided in this package follows the terms specified in the source link provided with it.

Support & Issues

If you encounter any problems with this package, please feel free to contact us.