netspresso 1.1.0

PyNetsPresso

PyNetsPresso

YOLOX | YOLOv8 | YOLOv5 | PIDNet | PyTorch-CIFAR-Models

Use PyNetsPresso for a seamless model optimization process. PyNetsPresso resolves AI-related constraints in business use cases and enables cost-efficiency and enhanced performance by removing the requirement for high-spec servers and network connectivity and preventing high latency and personal data breaches.

The PyNetsPresso is a python interface with the NetsPresso web application and REST API.

Easily compress various models with our resources. Please browse the Docs for details, and join our Discussion Forum for providing feedback or sharing your use cases.

To get started with the PyNetsPresso, you will need to sign up either at NetsPresso or PyNetsPresso.

Steps	Types	Description
Train (Model Zoo)	Image Classification PyTorch-CIFAR-Models Object Detection YOLOX YOLOv5 Semantic Segmentation PIDNet Pose Estimation YOLOv8	Build and train models.
Compress	np.compressor	Compress and optimize the user’s pre-trained model.
Convert	np.launcher	Convert AI models to run efficiently on the desired hardware and provide easy installation for seamless usage of the converted AI models.

Installation

There are two ways you can install the PyNetsPresso: using pip or manually through our project GitHub repository.

To install this package, please use Python 3.8 or higher.

From PyPI (Recommended)

pip install netspresso

From Github

git clone https://github.com/nota-netspresso/pynetspresso.git
pip install -e .

Quick Start

Login

To use the PyNetsPresso, please enter the email and password registered in NetsPresso.

from netspresso.client import SessionClient
from netspresso.compressor import ModelCompressor

session = SessionClient(email='YOUR_EMAIL', password='YOUR_PASSWORD')
compressor = ModelCompressor(user_session=session)

Upload Model

To upload your trained model, simply enter the required information.

When a model is successfully uploaded, a unique 'model.model_id' is generated to allow repeated use of the uploaded model.

from netspresso.compressor import Task, Framework

model = compressor.upload_model(
    model_name="YOUR_MODEL_NAME",
    task=Task.IMAGE_CLASSIFICATION,
    framework=Framework.TENSORFLOW_KERAS,
    file_path="YOUR_MODEL_PATH", # ex) ./model.h5
    input_shapes="YOUR_MODEL_INPUT_SHAPES",  # ex) [{"batch": 1, "channel": 3, "dimension": [32, 32]}]
)

Automatic Compression

Automatically compress the model by setting the compression ratio for the model.

Enter the ID of the uploaded model, the name and storage path of the compressed model, and the compression ratio.

compressed_model = compressor.automatic_compression(
    model_id=model.model_id,
    model_name="YOUR_COMPRESSED_MODEL_NAME",
    output_path="OUTPUT_PATH",  # ex) ./compressed_model.h5
    compression_ratio=0.5,
)

Convert Model and Benchmark the Converted Model

Convert an ONNX model into a TensorRT model, and benchmark the TensorRT model on the Jetson Nano.

from loguru import logger
from netspresso.launcher import ModelConverter, ModelBenchmarker
from netspresso.launcher.utils.devices import filter_devices_with_device_name
from netspresso.launcher.schemas import ModelFramework, TaskStatus, DeviceName
from netspresso.launcher.schemas.model import BenchmarkTask, ConversionTask, Model, TargetDevice

converter = ModelConverter(user_session=session)

model: Model = converter.upload_model("./examples/sample_models/test.onnx")

available_devices: list[TargetDevice] = filter_devices_with_device_name(name=DeviceName.JETSON_NANO,
                                                                        devices=model.available_devices)
target_device = available_devices[0] # Jetson Nano - Jetpack 4.6
conversion_task: ConversionTask = converter.convert_model(model=model,
                                                            input_shape=model.input_shape,
                                                            target_framework=ModelFramework.TENSORRT,
                                                            target_device=available_devices[0],
                                                            wait_until_done=True)

logger.info(conversion_task)

CONVERTED_MODEL_PATH = "converted_model.trt"
converter.download_converted_model(conversion_task, dst=CONVERTED_MODEL_PATH)


benchmarker = ModelBenchmarker(user_session=session)
benchmark_model: Model = benchmarker.upload_model(CONVERTED_MODEL_PATH)
benchmark_task: BenchmarkTask = benchmarker.benchmark_model(model=benchmark_model,
                                                            target_device=target_device,
                                                            wait_until_done=True)
logger.info(f"model inference latency: {benchmark_task.latency} ms")
logger.info(f"model gpu memory footprint: {benchmark_task.memory_footprint_gpu} ms")
logger.info(f"model cpu memory footprint: {benchmark_task.memory_footprint_cpu} ms")

NetsPresso Model Compressor Best Practice

If you want to experience Model Compressor online without any installation, please refer to the NetsPresso-Model-Compressor-ModelZoo repo that runs on Google Colab.

Contact

Join our Discussion Forum for providing feedback or sharing your use cases, and if you want to talk more with Nota, please contact us here.
Or you can also do it via email([e-mail]) or phone[phone])!