netspresso 1.2.0

PyNetsPresso

PyNetsPresso

YOLO Fastest | YOLOX | YOLOv8 | YOLOv7 | YOLOv5 | PIDNet | PyTorch-CIFAR-Models

Use PyNetsPresso for a seamless model optimization process. PyNetsPresso resolves AI-related constraints in business use cases and enables cost-efficiency and enhanced performance by removing the requirement for high-spec servers and network connectivity and preventing high latency and personal data breaches.

The PyNetsPresso is a python interface with the NetsPresso web application and REST API.

Easily compress various models with our resources. Please browse the Docs for details, and join our Discussion Forum for providing feedback or sharing your use cases.

To get started with the PyNetsPresso, you will need to sign up either at NetsPresso or PyNetsPresso.

Steps	Types	Description
Train (Model Zoo)	Image Classification PyTorch-CIFAR-Models Object Detection YOLO Fastest YOLOX YOLOv5 YOLOv7 Semantic Segmentation PIDNet Pose Estimation YOLOv8	Build and train models.
Compress	np.compressor	Compress and optimize the user’s pre-trained model.
Convert	np.launcher	Convert AI models to run efficiently on the desired hardware and provide easy installation for seamless usage of the converted AI models.

Installation

There are two ways you can install the PyNetsPresso: using pip or manually through our project GitHub repository.

To install this package, please use Python 3.8 or higher.

From PyPI (Recommended)

pip install netspresso

From Github

git clone https://github.com/nota-netspresso/pynetspresso.git
cd pynetspresso
pip install -e .

Quick Start

Login

To use the PyNetsPresso, please enter the email and password registered in NetsPresso.

from netspresso.client import SessionClient
from netspresso.compressor import ModelCompressor

session = SessionClient(email='YOUR_EMAIL', password='YOUR_PASSWORD')
compressor = ModelCompressor(user_session=session)

Upload Model

To upload your trained model, simply enter the required information.

When a model is successfully uploaded, a unique 'model.model_id' is generated to allow repeated use of the uploaded model.

from netspresso.compressor import Task, Framework

model = compressor.upload_model(
    model_name="YOUR_MODEL_NAME",
    task=Task.IMAGE_CLASSIFICATION,
    framework=Framework.TENSORFLOW_KERAS,
    file_path="YOUR_MODEL_PATH", # ex) ./model.h5
    input_shapes="YOUR_MODEL_INPUT_SHAPES",  # ex) [{"batch": 1, "channel": 3, "dimension": [32, 32]}]
)

Automatic Compression

Automatically compress the model by setting the compression ratio for the model.

Enter the ID of the uploaded model, the name and storage path of the compressed model, and the compression ratio.

compressed_model = compressor.automatic_compression(
    model_id=model.model_id,
    model_name="YOUR_COMPRESSED_MODEL_NAME",
    output_path="OUTPUT_PATH",  # ex) ./compressed_model.h5
    compression_ratio=0.5,
)

Convert Model and Benchmark the Converted Model

Convert an ONNX model into a TensorRT model, and benchmark the TensorRT model on the Jetson Nano.

from loguru import logger
from netspresso.launcher import ModelConverter, ModelBenchmarker, ModelFramework, TaskStatus, DeviceName, SoftwareVersion

converter = ModelConverter(user_session=session)

model = converter.upload_model("./examples/sample_models/test.onnx")


conversion_task = converter.convert_model(
    model=model,
    input_shape=model.input_shape,
    target_framework=ModelFramework.TENSORRT,
    target_device_name=DeviceName.JETSON_AGX_ORIN,
    target_software_version=SoftwareVersion.JETPACK_5_0_1,
    wait_until_done=True
)

logger.info(conversion_task)

CONVERTED_MODEL_PATH = "converted_model.trt"
converter.download_converted_model(conversion_task, dst=CONVERTED_MODEL_PATH)


benchmarker = ModelBenchmarker(user_session=session)
benchmark_model = benchmarker.upload_model(CONVERTED_MODEL_PATH)
benchmark_task = benchmarker.benchmark_model(
    model=benchmark_model,
    target_device_name=DeviceName.JETSON_AGX_ORIN,
    target_software_version=SoftwareVersion.JETPACK_5_0_1,
    wait_until_done=True
)
logger.info(f"model inference latency: {benchmark_task.latency} ms")
logger.info(f"model gpu memory footprint: {benchmark_task.memory_footprint_gpu} MB")
logger.info(f"model cpu memory footprint: {benchmark_task.memory_footprint_cpu} MB")

Available Options for Launcher (Convert, Benchmark)

Available Target Frameworks for Conversion with Source Models

Target / Source Model	ONNX	TENSORFLOW_KERAS	TENSORFLOW
TENSORRT	✔️
DRPAI	✔️
OPENVINO	✔️
TENSORFLOW_LITE	✔️	✔️	✔️

Available Devices for Framework

Device / Framework	ONNX	TENSORRT	TENSORFLOW_LITE	DRPAI	OPENVINO
RASPBERRY_PI_4B	✔️		✔️
RASPBERRY_PI_3B_PLUS	✔️		✔️
RASPBERRY_PI_ZERO_W	✔️		✔️
RASPBERRY_PI_ZERO_2W	✔️		✔️
RENESAS_RZ_V2L	✔️			✔️
RENESAS_RZ_V2M	✔️			✔️
RENESAS_RA8D1			✔️(only INT8)
ALIF_ENSEMBLE_E7_DEVKIT_GEN2			✔️(only INT8)
JETSON_NANO	✔️	✔️
JETSON_TX2	✔️	✔️
JETSON_XAVIER	✔️	✔️
JETSON_NX	✔️	✔️
JETSON_AGX_ORIN	✔️	✔️
AWS_T4	✔️	✔️
Intel_XEON_W_2233					✔️

Available Software Versions for Jetson Devices

Software Versions requires only Jetson Device. If you are using a different device, you do not need to enter it.

Software Version / Device	JETSON_NANO	JETSON_TX2	JETSON_XAVIER	JETSON_NX	JETSON_AGX_ORIN
JETPACK_4_4_1	✔️
JETPACK_4_6	✔️	✔️	✔️	✔️
JETPACK_5_0_1					✔️
JETPACK_5_0_2				✔️

NetsPresso Model Compressor Best Practice

If you want to experience Model Compressor online without any installation, please refer to the NetsPresso-Model-Compressor-ModelZoo repo that runs on Google Colab.

Contact

Join our Discussion Forum for providing feedback or sharing your use cases, and if you want to talk more with Nota, please contact us here.
Or you can also do it via email([e-mail]) or phone[phone])!