jetson-examples 0.0.8

Running Gen AI models and applications on NVIDIA Jetson devices with one-line command

jetson-examples

This repository provides examples for running AI models and applications on NVIDIA Jetson devices. For generative AI, it supports a variety of examples including text generation, image generation, vision transformers, vector databases, and audio models. To run the examples, you need to install the jetson-examples package and use the Seeed Studio reComputer, the edge AI device powered by Jetson Orin. The repo aims to make it easy to deploy state-of-the-art AI models, with just one line of command, on Jetson devices for tasks like language understanding, computer vision, and multimodal processing.

This repo builds upon the work of the Jetson Containers, which provides a modular container build system for various AI/ML packages on NVIDIA Jetson devices. It also leverages resources and tutorials from the Jetson Generative AI Lab, which showcases bringing generative AI to the edge, powered by Jetson hardware.

Install

pip install jetson-examples

• more installation methods
• If you have already installed, you can use pip install jetson-examples --upgrade to update.

Quickstart

To run and chat with LLaVA:

reComputer run llava

Example list

reComputer supports a list of examples from jetson-ai-lab

Here are some examples that can be run:

Example	Type	Model/Data Size	Image Size	Command
🆕 llama3	Text (LLM)	4.9GB	10.5GB	reComputer run llama3
🆕 ollama	Inference Server	*	10.5GB	reComputer run ollama
LLaVA	Text + Vision (VLM)	13GB	14.4GB	reComputer run llava
Live LLaVA	Text + Vision (VLM)	13GB	20.3GB	reComputer run live-llava
stable-diffusion-webui	Image Generation	3.97G	7.3GB	reComputer run stable-diffusion-webui
nanoowl	Vision Transformers(ViT)	613MB	15.1GB	reComputer run nanoowl
nanodb	Vector Database	76GB	7.0GB	reComputer run nanodb
whisper	Audio	1.5GB	6.0GB	reComputer run whisper
yolov8-counter	Computer Vision	6M	13.8GB	reComputer run yolov8_counter

Note: You should have enough space to run example, like LLaVA, at least 27.4GB totally

More Examples can be found examples.md

Want to add a Example by yourself? Check this develop.md

TODO List

• check disk space enough or not before run
• allow to setting some configs, such as BASE_PATH
• detect host environment and install what we need
• support jetson-containers update
• all type jetson support checking list
• better table to show example's difference
• try jetpack 6.0