Running Gen AI models and applications on NVIDIA Jetson devices with one-line command
Project description
jetson-examples
This repository provides examples for running AI models and applications on NVIDIA Jetson devices. For generative AI, it supports a variety of examples including text generation, image generation, vision transformers, vector databases, and audio models. To run the examples, you need to install the jetson-examples package and use the Seeed Studio reComputer, the edge AI device powered by Jetson Orin. The repo aims to make it easy to deploy state-of-the-art AI models, with just one line of command, on Jetson devices for tasks like language understanding, computer vision, and multimodal processing.
This repo builds upon the work of the Jetson Containers, which provides a modular container build system for various AI/ML packages on NVIDIA Jetson devices. It also leverages resources and tutorials from the Jetson Generative AI Lab, which showcases bringing generative AI to the edge, powered by Jetson hardware.
Install
pip install jetson-examples
- more installation methods
- If you have already installed, you can use
pip install jetson-examples --upgrade
to update.
Quickstart
To run and chat with LLaVA:
reComputer run llava
Example list
reComputer supports a list of examples from jetson-ai-lab
Here are some examples that can be run:
Example | Type | Model/Data Size | Image Size | Command |
---|---|---|---|---|
🆕 llama3 | Text (LLM) | 4.9GB | 10.5GB | reComputer run llama3 |
🆕 ollama | Inference Server | * | 10.5GB | reComputer run ollama |
LLaVA | Text + Vision (VLM) | 13GB | 14.4GB | reComputer run llava |
Live LLaVA | Text + Vision (VLM) | 13GB | 20.3GB | reComputer run live-llava |
stable-diffusion-webui | Image Generation | 3.97G | 7.3GB | reComputer run stable-diffusion-webui |
nanoowl | Vision Transformers(ViT) | 613MB | 15.1GB | reComputer run nanoowl |
nanodb | Vector Database | 76GB | 7.0GB | reComputer run nanodb |
whisper | Audio | 1.5GB | 6.0GB | reComputer run whisper |
yolov8-counter | Computer Vision | 6M | 13.8GB | reComputer run yolov8_counter |
Note: You should have enough space to run example, like
LLaVA
, at least27.4GB
totally
More Examples can be found examples.md
Want to add a Example by yourself? Check this develop.md
TODO List
- check disk space enough or not before run
- allow to setting some configs, such as
BASE_PATH
- detect host environment and install what we need
- support jetson-containers update
- all type jetson support checking list
- better table to show example's difference
- try jetpack 6.0
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for jetson_examples-0.0.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7da236a358fe0b8109336cf4a89cfbfd597284faca8cc5356d5fbbc9d50fef8b |
|
MD5 | 797dbcc80ddd16024edb7ef454364a94 |
|
BLAKE2b-256 | e8f6a173f163d001776cd66b20fc838d85ed75d9f2549b570ab3ff7d5f4898df |