LuxonisML

🌟 Overview

LuxonisML is the core library of the Luxonis MLOps stack. It defines the Luxonis Data Format (LDF), and it provides the dataset, loader, parser, tracking, and utility layers that the other Luxonis tools build on. LuxonisTrain, ModelConverter, and LuxonisEval all depend on it.

✨ Key Features

• One Dataset Format: Build a computer vision dataset once, then train on it with any of the Luxonis tools.
• Bring Your Own Data: Convert COCO, YOLO, VOC, RoboFlow, and other common formats with a single call.
• Storage Agnostic: Keep the images locally, in Google Cloud Storage, or in S3, without a change to your training code.
• Rich Annotations: Bounding boxes, keypoints, semantic and instance segmentation, classification, arrays, and free-form metadata.
• One Tracking API: Log metrics to TensorBoard, Weights & Biases, or MLflow through one interface.

[!WARNING] The project is in a beta state and might be unstable or contain bugs - please report any feedback.

🚀 Quick Start

1. Install LuxonisML

   pip install luxonis-ml[data]
   

   This will create the luxonis_ml executable in your PATH.

2. Convert a dataset into LDF

   We will use a sample COCO dataset from RoboFlow in this example.

   luxonis_ml data parse "roboflow://team-roboflow/coco-128/2/coco" --name coco_test
   

   [!IMPORTANT] A roboflow:// source needs the ROBOFLOW_API_KEY environment variable. Get your key from the Roboflow settings. The Roboflow documentation gives the steps.

3. Inspect the result

   luxonis_ml data info coco_test
   luxonis_ml data inspect coco_test
   

4. Load it in your training code

   from luxonis_ml.data import LuxonisDataset, LuxonisLoader
   
   loader = LuxonisLoader(LuxonisDataset("coco_test"), view="train")
   
   for sample in loader:
       images = sample.images
       labels = sample.labels
   

[!NOTE] For hands-on examples of how to prepare data with LuxonisML and train AI models using LuxonisTrain, check out this guide.

📜 Table Of Contents

• 🌟 Overview
  • ✨ Key Features
• 🚀 Quick Start
• 🧩 Modules
• 🛠️ Installation
• 📝 Usage
  • 💾 Creating a Dataset
  • 🔄 Converting an Existing Dataset
  • 📤 Loading the Data
  • 📈 Tracking an Experiment
• 💻 CLI
• 🔑 Credentials
• 📚 Documentation
• 🤝 Contributing

🧩 Modules

Each module links to its own API reference:

🛠️ Installation

LuxonisML requires Python 3.10 or higher. We recommend using a virtual environment to manage dependencies.

Install via pip:

pip install luxonis-ml[data]

Each module has its own extra, so you install only what you use:

The data, ldf, tracker, and utils modules fail on import when you do not install their extra. The message names the extra. luxonis_ml.telemetry is an exception: it imports without posthog and falls back to a no-op backend.

☁️ Additional Dependencies

These extras add support for specific cloud services and integrations:

[!NOTE] LuxonisML installs these four dependencies for you on first use. If you open a gs://, gcs://, s3://, mlflow://, or roboflow:// path and the package is absent, LuxonisML installs it and continues. Install the extra yourself when you want a reproducible environment or an offline machine.

Examples:

# the data module, with Google Cloud Storage and Roboflow support
pip install luxonis-ml[data,gcs,roboflow]

# everything
pip install luxonis-ml[all]

For a development environment, read CONTRIBUTING.md. The development tooling lives in uv dependency groups, not in a published extra.

📝 Usage

💾 Creating a Dataset

A dataset is a named collection of records. Each record points to one image and carries one annotation. The coordinates are relative to the image size.

from luxonis_ml.data import LuxonisDataset

dataset = LuxonisDataset("parking_lot")


def records():
    yield {
        "file": "images/frame_001.jpg",
        "task_name": "detection",
        "annotation": {
            "class": "car",
            "boundingbox": {
                "x": 0.1,
                "y": 0.2,
                "w": 0.3,
                "h": 0.4,
            },
        },
    }


dataset.add(records())
dataset.make_splits({"train": 0.8, "val": 0.1, "test": 0.1})

LDF also supports keypoints, semantic and instance segmentation, classification, arrays, and free-form metadata. See luxonis_ml.data.datasets for the dataset contract, and the luxonis_ml.ldf module for every annotation schema.

🔄 Converting an Existing Dataset

LuxonisParser reads the common dataset formats. It detects the format from the directory structure when you do not name one.

from luxonis_ml.data import LuxonisParser

dataset = LuxonisParser(
    "path/to/coco_dataset",
    dataset_name="coco",
).parse()

The dataset path can be one of the following:

• a local directory, or a ZIP archive
• s3://bucket/path/to/directory for AWS S3
• gs://bucket/path/to/directory or gcs://bucket/path/to/directory for Google Cloud Storage
• roboflow://{workspace}/{project}/{version}/{format} for RoboFlow
• ultralytics://{username}/datasets/{slug} for Ultralytics

See luxonis_ml.data.parsers for the supported formats and their expected layouts.

📤 Loading the Data

LuxonisLoader reads one or more splits. It resizes the images, applies the augmentations, and returns the labels for each task.

from luxonis_ml.data import LuxonisLoader

loader = LuxonisLoader(dataset, view="train", height=640, width=640)

for sample in loader:
    images = sample.images
    labels = sample.labels

Labels use "task_name/task_type" keys, such as "detection/boundingbox". See luxonis_ml.data.loaders for the output shapes, and luxonis_ml.data.augmentations for the augmentation configuration.

📈 Tracking an Experiment

from luxonis_ml.tracker import LuxonisTracker

tracker = LuxonisTracker(
    project_name="parking_lot",
    run_name="baseline",
    is_tensorboard=True,
)

tracker.log_metric("loss", 0.42, step=1)
tracker.close()

[!NOTE] The tracker extra does not install every dependency of luxonis_ml.tracker. The module imports mlflow and cv2 at import time, so install mlflow and opencv-python as well: pip install "luxonis-ml[tracker,mlflow]" opencv-python. Install torch for TensorBoard and wandb for Weights & Biases.

💻 CLI

The package installs the luxonis_ml executable.

Available commands:

• data - Parse, inspect, export, merge, push, pull, and delete datasets
• archive - Inspect and extract NN Archive files
• fs - Copy and list files across the supported storage backends
• checkhealth - Report whether the ldf, data, utils, and nn_archive modules import correctly

To get help on any command:

luxonis_ml <command> --help

Examples:

luxonis_ml data parse ./coco_dataset --name coco
luxonis_ml data ls
luxonis_ml data health coco
luxonis_ml data export coco --type ultralyticsndjson

🔑 Credentials

When using cloud services, avoid hard-coding credentials or placing them directly in your configuration files. Instead:

• Use environment variables to store sensitive information.
• Use a .env file and load it securely, ensuring it's excluded from version control.

Supported Cloud Services:

• AWS S3, requires:
  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
  • AWS_S3_ENDPOINT_URL
• Google Cloud Storage, requires:
  • GOOGLE_APPLICATION_CREDENTIALS
• RoboFlow, requires:
  • ROBOFLOW_API_KEY
• Ultralytics, requires:
  • ULTRALYTICS_API_KEY

For logging and tracking, we support:

• MLFlow, requires:
  • MLFLOW_S3_BUCKET
  • MLFLOW_S3_ENDPOINT_URL
  • MLFLOW_TRACKING_URI

Dataset storage is configured with:

• LUXONISML_BASE_PATH - local base path for datasets and cache files, ~/luxonis_ml by default
• LUXONISML_TEAM_ID - team identifier used by dataset storage, offline by default
• LUXONISML_BUCKET - the cloud bucket that holds remote datasets

[!NOTE] LuxonisML sends no telemetry. It only provides a telemetry client that other Luxonis packages can use. To turn that client off, set LUXONIS_TELEMETRY_ENABLED=false. See the luxonis_ml.telemetry documentation for the data an event carries.

📚 Documentation

The API documentation is generated from the docstrings in the source code, and it is published on the Luxonis documentation portal:

https://docs.luxonis.com/software-v3/ai-inference/model-source/training/luxonis-ml/luxonis-ml-api-reference/

To build the documentation locally, run:

uv run pydoctor luxonis_ml

The command writes the site to apidocs/index.html.

🤝 Contributing

We welcome contributions! Please read our Contribution Guide to get started. Whether it's reporting bugs, improving documentation, or adding new features, your help is appreciated.

📄 License

This project is licensed under the Apache 2.0 License.