euler-loading

One PyTorch Dataset across arbitrarily many data modalities — matched by ID, not by filename luck.

Multi-modal datasets arrive as separate directory trees — RGB here, depth there, segmentation in a zip, calibration somewhere above it all. Keeping them in step usually means a pile of fragile path arithmetic.

euler-loading replaces that. Each modality is indexed by ds-crawler, and euler-loading intersects the file IDs across modalities so every sample holds exactly one file per modality. Hierarchical files such as per-scene calibration are matched by their position in the tree and shared across the samples below them.

flowchart LR
    A["rgb/"] --> X
    B["depth/"] --> X
    C["segmentation.zip"] --> X
    X(["intersect file IDs"]) --> S["sample dict<br/>rgb · depth · segmentation<br/>id · full_id · meta · attributes"]
    D["calib.json<br/><i>hierarchical</i>"] -.->|"matched by tree position"| S

It never interprets file contents. It resolves which file to load and hands the path — or an in-memory buffer, for zip-backed modalities — to a loader function, which you supply or let euler-loading resolve from the dataset's dataset-head.json contract.

Install

pip install "euler-loading[gpu]"

Requires Python 3.9+. The [gpu] extra pulls in PyTorch for the tensor loaders; without it the package still works using the CPU (NumPy) loaders.

For Synscapes EXR depth files, install euler-loading[synscapes] (NumPy) or euler-loading[gpu,synscapes] (tensors). See the Synscapes loaders for formats and usage.

Quick start

from euler_loading import Modality, MultiModalDataset
from euler_loading.loaders.gpu import vkitti2

dataset = MultiModalDataset(
    modalities={
        "rgb":   Modality("/data/vkitti2/rgb",   loader=vkitti2.rgb,   split="train"),
        "depth": Modality("/data/vkitti2/depth", loader=vkitti2.depth, split="train"),
    },
    hierarchical_modalities={
        "intrinsics": Modality("/data/vkitti2/textgt", loader=vkitti2.read_intrinsics),
    },
)

sample = dataset[0]
sample["rgb"]         # torch.Tensor (3, H, W) float32 in [0, 1]
sample["depth"]       # torch.Tensor (1, H, W) float32, metres
sample["intrinsics"]  # {file_id: (3, 3) tensor} for every calib file above this sample
sample["id"]          # leaf file ID, shared across modalities
sample["full_id"]     # full hierarchical path, e.g. "/Scene01/Camera_0/00000"

Drop it straight into a DataLoader — it is a standard torch.utils.data.Dataset:

loader = DataLoader(dataset, batch_size=16, num_workers=4, pin_memory=True)

Automatic loader resolution

Omit loader= and euler-loading resolves the loader declared by that modality's ds-crawler dataset contract:

dataset = MultiModalDataset(
    modalities={
        "rgb": Modality("/data/vkitti2/rgb", split="train"),
    },
)

The modality root must contain .ds_crawler/dataset-head.json (or the scoped equivalent) with a named euler_loading entry in its addons object. A minimal RGB contract looks like this:

{
  "contract": {
    "kind": "dataset_head",
    "version": "1.0"
  },
  "dataset": {
    "id": "vkitti2_rgb",
    "name": "Virtual KITTI 2 RGB"
  },
  "modality": {
    "key": "rgb",
    "meta": {
      "range": [0, 255]
    }
  },
  "addons": {
    "euler_loading": {
      "version": "1.0",
      "loader": "vkitti2",
      "function": "rgb"
    }
  }
}

loader selects a built-in loader module and function selects the callable inside it. Modality(path, function="sky_mask") picks another callable from the same module, e.g. boolean sky masks from a class_segmentation index, with the sky taken from the head's skyclass. Automatic resolution uses the GPU variant; pass a loader explicitly when you want the CPU variant or a custom callable. See Automatic loader resolution for the full contract and writer rules.

Checking a dataset

Before any of that runs, point the bundled command at a folder to see what euler-loading would make of it:

euler-loading /data/vkitti2

For every ds-crawler artifact set below that path it resolves the loader the dataset declares, decodes a sample of the indexed files with it, and prints the shape, dtype and value range that came back:

vkitti_2.0.3_rgb  [ok]
  path      /data/vkitti2/vkitti_2.0.3_rgb
  contract  vkitti2_rgb, "Virtual KITTI 2 RGB", modality key: rgb
  loader    vkitti2.rgb -> euler_loading.loaders.gpu.vkitti2.rgb
  index     21260 files; available splits: train, val
  decoded   1/1
    Scene01/clone/frames/rgb/Camera_0/rgb_00000.jpg  ->  Tensor (3, 375, 1242) float32 in [0, 1]  (11.4 ms)

Nothing is written, directories and .zip archives are treated alike, and the exit status is non-zero if any modality failed to resolve or decode — so it also works as a CI step. --all decodes every file instead of a sample, --cpu checks the NumPy loaders on a torch-free install, and --json prints the same report for a machine. See Dry-running loaders.

What you get

Built-in loaders

Every dataset module exists in two variants: loaders.gpu.* returns torch.Tensor in CHW layout, loaders.cpu.* returns numpy.ndarray in HWC. All of them accept both filesystem paths and in-memory buffers.

Full inventory with shapes, dtypes and units: docs/loaders.md. The machine-readable version is loaders.json.

Documentation

• Serializable resize/crop plans: Phase 1 export, explicit backend and calibration bindings, resolution, and output profiles.

Development

git clone https://github.com/d-rothen/euler-loading.git
cd euler-loading
pip install -e ".[gpu,dev]"
pytest

See CONTRIBUTING.md for the loader-authoring workflow and release process.

License

MIT © Daniel Rothenpieler

See Phase 2 materialization for the opt-in captured spatial workflow (2.24.0, unreleased).