Visin Fusion

A Python library for semantic segmentation from camera images and projected LiDAR. Five research model families share one call: model(rgb, lidar) returns class logits shaped [batch, classes, height, width].

Install

python -m pip install visin-fusion

From a source checkout instead: git clone https://github.com/visin-platform/visin-fusion.git, then python -m pip install -e . in it.

The install provides the models and the training and evaluation pipeline. Add .[visin] for Visin reporting and datasets, or .[dev] for tests. See the installation and API guide for details.

Use a model

import torch
from visin_fusion.models import CLFTv2

model = CLFTv2(num_classes=4, mode="cross_fusion", pretrained=False).eval()
rgb = torch.randn(1, 3, 256, 256)
lidar = torch.randn(1, 3, 256, 256)  # camera-aligned LiDAR projection
with torch.inference_mode():
    logits = model(rgb, lidar)  # [1, 4, 256, 256]

Each model page links its research papers. See model comparison for the fusion points and Python API for modes, training and checkpoints. The LiDAR input is a three-channel projected image, not a raw point cloud.

Optional pipeline

For ZOD, Waymo, and Iseauto datasets, follow Download and train: install, download, and run the pipeline with CLI commands.

Train, test, visualize and benchmark CLFTv2 on the included sample dataset (the getting started page also shows how to use your own data):

visin-fusion quickstart

The pipeline guide covers configs, Docker, SLURM and individual stages. Add .[visin] and a Visin pipeline key to report runs; see the Visin integration guide. To rank checkpoints on a written-down protocol, see ranking on a suite.

Tests

python -m pip install -e '.[visin,dev]'
pytest tests/unit
python tools/coverage.py  # unit + end-to-end; requires >90% package coverage

See tests/README.md for focused end-to-end commands. The full documentation is at visin-platform.github.io/visin-fusion.

License

MIT.