stereo_matching

A unified Python library for stereo depth estimation

Inference - CLI - 3D Visualization - ONNX - Quantization

stereo_matching provides a single, consistent API across 8 model families and 31 registered variant IDs. You can swap RAFT-Stereo, CREStereo, AANet, FoundationStereo, IGEV-Stereo, IGEV++, S2M2, and UniMatch without rewriting your preprocessing or postprocessing code.

It is built around the practical stereo workflow: run inference with one line, inspect models from the CLI, and turn calibrated disparity into depth maps and point clouds with the same library.

Current scope: inference, model/config loading, preprocessing, postprocessing, CLI prediction, reduced-precision inference, ONNX export and quantization, and visualization are implemented. Dataset loaders, a packaged evaluator, trainer classes, and built-in losses are not included yet. Their documentation pages describe custom integration patterns and clearly mark reserved APIs.

Installation

pip install stereo_matching

See docs/dependencies.md for runtime, development, and model-specific dependencies.

Quickstart

The pipeline API is the fastest way to run any registered stereo model:

from stereo_matching import pipeline

pipe = pipeline("stereo-matching", model="raft-stereo")
result = pipe("left.png", "right.png", focal_length=721.5, baseline=0.54)

disparity = result.disparity          # np.ndarray, float32, (H, W)
depth_map = result.depth              # np.ndarray, float32, (H, W) or None
colored   = result.colored_disparity  # np.ndarray, uint8,   (H, W, 3)

For full control over preprocessing, forward pass, and postprocessing, use Auto Classes:

from stereo_matching import AutoStereoModel, AutoProcessor
import torch

model = AutoStereoModel.from_pretrained("igev-stereo", device="cuda")
processor = AutoProcessor.from_pretrained("igev-stereo")

inputs = processor("left.png", "right.png")
with torch.no_grad():
    disparity = model(inputs["left_values"].cuda(), inputs["right_values"].cuda())
result = processor.postprocess(disparity, inputs["original_sizes"], colorize=True)

Or from the command line:

stereo-matching predict --left left.png --right right.png --model raft-stereo

Why use stereo_matching?

1. One API, every model. Switch from RAFT-Stereo to FoundationStereo or UniMatch by changing a single string. pipeline(), AutoStereoModel, and AutoProcessor keep the calling pattern consistent across families.

2. Consistent model loading. Registered variants resolve through the same pipeline(), AutoStereoModel, and AutoProcessor entry points, so model selection stays simple even as the registry grows.

3. Self-contained model packages. Each family lives under src/stereo_matching/models/<family>/ with a config file, a single vendored modeling file, and lazy self-registration in the global registry.

4. Calibrated outputs beyond disparity. Pass focal_length and baseline once and the library can return metric depth, colorized disparity, and point clouds for export or interactive viewing.

5. Deployment workflows. Cast PyTorch models to FP16/BF16, dynamically quantize linear layers to INT8, or export a two-input ONNX graph and quantize it with ONNX Runtime.

Supported Models

8 model families - 31 registered IDs - see docs/models.md for the full list and per-variant notes.

All families support pipeline(), Auto Classes, and CLI prediction.

What can you do?

# Single stereo pair
result = pipe("left.png", "right.png")

# Batch
results = pipe(
    ["left0.png", "left1.png"],
    ["right0.png", "right1.png"],
    batch_size=2,
)

# CLI prediction
stereo-matching predict --left left.png --right right.png --model raft-stereo --output-dir results/

# Run the demo after selecting variants in its MODELS list
python examples/demo.py

from stereo_matching import AutoStereoModel, export_onnx, quantize_onnx

model = AutoStereoModel.from_pretrained("raft-stereo", device="cuda")
export_onnx(model, "raft_stereo.fp32.onnx", input_height=384, input_width=640)
quantize_onnx("raft_stereo.fp32.onnx", "raft_stereo.int8.onnx")

fp16_model = model.quantize("fp16")  # direct reduced-precision PyTorch inference

from stereo_matching import AutoStereoModel, AutoProcessor

model = AutoStereoModel.from_pretrained("foundation-stereo", device="cuda")
processor = AutoProcessor.from_pretrained("foundation-stereo")

from stereo_matching import pipeline, viz
import numpy as np
from PIL import Image

pipe = pipeline("stereo-matching", model="raft-stereo")
result = pipe("left.png", "right.png", focal_length=721.5, baseline=0.54)
left_rgb = np.array(Image.open("left.png").convert("RGB"))

viz.point_cloud(
    result,
    image=left_rgb,
    focal_length=721.5,
    baseline=0.54,
    save_ply="scene.ply",
    save_glb="scene.glb",
)

pip install gradio gradio_sync3dcompare
python examples/compare_demo.py

Documentation

• docs/models.md - families, variants, and checkpoint sources
• docs/pipeline.md - pipeline(), StereoOutput, and processing details
• docs/export.md - two-input ONNX export and runtime inference
• docs/quantization.md - FP16, BF16, PyTorch INT8, and ONNX quantization
• docs/cli.md - implemented commands and the reserved evaluation interface
• docs/dependencies.md - runtime, development, and model-specific requirements
• docs/data.md - custom dataset integration; no bundled loaders yet
• docs/training.md - manual PyTorch training and current limitations
• docs/evaluation.md - metrics and a custom evaluation loop
• docs/adding_a_model.md - registry and package structure
• docs/release_notes.md - released and unreleased changes

Development checks

pip install -e ".[dev]"
ruff check .
pytest tests -m "not slow"
python -m build
twine check dist/*

Mypy is currently advisory. Real pretrained-model inference runs in the weekly slow workflow; see CONTRIBUTING.md.

Adding a New Model

1. Create src/stereo_matching/models/your_model/
2. Add configuration_your_model.py
3. Add modeling_your_model.py
4. Add __init__.py with MODEL_REGISTRY.register(...)
5. Import the package in src/stereo_matching/__init__.py

AutoStereoModel, AutoProcessor, and pipeline() resolve the new model automatically. See docs/adding_a_model.md for the full pattern.

Acknowledgments

This library builds on the work of 8 stereo matching research families. See docs/models.md#citations for the citation block.

License

MIT