pysuperansac 1.0

A RANSAC implementation for robust estimation.

SupeRANSAC: One RANSAC to Rule them All (Arxiv Paper)

About

SupeRANSAC is a Python library that provides bindings for an advanced RANSAC C++ implementation using pybind11. It supports a wide variety of sampling, scoring, local optimization, and inlier selection techniques for robust model estimation tasks. It provides estimators for homography, essential, fundamental matrix, rigid and absolute pose estimation.

Installation

Clone the repository and its submodules:

git clone [e-mail]:danini/superansac.git

Make sure that you have the necessary OpenCV libraries installed:

sudo apt-get update
sudo apt-get install libopencv-dev libopencv-contrib-dev libarpack++2-dev libarpack2-dev libsuperlu-dev cmake build-essential libboost-all-dev libeigen3-dev

Install SupeRANSAC by running

pip install .

Results

Below, the AUC@10° scores are shown averaged over the LaMAR, PhotoTourism, ETH3D, Kitti, 7Scenes, and ScanNet datasets. We show results both with SuperPoint + LightGlue (top row) and RoMA (bottom) features for essential matrix (left), fundamental matrix (middle) and homography (right) estimation.

Essential matrix	Fundamental matrix	Homography

Below, absolute pose (from 2D-3D correspondences) and rigid transformation (from 3D-3D matches) estimation results are shown. For absolute pose estimation, we integrated SupeRANSAC into the hloc library. For rigid transformation estimation, we employ GeoTransformer features.

Absolute Pose with hloc	Rigid transform with GeoTransformer

Jupyter Notebook Examples

• Homography Fitting with SuperPoint + LightGlue Matches
  Example available at: notebook

• Homography Fitting with RoMA Matches
  Example available at: notebook

• Fundamental Matrix Fitting with SuperPoint + LightGlue Matches
  Example available at: notebook

• Fundamental Matrix Fitting with RoMA Matches
  Example available at: notebook

• Essential Matrix Fitting with SuperPoint + LightGlue Matches
  Example available at: notebook

• Essential Matrix Fitting with RoMA Matches
  Example available at: notebook

• Absolute Pose Estimation
  Example available at: notebook

• 6D Rigid Transformation Estimation
  Example available at: notebook

Usage

The library provides several model estimation functions and settings that allow customization of the RANSAC pipeline. Below are the primary features and their usage.

Importing the Library

import pysuperansac
from pysuperansac import ScoringType, SamplerType, LocalOptimizationType, InlierSelectorType, NeighborhoodType, CameraType, RANSACSettings

Example: Estimate a Homography Matrix

# Example correspondences and image sizes
correspondences = [
    ([x1, y1], [x2, y2]),
    ([x3, y3], [x4, y4]),
    # Add more correspondences...
]
image_sizes = [width1, height1, width2, height2]

# Configure RANSAC settings
config = RANSACSettings()
config.min_iterations = 100
config.max_iterations = 1000
config.inlier_threshold = 3.0
config.confidence = 0.99
config.scoring = ScoringType.MAGSAC
config.sampler = SamplerType.PROSAC
config.local_optimization = LocalOptimizationType.GCRANSAC
config.final_optimization = LocalOptimizationType.LSQ

# Estimate homography
result = pysuperansac.estimateHomography(correspondences, image_sizes, config=config)
print("Estimated Homography Matrix:", result) 

Supported Estimation Functions

1. Homography Estimation

• Function: pysuperansac.estimateHomography
• Description: Estimates a homography matrix from 2D-2D point correspondences.
• Parameters:
  • correspondences: A list of paired 2D points.
  • image_sizes: A tuple of source and target image sizes.
  • config: An instance of RANSACSettings.
  • probabilities (optional): Correspondence probabilities.

2. Fundamental Matrix Estimation

• Function: pysuperansac.estimateFundamentalMatrix
• Description: Estimates a fundamental matrix from 2D-2D point correspondences.
• Parameters:
  • correspondences: A list of paired 2D points.
  • image_sizes: A tuple of source and target image sizes.
  • config: An instance of RANSACSettings.
  • probabilities (optional): Correspondence probabilities.

3. Essential Matrix Estimation

• Function: pysuperansac.estimateEssentialMatrix
• Description: Estimates an essential matrix using 2D-2D point correspondences and intrinsic matrices.
• Parameters:
  • correspondences: A list of paired 2D points.
  • image_sizes: A tuple of source and target image sizes.
  • config: An instance of RANSACSettings.
  • intrinsics_src: Source camera intrinsic matrix.
  • intrinsics_dst: Destination camera intrinsic matrix.
  • probabilities (optional): Correspondence probabilities.

4. Rigid Transformation Estimation

• Function: pysuperansac.estimateRigidTransform
• Description: Estimates a 6D rigid transformation matrix from 3D-3D point correspondences.
• Parameters:
  • correspondences: A list of paired 3D points.
  • bounding_box_sizes: Size of the bounding box.
  • config: An instance of RANSACSettings.
  • probabilities (optional): Correspondence probabilities.

5. Absolute Pose Estimation

• Function: pysuperansac.estimateAbsolutePose
• Description: Estimates the absolute pose of a camera using 2D-3D correspondences.
• Parameters:
  • correspondences: A list of paired 2D-3D points.
  • camera_type: Type of the camera (e.g., CameraType.SimpleRadial).
  • camera_params: Camera parameters.
  • config: An instance of RANSACSettings.
  • probabilities (optional): Correspondence probabilities.

Advanced Configuration: RANSACSettings

The RANSACSettings class provides fine-grained control over the RANSAC pipeline. Customize it as needed:

config = RANSACSettings()
config.min_iterations = 100
config.max_iterations = 1000
config.inlier_threshold = 3.0
config.confidence = 0.99
config.scoring = ScoringType.MAGSAC
config.sampler = SamplerType.PROSAC
config.local_optimization = LocalOptimizationType.LSQ
config.neighborhood = NeighborhoodType.Grid

Enumerations and Their Values

The library supports several enumeration types to customize sampling, scoring, and other pipeline components:

Scoring Types

• ScoringType.RANSAC
• ScoringType.MSAC
• ScoringType.MAGSAC
• ScoringType.ACRANSAC

Sampler Types

• SamplerType.Uniform
• SamplerType.PROSAC
• SamplerType.NAPSAC
• SamplerType.ProgressiveNAPSAC
• SamplerType.ImportanceSampler
• SamplerType.ARSampler

Local Optimization Types

• LocalOptimizationType.Nothing
• LocalOptimizationType.LSQ
• LocalOptimizationType.IteratedLSQ
• LocalOptimizationType.NestedRANSAC
• LocalOptimizationType.GCRANSAC

Neighborhood Types

• NeighborhoodType.Grid
• NeighborhoodType.BruteForce

Camera Types

• CameraType.SimpleRadial
• CameraType.SimplePinhole

Evaluation - Python

The testing scripts for SupeRANSAC and the baselines are located in folder tests/. The results for essential, fundamental, and homography matrix estimation are obtained by running

python tests/X/tester-X-superansac.py

where X is essential/fundamental/homography matrix. Note that this process may take a while. However, before running, you need to install the feature detectors and set up the datasets as described below.

Install feature detectors

To use SuperPoint + LightGlue features, install LightGlue as follows:

git clone https://github.com/cvg/LightGlue.git && cd LightGlue
python -m pip install -e .

To use RoMA features, install it as follows:

git clone https://github.com/Parskatt/RoMa && cd RoMa
pip install -e .

Evaluation on the PhotoTourism dataset

Download the data from the CVPR tutorial "RANSAC in 2020":

wget http://cmp.felk.cvut.cz/~mishkdmy/CVPR-RANSAC-Tutorial-2020/RANSAC-Tutorial-Data-EF.tar
tar -xf  RANSAC-Tutorial-Data-EF.tar

Then run the notebook examples/relative_pose_evaluation_phototourism.ipynb.

Evaluation on the ScanNet dataset

Download the data from the test set for relative pose estimation used in SuperGlue (~250Mb for 1500 image pairs only):

wget https://www.polybox.ethz.ch/index.php/s/lAZyxm62WUh27Zl/download
unzip ScanNet_test.zip -d <path to extract the ScanNet test set>

Then run the notebook examples/relative_pose_evaluation_scannet.ipynb.

Evaluation on the 7Scenes dataset

Download the 7Scenes dataset and put it where it suits you. You can also only download one scene and later specify this scene in the dataloader constructor.

Then run the script runners/run_7scenes.py.

Evaluation on the ETH3D dataset

Download the ETH3D dataset (training split of the high-res multi-view, undistorted images + GT extrinsics & intrinsics should be enough) and put it where it suits you. The input argument 'downsize_factor' can be used to downscale the images, because they can be quite large otherwise.

Then run the script runners/run_eth3d.py.

Evaluation on the LaMAR dataset

Download the CAB scene of the LaMAR dataset, and unzip it to your favourite location. Note that we only use the images in CAB/sessions/query_val_hololens.

Evaluation on the KITTI dataset

Download the KITTI odometry dataset (grayscale images and poses), and unzip them to your favourite location.