Skip to main content

No project description provided

Project description

RoMa 🏛️:
Robust Dense Feature Matching
⭐CVPR 2024⭐

Johan Edstedt · Qiyu Sun · Georg Bökman · Mårten Wadenbäck · Michael Felsberg

Paper | Project Page


example
RoMa is the robust dense feature matcher capable of estimating pixel-dense warps and reliable certainties for almost any image pair.

Setup/Install

In your python environment (tested on Linux python 3.10), run:

pip install -e .

Demo / How to Use

We provide two demos in the demos folder. Here's the gist of it:

from romatch import roma_outdoor
roma_model = roma_outdoor(device=device)
# Match
warp, certainty = roma_model.match(imA_path, imB_path, device=device)
# Sample matches for estimation
matches, certainty = roma_model.sample(warp, certainty)
# Convert to pixel coordinates (RoMa produces matches in [-1,1]x[-1,1])
kptsA, kptsB = roma_model.to_pixel_coordinates(matches, H_A, W_A, H_B, W_B)
# Find a fundamental matrix (or anything else of interest)
F, mask = cv2.findFundamentalMat(
    kptsA.cpu().numpy(), kptsB.cpu().numpy(), ransacReprojThreshold=0.2, method=cv2.USAC_MAGSAC, confidence=0.999999, maxIters=10000
)

New: You can also match arbitrary keypoints with RoMa. See match_keypoints in RegressionMatcher.

Settings

Resolution

By default RoMa uses an initial resolution of (560,560) which is then upsampled to (864,864). You can change this at construction (see roma_outdoor kwargs). You can also change this later, by changing the roma_model.w_resized, roma_model.h_resized, and roma_model.upsample_res.

Sampling

roma_model.sample_thresh controls the thresholding used when sampling matches for estimation. In certain cases a lower or higher threshold may improve results.

Reproducing Results

The experiments in the paper are provided in the experiments folder.

Training

  1. First follow the instructions provided here: https://github.com/Parskatt/DKM for downloading and preprocessing datasets.
  2. Run the relevant experiment, e.g.,
torchrun --nproc_per_node=4 --nnodes=1 --rdzv_backend=c10d experiments/roma_outdoor.py

Testing

python experiments/roma_outdoor.py --only_test --benchmark mega-1500

License

All our code except DINOv2 is MIT license. DINOv2 has an Apache 2 license DINOv2.

Acknowledgement

Our codebase builds on the code in DKM.

Tiny RoMa

If you find that RoMa is too heavy, you might want to try Tiny RoMa which is built on top of XFeat.

from romatch import tiny_roma_v1_outdoor
tiny_roma_model = tiny_roma_v1_outdoor(device=device)

Mega1500:

AUC@5 AUC@10 AUC@20
XFeat 46.4 58.9 69.2
XFeat* 51.9 67.2 78.9
Tiny RoMa v1 56.4 69.5 79.5
RoMa - - -

Mega-8-Scenes (See DKM):

AUC@5 AUC@10 AUC@20
XFeat - - -
XFeat* 50.1 64.4 75.2
Tiny RoMa v1 57.7 70.5 79.6
RoMa - - -

IMC22 :'):

mAA@10
XFeat 42.1
XFeat* -
Tiny RoMa v1 42.2
RoMa -

BibTeX

If you find our models useful, please consider citing our paper!

@article{edstedt2024roma,
title={{RoMa: Robust Dense Feature Matching}},
author={Edstedt, Johan and Sun, Qiyu and Bökman, Georg and Wadenbäck, Mårten and Felsberg, Michael},
journal={IEEE Conference on Computer Vision and Pattern Recognition},
year={2024}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

romatch_roicat-0.1.0.tar.gz (49.5 kB view details)

Uploaded Source

Built Distribution

romatch_roicat-0.1.0-py3-none-any.whl (64.6 kB view details)

Uploaded Python 3

File details

Details for the file romatch_roicat-0.1.0.tar.gz.

File metadata

  • Download URL: romatch_roicat-0.1.0.tar.gz
  • Upload date:
  • Size: 49.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for romatch_roicat-0.1.0.tar.gz
Algorithm Hash digest
SHA256 73da00f77cbef89846e1feaedbe766ddb0902df48d9522ea2435e97ddae52dd6
MD5 2725cf5f3b12b2c1b1c3e87d9997925c
BLAKE2b-256 3ef58e6aec515e10087f388f7d44da0b6097fc5b5078fbdb76e808b1f0f015fa

See more details on using hashes here.

File details

Details for the file romatch_roicat-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for romatch_roicat-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 19547d90bf0d14807a5d16ed0edbbb322d1725f999a9e0ffad6fac03952822b2
MD5 8f2f90ec1aa9cef8aa6c685ed3bb1f6e
BLAKE2b-256 88989985c38615b25534c93590ece73a1998552fe8f7f36b7d9a85d0d20c8fd8

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page