Fourier-Mellin Python Library using OpenCV

This repository implements the Fourier-Mellin transform for image registration and video stabilization using semilog polar coordinates with OpenCV. The implementation is written in C++, but python bindings are provided. For information about the pipeline, see this article. Tested on Ubuntu 20.04.

Disclaimer

This repository is an unstable release and not intended for production. Many other repositories have implemented the Fourier-Mellin transform, for example imreg_fmt by sthoduka or fourier-mellin by polakluk.

Note that the image registration only works effectively for Eucledian/similar transformations without affine or perspective distortions.

Video Stabilization

https://github.com/brht0/fourier-mellin/assets/90235713/c5524042-6bc6-46b5-837e-adb62ddcf9b1

Original video can be found on Youtube (No affiliation)

Image Registration

Image registration with a transformed image, overlayed.

Installation with git clone

It is recommended to use a python virtual environment. The repository will be added to PyPI in time.

# inside your own project
python3 -m venv .venv
source .venv/bin/activate
pip install git+https://github.com/brht0/fourier-mellin

Examples

Many examples, such as video stabilization, are included inside the examples/ subdirectory. The following example registers two images.

import cv2
import numpy as np
import fourier_mellin

reference = cv2.imread('lenna.png')
transformed = cv2.imread('lenna_transformed.png')

rows, cols = reference.shape[:2]
fm = fourier_mellin.FourierMellin(cols, rows)

transformed_reference, transform = fm.register_image(reference, transformed)
overlay = cv2.addWeighted(transformed, 0.5, transformed_reference, 0.5, 0.0, dtype=cv2.CV_32F)

cv2.imwrite("overlay.jpg", overlay)
cv2.imwrite("transformed.jpg", transformed)

Building without pip

Building without pip is not required for use with python. Building without pip requires installing additional dependencies, such as pybind11. This step may be skipped, in case only python bindings are used.

cmake -B build/release -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build/release -j 8

Todo

• Register images directly from files
• Benchmarking
• Optimization
• CUDA with OpenCV
• remove filters, as cv::phaseCorrelate already applies Hanning Window
• Documentation
• Proper threading support
• angles beyond -90 to 90 deg
• updated readme with up to date information
• pure python version that for windows/macos
• memory problem when passing in reference to some functions, where image is replaced (clearer documentation?)
• link time optimizer flto