Pandora plugin to create the cost volume with the neural network mc-cnn
Project description
Pandora plugin mccnn
MC-CNN neural network plugin for .
Overview • Install • Usage • Related • References
Overview
Pandora stereo matching framework is designed to provide some state of the art stereo algorithms and to add others one as plugins.
This Pandora plugin aims to compute the cost volume using the similarity measure produced by MC-CNN neural network [MCCNN], with the MCCNN library .
Install
pandora_plugin_mccnn is available on Pypi and can be installed by:
pip install pandora_plugin_mccnn
This command will installed required dependencies as Pandora and MCCNN.
Usage
Let's refer to Pandora's readme or online documentation for further information about Pandora general functionalities.
More specifically, you can find :
Pretrained Weights for MCCNN networks
Download weights files
Pretrained weights for mc-cnn fast and mc-cnn accurate neural networks are available in the weights directory :
- mc_cnn_fast_mb_weights.pt and mc_cnn_accurate_mb_weights.pt are the weights of the pretrained networks on the Middlebury dataset [Middlebury]
- mc_cnn_fast_data_fusion_contest.pt and mc_cnn_accurate_data_fusion_contest.pt are the weights of the pretrained networks on the Data Fusion Contest dataset [DFC]
To download the pretrained weights:
wget https://raw.githubusercontent.com/CNES/Pandora_MCCNN/master/mc_cnn/weights/mc_cnn_fast_mb_weights.pt
wget https://raw.githubusercontent.com/CNES/Pandora_MCCNN/master/mc_cnn/weights/mc_cnn_fast_data_fusion_contest.pt
wget https://raw.githubusercontent.com/CNES/Pandora_MCCNN/master/mc_cnn/weights/mc_cnn_accurate_mb_weights.pt
wget https://raw.githubusercontent.com/CNES/Pandora_MCCNN/master/mc_cnn/weights/mc_cnn_accurate_data_fusion_contest.pt
Access weights from pip package
Pretrained weights are stored into the pip package and downloaded for any installation of mc_cnn pip package.
To access it, use the weights
submodule :
from mc_cnn.weights import get_weights
mc_cnn_fast_mb_weights_path = get_weights(arch="fast", training_dataset="middlebury")
mc_cnn_fast_data_fusion_contest_path = get_weights(arch="fast", training_dataset="dfc")
mc_cnn_accurate_mb_weights_path = get_weights(arch="accurate", training_dataset="middlebury")
mc_cnn_accurate_data_fusion_contest = get_weights(arch="accurate", training_dataset="dfc")
Output example
The figures below show disparity maps produced on mountain, and desert areas generated with the Census and MCCNN similarity measures :
Left image | Left disparity map using Census measure | Left disparity map using mc-cnn fast pretrained on Middlebury | Left disparity map using mc-cnn fast pretrained DFC |
---|---|---|---|
Related
Pandora - A stereo matching framework
MCCNN - Pytorch/python implementation of mc-cnn neural network
References
Please cite the following paper when using Pandora and pandora_plugin_mccnn:
Defonte, V., Dumas, L., Cournet, M., & Sarrazin, E. (2021, July). Evaluation of MC-CNN Based Stereo Matching Pipeline for the CO3D Earth Observation Program. In 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS (pp. 7670-7673). IEEE.
Cournet, M., Sarrazin, E., Dumas, L., Michel, J., Guinet, J., Youssefi, D., Defonte, V., Fardet, Q., 2020. Ground-truth generation and disparity estimation for optical satellite imagery. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.
[MCCNN] Zbontar, J., & LeCun, Y. (2016). Stereo matching by training a convolutional neural network to compare image patches. J. Mach. Learn. Res., 17(1), 2287-2318.
[Middlebury] Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., & Westling, P. (2014, September). High-resolution stereo datasets with subpixel-accurate ground truth. In German conference on pattern recognition (pp. 31-42). Springer, Cham.
[DFC] Bosch, M., Foster, K., Christie, G., Wang, S., Hager, G. D., & Brown, M. (2019, January). Semantic stereo for incidental satellite images. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1524-1532). IEEE.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for pandora_plugin_mccnn-1.3.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 64587ed463b14dd6fc7c1e55fd1a7e2842c00cd76383ff4a5d3042d61d67d97a |
|
MD5 | 0263afd351ff71f9550029d1348a9e1c |
|
BLAKE2b-256 | 129a8360ebece44a622724e1ff804442cca8b85c363fd007991108847fd9846c |