d2nt 0.1.0

D2NT: A High-Performing Depth-to-Normal Translator

D2NT: A High-Performing Depth-to-Normal Translator

This repo is the official implementation of the paper:

"D2NT: A High-Performing Depth-to-Normal Translator"

[arXiv] [homepage] [video]

Installation

Install from PyPI (Recommended)

pip install d2nt

Install from Source

# Clone the repository
git clone https://github.com/fengyi233/depth-to-normal-translator.git
cd depth-to-normal-translator

# Install in development mode
pip install -e .

# Or install normally
pip install .

Prerequisites

• Python >= 3.7
• numpy >= 1.20.0
• opencv-python >= 4.5.0
• matplotlib >= 3.5.0 (optional, for visualization)

Dataset Preparation

Public real-world datasets generally obtain surface normals by local plane fitting, which makes the surface normal ground truth unreliable. Therefore, we use the synthesis 3F2N dataset provided in this paper to evaluate estimation performance.

The 3F2N dataset can be downloaded from:
BaiduDisk
GoogleDrive
The dataset is organized as follows:

3F2N
 |-- Easy
 |  |-- android
 |  |  |-- depth
 |  |  |-- normal
 |  |  |-- params.txt
 |  |  |-- pose.txt
 |  |-- cube
 |  |-- ...
 |  |-- torusknot
 |-- Medium
 |  |-- ...
 |-- Hard
 |  |-- ...

Usage

Python Package Usage

After installation, you can use the depth2normal() function directly:

import numpy as np
from d2nt import depth2normal

# Prepare depth map (example)
depth = np.random.rand(480, 640) * 10.0

# Camera intrinsic matrix
# Format: [[fx,  0, u0],
#          [ 0, fy, v0],
#          [ 0,  0,  1]]
cam_intrinsic = np.array([
    [525.0, 0, 320.0],  # fx=525.0, u0=320.0
    [0, 525.0, 240.0],  # fy=525.0, v0=240.0
    [0, 0, 1]
])

# Convert depth to normal
normal = depth2normal(depth, cam_intrinsic, version='d2nt_v3')

print(f"Normal map shape: {normal.shape}")  # (480, 640, 3)

Algorithm Versions

• d2nt_basic: Basic version without any optimization method
• d2nt_v2: With Discontinuity-Aware Gradient (DAG) filter
• d2nt_v3: With DAG filter and MRF-based Normal Refinement (MNR) module (recommended)

Python Demo

Navigate to python/ directory and run demo.py, a result and the corresponding error map (degrees) will be displayed. We also implement 3F2N SNE in python. The matlab and c++ implementation can be found in this repository.

It is recommended to run 'demo.py' in your Python Console instead of the Terminal for the sake of visualization. You can change the parameter 'VERSION' to select the D2NT version.
'd2nt_basic' represents for the depth-to-normal translator without any optimization method.
'd2nt_v2' represents for the D2NT with Discontinuity-Aware Gradient (DAG) filter.
'd2nt_v3' represents for the D2NT with DAG filter and MRF-based Normal Refinement (MNR) module.

Cite

This code is for non-commercial use. If you find our work useful in your research, please consider citing our paper:

@inproceedings{icra_2023_d2nt,
	author      = {{Yi Feng, Bohuan Xue, Ming Liu, Qijun Chen, and Rui Fan}},
	title       = {{D2NT: A High-Performing Depth-to-Normal Translator}},
	booktitle   = {{IEEE International Conference on Robotics and Automation (ICRA)}},
	year        = {{2023}}
}