torchhull: A fast Visual Hull implementation

torchhull is an extremely fast Torch C++/CUDA implementation for computing visual hulls from mask images and comes with Python bindings through charonload:

• ⚡ Up to real-time capable speed depending on chosen resolution
• 🗜️ Memory-efficient computation by constructing sparse voxel octrees
• 🌊 Watertight mesh generation via Marching Cubes
• 🎈 Smooth surfaces with sparse Gaussian blur preprocessing tailored for mask images
• 🛠️ Support for partially visible objects, i.e. clipped mask images, and fully observed objects

In particular, torchhull is a GPU implementation of the following paper:

@article{scharr2017fast,
  title={{Fast High Resolution Volume Carving for 3D Plant Shoot Reconstruction}},
  author={Scharr, Hanno and Briese, Christoph and Embgenbroich, Patrick and Fischbach, Andreas and Fiorani, Fabio and M{\"u}ller-Linow, Mark},
  journal={Frontiers in Plant Science},
  volume={8},
  pages={303692},
  year={2017},
  publisher={Frontiers}
}

Installation

torchhull requires the following prerequites (for JIT compilation):

• Python >= 3.10
• CUDA >= 12.1
• C++17 compiler

The package itself can be installed from PyPI:

pip install torchhull

Quick Start

torchhull gets as input mask images with camera information:

• masks: Single-channel images M with binary values {0, 1}.
• transforms: Fused extrinsic and intrinsic matrix K * T, i.e. from world coordinates to image coordinates (right before perspective division), either in OpenGL or OpenCV convention.

The visual hull is then evaluated inside a cube with bottom-front-left corner cube_corner_bfl and extent cube_length at extracted at octree level level. The remaining flags control how the output mesh (verts, faces) should look like.

import torchhull

# Optional
masks = torchhull.gaussian_blur(masks, # [B, H, W, 1]
                                kernel_size,
                                sigma,
                                sparse=True,
                               )

verts, faces = torchhull.visual_hull(masks,  # [B, H, W, 1]
                                     transforms,  # [B, 4, 4]
                                     level,
                                     cube_corner_bfl,
                                     cube_length,
                                     masks_partial=False,
                                     transforms_convention="opengl",
                                     unique_verts=True,
                                    )

License

This software is provided under MIT license, with parts under BSD 3-Clause license. See LICENSE for more information.

Contact

Patrick Stotko - stotko@cs.uni-bonn.de