PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction (Python Package Version)

This repository contains the refactored Python package for PGSR. It is ported from commit de24f1a38b350387e8d8fe381b2cd70c1ae946e7. The original components have been reorganized into a standard Python package and adapted to the reusable APIs provided by gaussian-splatting.

Features

• Code organized as a standard Python package
• gsplat and gsplat-2dgs rendering backends
• Planar scale regularization
• Depth-normal consistency
• Multi-view photometric and geometric regularization
• Virtual-camera reprojection
• Multi-view trimming and opacity-reset densification
• Optional camera-pose optimization
• Rendering, mesh extraction, and interactive viewing

Prerequisites

• PyTorch with CUDA support
• A CUDA Toolkit version compatible with the installed PyTorch build
• Python 3.10 or later

Optional features can be installed through package extras:

pip install --upgrade "pgsr[mesh,viewer]"

If you have trouble installing gaussian-splatting, install it from source:

pip install wheel setuptools
pip install --upgrade git+https://github.com/yindaheng98/gaussian-splatting.git@master --no-build-isolation

PyPI Install

pip install --upgrade pgsr

Or install the latest version from source:

pip install wheel setuptools
pip install --upgrade git+https://github.com/yindaheng98/PGSR.git@main --no-build-isolation

Development Install

git clone https://github.com/yindaheng98/PGSR.git
cd PGSR
pip install --editable .

Quick Start

1. Prepare a dataset in the COLMAP format used by gaussian-splatting. For example, download the Tanks and Temples + Deep Blending dataset:

wget https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/datasets/input/tandt_db.zip -P ./data
unzip data/tandt_db.zip -d data/

2. Train PGSR with densification:

python -m pgsr.train -s data/truck -d output/truck -i 30000 --mode densify --backend gsplat --no_image_mask --no_depth_data

3. Render the trained model:

python -m pgsr.render -s data/truck -d output/truck -i 30000 --backend gsplat --no_image_mask

4. Extract a mesh (requires the mesh extra):

python -m pgsr.mesh -s data/truck -d output/truck -i 30000 --backend gsplat --no_image_mask -o max_depth=10.0 -o voxel_size=0.01

5. Open the interactive viewer (requires the viewer extra):

python -m pgsr.viewer -d output/truck -i 30000 --backend gsplat --port 8080

💡 This package does not include dataset preprocessing or evaluation scripts. Refer to the original PGSR repository for the DTU, Tanks and Temples, and Mip-NeRF 360 workflows.

💡 See .vscode/launch.json for more examples. Run python -m pgsr.train --help, python -m pgsr.render --help, python -m pgsr.mesh --help, or python -m pgsr.viewer --help for all command-line options.

Backends and Training Modes

Two rendering backends are available:

• gsplat (default)
• gsplat-2dgs

The training entry point supports the following modes:

• base: PGSR regularization without densification
• densify (default): PGSR regularization with multi-view trimming, opacity reset, and densification
• camera: base with trainable camera poses
• camera-densify: densify with trainable camera poses

Use repeated -o key=value arguments to override trainer configuration values:

python -m pgsr.train -s data/truck -d output/truck -o densify_grad_threshold=0.0001 -o opacity_cull_threshold=0.05

API Usage

This project builds on gaussian-splatting and provides PGSR Gaussian models and composed trainers. Refer to that package for the core Gaussian model, dataset, trainer, and training-loop concepts.

The high-level factory prepares the dataset, Gaussian model, and trainer:

from pgsr.train import prepare_training

dataset, gaussians, trainer = prepare_training(
    sh_degree=3,
    source="data/truck",
    device="cuda",
    mode="densify",
    backend="gsplat",
    load_mask=False,
    load_depth=False,
    configs={"densify_grad_threshold": 0.0001},
)

The lower-level factories can also be used independently:

from gaussian_splatting.prepare import prepare_dataset
from pgsr.prepare import prepare_gaussians, prepare_trainer

dataset = prepare_dataset(
    source="data/truck",
    device="cuda",
    load_mask=False,
    load_depth=False,
)
gaussians = prepare_gaussians(
    sh_degree=3,
    source="data/truck",
    device="cuda",
    backend="gsplat",
)
trainer = prepare_trainer(
    gaussians=gaussians,
    dataset=dataset,
    mode="densify",
    configs={"densify_grad_threshold": 0.0001},
)

PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction

Danpeng Chen, Hai Li, Weicai Ye, Yifan Wang, Weijian Xie, Shangjin Zhai, Nan Wang, Haomin Liu, Hujun Bao, Guofeng Zhang

Project Page | arXiv

We present a Planar-based Gaussian Splatting Reconstruction representation for efficient and high-fidelity surface reconstruction from multi-view RGB images without any geometric prior (depth or normal from pre-trained model).

Updates

• [2024.07.18]: We fine-tuned the hyperparameters based on the original paper. The Chamfer Distance on the DTU dataset decreased to 0.47.

The Chamfer Distance↓ on the DTU dataset

The F1 Score↑ on the TnT dataset

Acknowledgements

This project is built upon 3DGS. Densify is based on AbsGau and GOF. DTU and Tanks and Temples dataset preprocess are based on Neuralangelo scripts. Evaluation scripts for DTU and Tanks and Temples dataset are based on DTUeval-python and TanksAndTemples respectively. We thank all the authors for their great work and repos.

Citation

If you find this code useful for your research, please use the following BibTeX entry.

@article{chen2024pgsr,
  title={PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction},
  author={Chen, Danpeng and Li, Hai and Ye, Weicai and Wang, Yifan and Xie, Weijian and Zhai, Shangjin and Wang, Nan and Liu, Haomin and Bao, Hujun and Zhang, Guofeng},
  journal={arXiv preprint arXiv:2406.06521},
  year={2024}
}