Skip to main content

3D Gaussian Splatting (Packaged Python Version)

Python versions PyPI version Downloads Total downloads CI CI

This repo is the refactored python training and inference code for 3D Gaussian Splatting. Forked from commit a2a91d9093fd791fb01f556fa717f8d9f2cfbdd7. We refactored the original code following the standard Python package structure, while keeping the algorithms used in the code identical to the original version.

Features

  • organize the code as a standard Python package
  • exposure compensation
  • camera and 3DGS parameters joint training
  • depth regularization
  • local relative depth regularization
  • image mask
  • integrated gsplat backend
  • integrated 2DGS from gsplat

Downstream Projects

Install

Prerequisites

PyPI Install

pip install --upgrade gaussian-splatting

or build latest from source:

pip install wheel setuptools
pip install --upgrade git+https://github.com/yindaheng98/gaussian-splatting.git@master --no-build-isolation

Development Install

git clone --recursive https://github.com/yindaheng98/gaussian-splatting
cd gaussian-splatting
pip install tqdm plyfile tifffile numpy opencv-python pillow open3d
pip install git+https://github.com/nerfstudio-project/gsplat.git
pip install --target . --upgrade . --no-deps

Quick Start

  1. Download dataset (T&T+DB COLMAP dataset, size 650MB):
wget https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/datasets/input/tandt_db.zip -P ./data
unzip data/tandt_db.zip -d data/
  1. Train 3DGS with densification (same with original 3DGS)
python -m gaussian_splatting.train -s data/truck -d output/truck -i 30000 --mode densify
  1. Render or view 3DGS
python -m gaussian_splatting.render -s data/truck -d output/truck -i 30000 --mode densify
python -m gaussian_splatting.viewer -d output/truck -i 30000
  1. Joint training 3DGS and camera (load the trained 3DGS)
python -m gaussian_splatting.train -s data/truck -d output/truck-camera -i 30000 --mode camera -l output/truck/point_cloud/iteration_30000/point_cloud.ply
  1. Render 3DGS with optimized cameras
python -m gaussian_splatting.render -s data/truck -d output/truck-camera -i 30000 --mode camera --load_camera output/truck-camera/cameras.json

💡 This repo does not contain code for creating dataset. If you want to create your own dataset, please refer to InstantSplat or use convert.py.

💡 See .vscode/launch.json for more example. See gaussian_splatting.train and gaussian_splatting.render for full options.

2D Gaussian Splatting (2DGS)

  1. Train 2DGS with densification
python -m gaussian_splatting.train -s data/truck -d output/truck-2dgs -i 30000 --mode normal-densify --backend gsplat-2dgs
  1. Render or view 2DGS
python -m gaussian_splatting.render -s data/truck -d output/truck-2dgs -i 30000 --backend gsplat-2dgs
python -m gaussian_splatting.viewer -d output/truck-2dgs -i 30000 --backend gsplat-2dgs
  1. Joint training 2DGS and camera (load the trained 2DGS)
python -m gaussian_splatting.train -s data/truck -d output/truck-2dgs-camera -i 30000 --mode normal-camera -l output/truck-2dgs/point_cloud/iteration_30000/point_cloud.ply --backend gsplat-2dgs
  1. Render 2DGS with optimized cameras
python -m gaussian_splatting.render -s data/truck -d output/truck-2dgs-camera -i 30000 --mode camera --load_camera output/truck-2dgs-camera/cameras.json --backend gsplat-2dgs

(Optional) Generate depth maps before training

  1. Prepare Depth-Anything-V2
git clone https://github.com/DepthAnything/Depth-Anything-V2.git
mkdir checkpoints
wget -O checkpoints/depth_anything_v2_vitl.pth https://huggingface.co/depth-anything/Depth-Anything-V2-Large/resolve/main/depth_anything_v2_vitl.pth?download=true
  1. Generate depth maps
# (Recommanded) save depth map as floating-point tiff file
python tools/run_depth_anything_v2.py --encoder vitl --img-path data/truck/images --outdir data/truck/depths
# (not Recommanded) save depth map as uint8 png file
python Depth-Anything-V2/run.py --encoder vitl --pred-only --grayscale --img-path data/truck/images --outdir data/truck/depths

API Usage

Gaussian models

GaussianModel is the basic 3DGS model.

from gaussian_splatting import GaussianModel
gaussians = GaussianModel(sh_degree).to(device)

If you want cameras-3DGS joint training, use CameraTrainableGaussianModel, the rendering process is different.

from gaussian_splatting import CameraTrainableGaussianModel
gaussians = CameraTrainableGaussianModel(sh_degree).to(device)

save and load params:

gaussians.save_ply("output/truck/point_cloud/iteration_30000/point_cloud.ply")
gaussians.load_ply("output/truck/point_cloud/iteration_30000/point_cloud.ply")

init 3DGS with sparse point cloud extracted by colmap:

from gaussian_splatting.dataset.colmap import colmap_init
colmap_init(gaussians, "data/truck")

Dataset

Basic colmap dataset:

from gaussian_splatting.dataset.colmap import ColmapCameraDataset, colmap_init
dataset = ColmapCameraDataset("data/truck")

save to JSON and load JSON dataset:

dataset.save_cameras("output/truck/cameras.json")
from gaussian_splatting import JSONCameraDataset
dataset = JSONCameraDataset("output/truck/cameras.json")

Dataset with trainable cameras:

from gaussian_splatting import TrainableCameraDataset
dataset = TrainableCameraDataset("data/truck") # init cameras from colmap
dataset = TrainableCameraDataset.from_json("output/truck/cameras.json") # init cameras from saved json

Inference

for camera in dataset:
  out = gaussians(camera)
  image = out["render"]
  ... # compute loss, save image or others

Trainers

gaussian_splatting.trainer contains a series of trainers for optimizing 3DGS models.

Core Trainers

Basic training methods​ that handle fundamental optimization tasks:

BaseTrainer only optimize the 3DGS parameters, without densification or camera optimization.

from gaussian_splatting.trainer import BaseTrainer
trainer = BaseTrainer(
    gaussians,
    scene_extent=dataset.scene_extent(),
    ... # see gaussian_splatting/trainer/base.py for full options
)

BaseDensificationTrainer optimize the 3DGS parameters with densification.

from gaussian_splatting.trainer import BaseDensificationTrainer
trainer = BaseDensificationTrainer(
    gaussians,
    scene_extent=dataset.scene_extent(),
    ... # see gaussian_splatting/trainer/densifier/densifier.py for full options
)

BaseCameraTrainer jointly optimize the 3DGS parameters and cameras, without densification.

from gaussian_splatting.trainer import BaseCameraTrainer
trainer = BaseCameraTrainer(
    gaussians,
    scene_extent=dataset.scene_extent(),
    dataset=dataset,
    ... # see gaussian_splatting/trainer/camera_trainable.py for full options
)

BaseDepthTrainer optimize the 3DGS parameters with depth regularization.

from gaussian_splatting.trainer import BaseDepthTrainer
trainer = BaseDepthTrainer(
    gaussians,
    scene_extent=dataset.scene_extent(),
    ... # see gaussian_splatting/trainer/base.py for full options
)

DepthCameraTrainer integrated BaseCameraTrainer with depth regularization.

from gaussian_splatting.trainer import SHLiftTrainer
trainer = SHLiftTrainer(
    gaussians,
    scene_extent=dataset.scene_extent(),
    dataset=dataset,
    ... # see gaussian_splatting/trainer/sh_lift.py for full options
)

Enhanced Trainers

Gaussian Splatting paper also introduce two tricks "opacity reset" and "lifting SH", they are also included. The basic methods can be integrated with opacity reset and lifting SH. For example:

BaseOpacityResetDensificationTrainer integrated BaseDensificationTrainer with opacity reset.

from gaussian_splatting.trainer import BaseOpacityResetDensificationTrainer
trainer = OpacityResetDensificationTrainer(
    gaussians,
    scene_extent=dataset.scene_extent(),
    ... # see gaussian_splatting/trainer/combinations.py for full options
)

DepthOpacityResetDensificationTrainer integrated BaseOpacityResetDensificationTrainer with depth regularization.

from gaussian_splatting.trainer import DepthOpacityResetDensificationTrainer
trainer = DepthOpacityResetDensificationTrainer(
    gaussians,
    scene_extent=dataset.scene_extent(),
    ... # see gaussian_splatting/trainer/combinations.py for full options
)

BaseSHLiftOpacityResetDensificationTrainer integrated BaseOpacityResetDensificationTrainer with lifting SH.

from gaussian_splatting.trainer import BaseSHLiftOpacityResetDensificationTrainer
trainer = BaseSHLiftOpacityResetDensificationTrainer(
    gaussians,
    scene_extent=dataset.scene_extent(),
    ... # see gaussian_splatting/trainer/combinations.py for full options
)

DepthSHLiftOpacityResetDensificationTrainer integrated DepthOpacityResetDensificationTrainer with lifting SH.

from gaussian_splatting.trainer import DepthSHLiftOpacityResetDensificationTrainer
trainer = DepthSHLiftOpacityResetDensificationTrainer(
    gaussians,
    scene_extent=dataset.scene_extent(),
    ... # see gaussian_splatting/trainer/combinations.py for full options
)

Similarly, there are BaseOpacityResetDensificationCameraTrainer, DepthOpacityResetDensificationCameraTrainer, BaseSHLiftOpacityResetDensificationCameraTrainer, DepthSHLiftOpacityResetDensificationCameraTrainer that integrated the above with camera training.

For more, please refer to train.py and trainer/combinations.py.

Training

To use any trainer:

for camera in dataset:
    loss, out = trainer.step(camera)

Discussion on Additional Features

Local Relative Depth Regularization

Problem: Global Depth Rescaling Limitation

The default implementation uses DepthAnythingV2 for depth estimation (tools/run_depth_anything_v2.py). These estimated depth maps are then scaled using one global factor per scene (trainer/depth.py or in the github.com/graphdeco-inria/gaussian-splatting/utils/make_depth_scale.py). However, this approach suffers from local inaccuracies due to limitations inherent in monocular depth predictions.

As demonstrated below, monocular depth estimation frequently introduces local distortions with global rescaling (for instance, people are pouring wine, but the spout is not positioned directly above the wine glass.):

Using globally scaled depth alone results in artifacts and incorrectly placed surfaces during rendering:

Overlaying rendered depth map with DepthAnythingV2-estimated depth map manifests these shortcomings clearly. While background walls and foreground table approximately match ground truth, depth estimates for people remain significantly inaccurate:

Root Cause: Spatial Error Patterns in DepthAnythingV2

Although the output depth estimation from DepthAnythingV2 appears visually plausible when inspected independently (as illustrated in the figure below), local depth scale variations remain substantial.

Therefore, a single global scaling cannot account adequately for these local discrepancies.

Solution: Local relative depth regularization

Considering that DepthAnythingV2 produces relatively accurate local depth relationships, this repo introduces local relative depth regularization strategy. Specifically, the strategy involves:

  • Divide the depth map into small overlapping windows.
  • Compute scaling and offset corrections individually per window.
  • Apply these local corrections to guide model predictions.

Implementation details are provided in the function compute_local_relative_depth_loss in trainer/depth.py.

The resulting improvements are clearly visible, significantly reducing artifacts:

Overlaying it with DepthAnythingV2-estimated depth map:

Local regularization notably improves background alignment (e.g., walls), but some inaccuracies remain for complex foreground shapes such as people. This clearly highlights inherent limitations and persistent spatial error patterns in the monocular DepthAnythingV2 estimations.

3D Gaussian Splatting for Real-Time Radiance Field Rendering

Bernhard Kerbl*, Georgios Kopanas*, Thomas Leimkühler, George Drettakis (* indicates equal contribution)
| Webpage | Full Paper | Video | Other GRAPHDECO Publications | FUNGRAPH project page |
| T&T+DB COLMAP (650MB) | Pre-trained Models (14 GB) | Viewers for Windows (60MB) | Evaluation Images (7 GB) |
Teaser image

This repository contains the official authors implementation associated with the paper "3D Gaussian Splatting for Real-Time Radiance Field Rendering", which can be found here. We further provide the reference images used to create the error metrics reported in the paper, as well as recently created, pre-trained models.

Abstract: Radiance Field methods have recently revolutionized novel-view synthesis of scenes captured with multiple photos or videos. However, achieving high visual quality still requires neural networks that are costly to train and render, while recent faster methods inevitably trade off speed for quality. For unbounded and complete scenes (rather than isolated objects) and 1080p resolution rendering, no current method can achieve real-time display rates. We introduce three key elements that allow us to achieve state-of-the-art visual quality while maintaining competitive training times and importantly allow high-quality real-time (≥ 30 fps) novel-view synthesis at 1080p resolution. First, starting from sparse points produced during camera calibration, we represent the scene with 3D Gaussians that preserve desirable properties of continuous volumetric radiance fields for scene optimization while avoiding unnecessary computation in empty space; Second, we perform interleaved optimization/density control of the 3D Gaussians, notably optimizing anisotropic covariance to achieve an accurate representation of the scene; Third, we develop a fast visibility-aware rendering algorithm that supports anisotropic splatting and both accelerates training and allows realtime rendering. We demonstrate state-of-the-art visual quality and real-time rendering on several established datasets.

BibTeX

@Article{kerbl3Dgaussians,
      author       = {Kerbl, Bernhard and Kopanas, Georgios and Leimk{\"u}hler, Thomas and Drettakis, George},
      title        = {3D Gaussian Splatting for Real-Time Radiance Field Rendering},
      journal      = {ACM Transactions on Graphics},
      number       = {4},
      volume       = {42},
      month        = {July},
      year         = {2023},
      url          = {https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/}
}

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gaussian_splatting-2.6.5.tar.gz (4.5 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

gaussian_splatting-2.6.5-cp312-cp312-win_amd64.whl (977.3 kB view details)

Uploaded CPython 3.12Windows x86-64

gaussian_splatting-2.6.5-cp311-cp311-win_amd64.whl (975.7 kB view details)

Uploaded CPython 3.11Windows x86-64

gaussian_splatting-2.6.5-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (7.5 MB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64

gaussian_splatting-2.6.5-cp310-cp310-win_amd64.whl (973.3 kB view details)

Uploaded CPython 3.10Windows x86-64

gaussian_splatting-2.6.5-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (7.4 MB view details)

Uploaded CPython 3.10manylinux: glibc 2.17+ x86-64

File details

Details for the file gaussian_splatting-2.6.5.tar.gz.

File metadata

  • Download URL: gaussian_splatting-2.6.5.tar.gz
  • Upload date:
  • Size: 4.5 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.12.10

File hashes

Hashes for gaussian_splatting-2.6.5.tar.gz
Algorithm Hash digest
SHA256 44c00e5a100e4729a0fb15a1bd82b8aac03b130d383a64042adb1bfe1f43b4db
MD5 9d07c824673bd422c3df88ebbf4dd8dc
BLAKE2b-256 f6e538d0cc4e9470912c0b00123d8b225508d0e8c324a0711b82bb3db4fd6f04

See more details on using hashes here.

File details

Details for the file gaussian_splatting-2.6.5-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for gaussian_splatting-2.6.5-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 ed867a74b67ca4fe0d011afae2db8a3d042593c5a3f9dd2dbe2b45b9563b9e19
MD5 0d83938090da320720ea538533db05e1
BLAKE2b-256 3738cfa0ab28f1c9aff4604990f91858d9b23085c905671eddabd8cb6bdd33ef

See more details on using hashes here.

File details

Details for the file gaussian_splatting-2.6.5-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for gaussian_splatting-2.6.5-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 fc23d4b6900133acdbe495ac3c51dadc42a4f1883ea4bf0ac006bc5d073dc8aa
MD5 574eb1b229605619068c8eb31e59f242
BLAKE2b-256 16d1c879238f886a7ba9ec6cc550d3175bad647afa7f0e9698a3000ad722545e

See more details on using hashes here.

File details

Details for the file gaussian_splatting-2.6.5-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for gaussian_splatting-2.6.5-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 bf3ac843ef5be2181d8c5995908c6f0a40dc7f191133e4d00a2764bb9b7713d2
MD5 c4eca4966271abe8126a078a615bd2bc
BLAKE2b-256 c2a055fa95723c4445e0a8d724b90e6ccfaa27648c6d75cba164c087384a4d67

See more details on using hashes here.

File details

Details for the file gaussian_splatting-2.6.5-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for gaussian_splatting-2.6.5-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 4a6f293de6804bee2f6c08e8859260723ba161658bf5b22e15b74d913ce3db42
MD5 186efed7534c8c55c5ef679fa3584fca
BLAKE2b-256 17b185956075299abde683ef454b351bd2f0a0f45f911cf5df049f959b8fa665

See more details on using hashes here.

File details

Details for the file gaussian_splatting-2.6.5-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for gaussian_splatting-2.6.5-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 e8317e2c3e24610fd3183a1571d0aed3bb3bcc3caf16761118fc90eb299b43c0
MD5 72ccb8911cce9ba60f33d6923a497c03
BLAKE2b-256 64b381a9f14550afa3d9e7bfe0888b90e823cfbc40d57999ff48da2f9c132ecf

See more details on using hashes here.

Release history Release notifications | RSS feed

2.9.0

6 files

2.8.4

6 files

2.8.3

6 files

2.8.2

6 files

2.8.1

6 files

2.8.0

6 files

2.7.1

6 files

2.7.0

6 files

2.6.7

6 files

2.6.6

6 files

This release

2.6.5 This release

6 files

2.6.4

6 files

2.6.3

6 files

2.6.2

6 files

2.6.1

6 files

2.6.0

2 files

2.5.2

2 files

2.5.1

2 files

2.5.0

2 files

2.3.9

6 files

2.3.8.3

6 files

2.3.8.2

6 files

2.3.8.1

6 files

2.3.8

6 files

2.3.7

6 files

2.3.6.1

4 files

2.3.6

4 files

2.3.5

4 files

2.3.4.1

4 files

2.3.4

4 files

2.3.3

4 files

2.3.2.1

4 files

2.3.2

4 files

2.3.1

4 files

2.3.0

4 files

2.2.2

4 files

2.2.1

4 files

2.2.0

4 files

2.1.0

4 files

2.0.2

4 files

2.0.1

4 files

2.0.0

4 files

1.25.1

4 files

1.25.0

4 files

1.24.0

4 files

1.23.3

4 files

1.23.2

4 files

1.23.1

4 files

1.23.0

4 files

1.22.2

6 files

1.22.1

6 files

1.22.0

6 files

1.21.3

6 files

1.21.2

6 files

1.21.1

6 files

1.21.0

6 files

1.20.9

6 files

1.20.8

6 files

1.20.7

6 files

1.20.6

6 files

1.20.5

6 files

1.20.4

6 files

1.20.3.1

6 files

1.20.3

6 files

1.20.2

6 files

1.20.1

6 files

1.20.0

6 files

1.19.5

6 files

1.19.4

6 files

1.19.3

4 files

1.19.2

6 files

1.19.1

6 files

1.19.0

6 files

1.18.1

6 files

1.18.0

6 files

1.17.11

6 files

1.17.10

6 files

1.17.9

6 files

1.17.8

6 files

1.17.7

6 files

1.17.6

6 files

1.17.5

6 files

1.17.4

6 files

1.17.3

6 files

1.17.2

6 files

1.17.1

6 files

1.17.0

6 files

1.16.14

6 files

1.16.13

6 files

1.16.12

6 files

1.16.11

6 files

1.16.10

6 files

1.16.9

6 files

1.16.8

6 files

1.16.7

6 files

1.16.6

6 files

1.16.5

6 files

1.16.4

6 files

1.16.3

6 files

1.16.2

6 files

1.16.1

6 files

1.16.0

6 files

1.15.6

6 files

1.15.5

6 files

1.15.4

6 files

1.15.3

6 files

1.15.2

6 files

1.15.1

6 files

1.15.0

6 files

1.14.9

6 files

1.14.8

6 files

1.14.7

6 files

1.14.6

6 files

1.14.5

6 files

1.14.4

4 files

1.14.3

4 files

1.14.2

4 files

1.14.1

4 files

1.14.0

4 files

1.13.9

4 files

1.13.8

4 files

1.13.7

4 files

1.13.6

4 files

1.13.5

4 files

1.13.4

4 files

1.13.3

4 files

1.13.2

4 files

1.13.1

4 files

1.13.0

4 files

1.12.2

4 files

1.12.1

4 files

1.12.0

4 files

1.11.6

4 files

1.11.5

4 files

1.11.4

4 files

1.11.3

4 files

1.11.2

4 files

1.11.1

4 files

1.11.0

4 files

1.10.9

4 files

1.10.8

4 files

1.10.7

4 files

1.10.6

4 files

1.10.5

4 files

1.10.4

4 files

1.10.3

4 files

1.10.2

4 files

1.10.1

4 files

1.10.0

4 files

1.9.1

4 files

1.9.0

4 files

1.8.9

4 files

1.8.8

4 files

1.8.7

4 files

1.8.6

4 files

1.8.5

4 files

1.8.4

4 files

1.8.3

4 files

1.8.2

4 files

1.8.1

4 files

1.8.0

4 files

1.7.7

4 files

1.7.6

4 files

1.7.5

4 files

1.7.4

4 files

1.7.3

6 files

1.7.2

6 files

1.7.1

6 files

1.7.0

6 files

1.6.8

6 files

1.6.7

6 files

1.6.6

6 files

1.6.5

6 files

1.6.4

6 files

1.6.3

6 files

1.6.2

6 files

1.6.1

6 files

1.6

6 files

1.5.2

6 files

1.5.1

6 files

1.5.0

6 files

1.4.5

6 files

1.4.4

6 files

1.4.3

6 files

1.4.2

6 files

1.4.1

6 files

1.4.0

6 files

1.3.2.1

6 files

1.3.2

6 files

1.3.1.1

6 files

1.3.1

6 files

1.3

6 files

1.2.1

6 files

1.2

6 files

1.1

6 files

1.0

6 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page