Transient rendering extensions for Mitsuba 3
Project description
mitransient
Transient light transport in Mitsuba 3
Diego Royo Miguel Crespo Jorge Garcia-Pueyo
Overview
mitransient is a library adds support to Mitsuba 3 for doing transient simulations, with amazing support for non-line-of-sight (NLOS) data capture simulations.
Main features
- Foundation ready to use: easy interface to convert your algorithms to the transient domain.
- Python-only library for doing transient rendering in both CPU and GPU.
- Several integrators already implemented: transient pathtracing (also adapted for NLOS scenes) and transient volumetric pathtracing.
- Cross-platform: Mitsuba 3 has been tested on Linux (x86_64), macOS (aarch64, x86_64), and Windows (x86_64).
- Temporal domain filtering.
[!TIP] Check out our online documentation (mitransient.readthedocs.io) and our code examples:
Featuring General Usage, Transient rendering, and Non-line-of-sight (NLOS) transient rendering
License and citation
This project was created by Miguel Crespo and expanded by Diego Royo and Jorge Garcia-Pueyo. Also see the original Mitsuba 3 license and contributors.
If you use our code in your project, please consider citing us using the following:
@misc{mitransient,
title = {mitransient},
author = {Royo, Diego and Crespo, Miguel and Garcia-Pueyo, Jorge},
year = 2023,
journal = {GitHub repository},
doi = {https://doi.org/10.5281/zenodo.11032518},
publisher = {GitHub},
howpublished = {\url{https://github.com/diegoroyo/mitransient}}
}
Additionally, the NLOS features were re-implemented from our publication Non-line-of-sight transient rendering. Please also consider citing us if you use them:
@article{royo2022non,
title = {Non-line-of-sight transient rendering},
author = {Diego Royo and Jorge García and Adolfo Muñoz and Adrian Jarabo},
year = 2022,
journal = {Computers & Graphics},
doi = {https://doi.org/10.1016/j.cag.2022.07.003},
issn = {0097-8493},
url = {https://www.sciencedirect.com/science/article/pii/S0097849322001200}
}
What is transient rendering?
Conventional rendering is referred to as steady state, where the light propagation speed is assumed to be infinite. In contrast, transient rendering breaks this assumption allowing us to simulate light in motion (see the teaser image for a visual example).
For example, path tracing algorithms integrate over multiple paths that connect a light source with the camera. For a known path, transient path tracing uses the very complex formula of time = distance / speed
(see [Two New Sciences by Galileo]) to compute the time
when each photon arrives at the camera from the path's distance
and light's speed
. This adds a new time
dimension to the captured images (i.e. it's a video now). The simulations now take new parameters as input: when to start recording the video, how long is each time step (framerate), and how many frames to record.
Note: note that the time
values we need to compute are very small (e.g. light takes only ~3.33 * 10^-9 seconds to travel 1 meter), time
is usually measured in optical path distance. See Wikipedia for more information. TL;DR opl = distance * refractive_index
Installation
We provide the package via PyPI. To install mitransient
you need to run:
pip install mitransient
[!IMPORTANT] In order to run
mitransient
you also need to install Mitsuba 3. If you have installed Mitsuba 3 viapip
(pip install mitsuba
) you will only have access to thellvm_ad_rgb
andcuda_ad_rgb
variants. If you want to use other variants (e.g. NLOS simulations can greatly benefit from thellvm_mono
variant which only propagates one wavelength), then we recommend that you compile Mitsuba 3 yourself following this tutorial and enable the following variants:["scalar_mono", "llvm_mono", "llvm_ad_mono", "cuda_mono", "cuda_ad_mono", "scalar_rgb", "llvm_rgb", "llvm_ad_rgb", "cuda_rgb", "cuda_ad_rgb"]
.
Requirements
Python >= 3.8
Mitsuba3 >= 3.5.0
- (optional) For computation on the GPU:
Nvidia driver >= 495.89
- (optional) For vectorized / parallel computation on the CPU:
LLVM >= 11.1
After installation
At this point, you should be able to import mitsuba
and import mitransient
in your Python code (careful about setting the correct PATH
environment variable if you have compiled Mitsuba 3 yourself, see the section below).
For NLOS data capture simulations, see https://github.com/diegoroyo/tal. tal
is a toolkit that allows you to create and simulate NLOS scenes with an easier shell interface instead of directly from Python.
If you use your own Mitsuba 3
If you have opted for using a custom (non-default installation through pip
) Mitsuba 3, you have several options for it. The idea here is to be able to control which version of Mitsuba will be loaded on demand.
- One solution is to directly execute
setpath.sh
provided after the compilation of the Mitsuba 3 repo (More info). This shell script will modify thePATH
andPYTHONPATH
variables to load first this version of Mitsuba. - Another solution following the previous one is to directly set yourself the
PYTHONPATH
environment variable as you wish. - Another solution for having a custom version globally available is by using
pip install . --editable
. This will create a symlink copy of the package files inside the correspondingsite-packages
folder and will be listed as a package installed ofpip
and will be available as other packages installed. If you recompile them, you will still have the newest version directly to use. Please follow these instructions:- Go to
<mitsuba-path>/mitsuba3/build/python/drjit
and executepip install . --editable
. - Go to
<mitsuba-path>/mitsuba3/build/python/mitsuba
and executepip install . --editable
.
- Go to
- If you are a user of Jupyter Notebooks, the easiest solution will be to add the following snippet of code to modify the notebook's
PYTHONPATH
:
import sys
sys.path.insert(0, '<mitsuba-path>/mitsuba3/build/python')
import mitsuba as mi
Usage
[!TIP] Check out the
examples
folder for practical usage!
You are now prepared to render your first transient scene with mitransient. Running the code below will render the famous Cornell Box scene in transient domain and show a video.
import mitsuba as mi
mi.set_variant('scalar_rgb')
import mitransient as mitr
scene = mi.load_dict(mitr.cornell_box())
transient_integrator = scene.integrator()
transient_integrator.prepare_transient(scene, sensor=0)
img_steady, img_transient = transient_integrator.render(scene)
mitr.utils.show_video(
np.moveaxis(img_transient, 0, 1),
axis_video=2,
)
Plugins implemented
mitransient
implements the following plugins which can be used in scene XML files. To view a description of their parameters, click on the name of your desired plugin.
film
:transient_hdr_film
: Transient equivalent of Mitsuba 3'shdrfilm
plugin.
integrators
:transient_path
: Transient path tracing for line-of-sight scenes. If you want to do NLOS simulations, usetransientnlospath
instead.transient_nlos_path
: Transient path tracing with specific sampling routines for NLOS scenes (e.g. laser sampling and hidden geometry sampling of the "Non-Line-of-Sight Transient Rendering" paper).transient_prbvolpath
: Path Replay Backpropagation for volumetric path tracing. Implemented by Miguel Crespo, untested.
sensor
:nlos_capture_meter
: Can be attached to one of the scene's geometries, and measures uniformly-spaced points on such geometry (e.g. relay wall).
Testing
Our test suite can be run using pytest
on the root folder of the repo.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file mitransient-1.0.4.tar.gz
.
File metadata
- Download URL: mitransient-1.0.4.tar.gz
- Upload date:
- Size: 37.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.11.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | c1f0f4168be10f4e049048b37d036f707e4f1391e8b4caa9df0e0e2802271b84 |
|
MD5 | 81841a9ab2030b0659a671c7f0db731e |
|
BLAKE2b-256 | a70ccf107266833c5d7efc92b06b3147068f60a73c1f2692749afae18ca879bd |
File details
Details for the file mitransient-1.0.4-py3-none-any.whl
.
File metadata
- Download URL: mitransient-1.0.4-py3-none-any.whl
- Upload date:
- Size: 43.5 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.11.3
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6df42d6a49e90f944093bda08503e6717ec14b8d74af4d28f6e2fa78eed8e5f5 |
|
MD5 | 5040e4ea5a5b1b025fc388627183bb7f |
|
BLAKE2b-256 | a8b1603b839c74d4c26dd38c1d8ca19eba0be49380b98100b40573db1a7a28cd |