Path-space differentiable renderer with drjit as the numerical backend.
Project description
psdr-jit
Path-space differentiable renderer with drjit
as the numerical backend.
Old repo: https://github.com/uci-rendering/psdr-cuda
Installation
We use OptiX 7
to accelerate ray tracing. Make sure cudatoolkit
(you may install it using conda
) and an appropriate NVIDIA display driver that support OptiX 7
are installed. The latest version we tested was cudatoolkit==11.7
. Then run
pip install psdr-jit
Local Intallation
To install psdr-jit
locally, first clone the repository recursivly (to include the git submodules):
git clone --recursive https://github.com/andyyankai/psdr-jit.git
or
git clone https://github.com/andyyankai/psdr-jit.git
cd psdr-jit
git submodule update --init --recursive
Then compile and install psdr-jit
via
pip install .
or
pip install --no-build-isolation -ve .
Notice that it would also install drjit
as psdr-jit
depends on it. cudatoolkit
is also required.
Getting Started
import drjit
import psdr_jit
Example of Diff rendering
import cv2
import sys
import torch
import psdr_jit as psdr
import drjit
from drjit.cuda.ad import Float as FloatD, Matrix4f as Matrix4fD
from drjit.cuda import Float as FloatC, Matrix4f as Matrix4fC
sc = psdr.Scene()
sc.opts.spp = 32 # Interior Term
sc.opts.sppe = 32 # Primary Edge
sc.opts.sppse = 32 # Secondary Edge
sc.opts.height = 512
sc.opts.width = 512
integrator = psdr.PathTracer(3)
sensor = psdr.PerspectiveCamera(60, 0.000001, 10000000.)
to_world = Matrix4fD([[1.,0.,0.,208.],
[0.,1.,0.,273.],
[0.,0.,1.,-800.],
[0.,0.,0.,1.],])
sensor.to_world = to_world
sc.add_Sensor(sensor)
sc.add_BSDF(psdr.DiffuseBSDF([0.0, 0.0, 0.0]), "light")
sc.add_BSDF(psdr.DiffuseBSDF(), "cat")
sc.add_BSDF(psdr.DiffuseBSDF([0.95, 0.95, 0.95]), "white")
sc.add_BSDF(psdr.DiffuseBSDF([0.20, 0.90, 0.20]), "green")
sc.add_BSDF(psdr.DiffuseBSDF([0.90, 0.20, 0.20]), "red")
sc.add_Mesh("./data/objects/cbox/cbox_luminaire.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,-0.5],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "light", psdr.AreaLight([20.0, 20.0, 8.0]))
sc.add_Mesh("./data/objects/cbox/cbox_smallbox.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "cat", None)
sc.add_Mesh("./data/objects/cbox/cbox_largebox.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "cat", None)
sc.add_Mesh("./data/objects/cbox/cbox_floor.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "white", None)
sc.add_Mesh("./data/objects/cbox/cbox_ceiling.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "white", None)
sc.add_Mesh("./data/objects/cbox/cbox_back.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "white", None)
sc.add_Mesh("./data/objects/cbox/cbox_greenwall.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "green", None)
sc.add_Mesh("./data/objects/cbox/cbox_redwall.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "red", None)
P = FloatD(0.)
drjit.enable_grad(P)
sc.param_map["Mesh[0]"].set_transform(Matrix4fD([[1.,0.,0.,P*100.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.],]))
sc.configure()
sc.configure([0])
img = integrator.renderD(sc, 0)
org_img = img.numpy().reshape((sc.opts.width, sc.opts.height, 3))
output = cv2.cvtColor(org_img, cv2.COLOR_RGB2BGR)
cv2.imwrite("psdr_jit_forward.exr", output)
drjit.set_grad(P, 1.0)
drjit.forward_to(img)
diff_img = drjit.grad(img)
diff_img = diff_img.numpy().reshape((sc.opts.width, sc.opts.height, 3))
output = cv2.cvtColor(diff_img, cv2.COLOR_RGB2BGR)
cv2.imwrite("psdr_jit_diff_debug.exr", output)
More Tutorial
gradient evaluation tutorial in tutorial
folder
Object level optimization example: Neural-PBIR
(ICCV 2023)
Scene level optimization example: psdr-room
(SIGGRAPH ASIA 2023)
SDF+Nerf optimization example: to be released
Project based on psdr-jit
TBA
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file psdr_jit-0.2.1.tar.gz
.
File metadata
- Download URL: psdr_jit-0.2.1.tar.gz
- Upload date:
- Size: 7.7 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.14
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0ad1e7f455c4aedd4128788282159b03ccc5cedc7a87692bb9e54e234212bcbe |
|
MD5 | 1019dce9f2d484f5bc6823f9ee2d51d0 |
|
BLAKE2b-256 | 153cf1a66429c00a424b7349075c6332e67c702a13779283c3202a04fd0d9a5d |
File details
Details for the file psdr_jit-0.2.1-cp310-cp310-win_amd64.whl
.
File metadata
- Download URL: psdr_jit-0.2.1-cp310-cp310-win_amd64.whl
- Upload date:
- Size: 5.3 MB
- Tags: CPython 3.10, Windows x86-64
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.10.14
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 337d90902d189f7d7a3ad51d2a4437ecc61ca27d1ab10b868536bcd3a21ea4d3 |
|
MD5 | ab3655105579b14a289ef5003cbb53c3 |
|
BLAKE2b-256 | e834a887c037b03dd479a6a8d05f9b12f816241f8d6b2743114b87980d3ed142 |