Skip to main content

Path-space differentiable renderer with drjit as the numerical backend.

Project description

PyPI version DOI

psdr-jit

Path-space differentiable renderer with drjit as the numerical backend. Old repo: https://github.com/uci-rendering/psdr-cuda

Installation

We use OptiX 7 to accelerate ray tracing. Make sure cudatoolkit (you may install it using conda) and an appropriate NVIDIA display driver that support OptiX 7 are installed. The latest version we tested was cudatoolkit==11.7. Then run

pip install psdr-jit

Local Intallation

To install psdr-jit locally, first clone the repository recursivly (to include the git submodules):

git clone --recursive https://github.com/andyyankai/psdr-jit.git

or

git clone https://github.com/andyyankai/psdr-jit.git
cd psdr-jit
git submodule update --init --recursive

Then compile and install psdr-jit via

pip install .

or

pip install --no-build-isolation -ve .

Notice that it would also install drjit as psdr-jit depends on it. cudatoolkit is also required.

Getting Started

import drjit
import psdr_jit

Example of Diff rendering

import cv2
import sys
import torch

import psdr_jit as psdr
import drjit
from drjit.cuda.ad import Float as FloatD, Matrix4f as Matrix4fD
from drjit.cuda import Float as FloatC, Matrix4f as Matrix4fC

sc = psdr.Scene()
sc.opts.spp = 32 # Interior Term
sc.opts.sppe = 32 # Primary Edge
sc.opts.sppse = 32 # Secondary Edge
sc.opts.height = 512
sc.opts.width = 512

integrator = psdr.PathTracer(3)	


sensor = psdr.PerspectiveCamera(60, 0.000001, 10000000.)
to_world = Matrix4fD([[1.,0.,0.,208.],
                     [0.,1.,0.,273.],
                     [0.,0.,1.,-800.],
                     [0.,0.,0.,1.],])
sensor.to_world = to_world
sc.add_Sensor(sensor)

sc.add_BSDF(psdr.DiffuseBSDF([0.0, 0.0, 0.0]), "light")
sc.add_BSDF(psdr.DiffuseBSDF(), "cat")
sc.add_BSDF(psdr.DiffuseBSDF([0.95, 0.95, 0.95]), "white")
sc.add_BSDF(psdr.DiffuseBSDF([0.20, 0.90, 0.20]), "green")
sc.add_BSDF(psdr.DiffuseBSDF([0.90, 0.20, 0.20]), "red")

sc.add_Mesh("./data/objects/cbox/cbox_luminaire.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,-0.5],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "light", psdr.AreaLight([20.0, 20.0, 8.0]))
sc.add_Mesh("./data/objects/cbox/cbox_smallbox.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "cat", None)
sc.add_Mesh("./data/objects/cbox/cbox_largebox.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "cat", None)
sc.add_Mesh("./data/objects/cbox/cbox_floor.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "white", None)
sc.add_Mesh("./data/objects/cbox/cbox_ceiling.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "white", None)
sc.add_Mesh("./data/objects/cbox/cbox_back.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "white", None)
sc.add_Mesh("./data/objects/cbox/cbox_greenwall.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "green", None)
sc.add_Mesh("./data/objects/cbox/cbox_redwall.obj", Matrix4fC([[1.,0.,0.,0.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.]]), "red", None)

P = FloatD(0.)
drjit.enable_grad(P)

sc.param_map["Mesh[0]"].set_transform(Matrix4fD([[1.,0.,0.,P*100.],[0.,1.,0.,0.],[0.,0.,1.,0.],[0.,0.,0.,1.],]))


sc.configure()
sc.configure([0])

img = integrator.renderD(sc, 0)
org_img = img.numpy().reshape((sc.opts.width, sc.opts.height, 3))
output = cv2.cvtColor(org_img, cv2.COLOR_RGB2BGR)
cv2.imwrite("psdr_jit_forward.exr", output)


drjit.set_grad(P, 1.0)
drjit.forward_to(img)
diff_img = drjit.grad(img)
diff_img = diff_img.numpy().reshape((sc.opts.width, sc.opts.height, 3))
output = cv2.cvtColor(diff_img, cv2.COLOR_RGB2BGR)
cv2.imwrite("psdr_jit_diff_debug.exr", output)

More Tutorial

gradient evaluation tutorial in tutorial folder

Object level optimization example: Neural-PBIR (ICCV 2023)

Scene level optimization example: psdr-room (SIGGRAPH ASIA 2023)

SDF+Nerf optimization example: to be released

Project based on psdr-jit

TBA

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

psdr_jit-0.2.1.tar.gz (7.7 MB view details)

Uploaded Source

Built Distribution

psdr_jit-0.2.1-cp310-cp310-win_amd64.whl (5.3 MB view details)

Uploaded CPython 3.10 Windows x86-64

File details

Details for the file psdr_jit-0.2.1.tar.gz.

File metadata

  • Download URL: psdr_jit-0.2.1.tar.gz
  • Upload date:
  • Size: 7.7 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.10.14

File hashes

Hashes for psdr_jit-0.2.1.tar.gz
Algorithm Hash digest
SHA256 0ad1e7f455c4aedd4128788282159b03ccc5cedc7a87692bb9e54e234212bcbe
MD5 1019dce9f2d484f5bc6823f9ee2d51d0
BLAKE2b-256 153cf1a66429c00a424b7349075c6332e67c702a13779283c3202a04fd0d9a5d

See more details on using hashes here.

File details

Details for the file psdr_jit-0.2.1-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for psdr_jit-0.2.1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 337d90902d189f7d7a3ad51d2a4437ecc61ca27d1ab10b868536bcd3a21ea4d3
MD5 ab3655105579b14a289ef5003cbb53c3
BLAKE2b-256 e834a887c037b03dd479a6a8d05f9b12f816241f8d6b2743114b87980d3ed142

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page