LVSM - Pytorch
Project description
LVSM - Pytorch (wip)
Implementation of LVSM, SOTA Large View Synthesis with Minimal 3d Inductive Bias, from Adobe Research
We will focus only on the Decoder-only architecture in this repository.
This paper lines up with another from ICLR 2025
Install
$ pip install lvsm-pytorch
Usage
import torch
from lvsm_pytorch import LVSM
rays = torch.randn(2, 4, 6, 256, 256)
images = torch.randn(2, 4, 3, 256, 256)
target_rays = torch.randn(2, 6, 256, 256)
target_images = torch.randn(2, 3, 256, 256)
model = LVSM(
dim = 512,
max_image_size = 256,
patch_size = 32,
depth = 2,
)
loss = model(
input_images = images,
input_rays = rays,
target_rays = target_rays,
target_images = target_images
)
loss.backward()
# after much training
pred_images = model(
input_images = images,
input_rays = rays,
target_rays = target_rays,
) # (2, 3, 256, 256)
assert pred_images.shape == target_images.shape
Citations
@inproceedings{Jin2024LVSMAL,
title = {LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias},
author = {Haian Jin and Hanwen Jiang and Hao Tan and Kai Zhang and Sai Bi and Tianyuan Zhang and Fujun Luan and Noah Snavely and Zexiang Xu},
year = {2024},
url = {https://api.semanticscholar.org/CorpusID:273507016}
}
@article{Zhang2024CamerasAR,
title = {Cameras as Rays: Pose Estimation via Ray Diffusion},
author = {Jason Y. Zhang and Amy Lin and Moneish Kumar and Tzu-Hsuan Yang and Deva Ramanan and Shubham Tulsiani},
journal = {ArXiv},
year = {2024},
volume = {abs/2402.14817},
url = {https://api.semanticscholar.org/CorpusID:267782978}
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
lvsm_pytorch-0.0.10.tar.gz
(1.4 MB
view hashes)
Built Distribution
Close
Hashes for lvsm_pytorch-0.0.10-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0b301a3ff1d9d0932505be89a5047030389f500b76f0ac428eb578c8e750afbc |
|
MD5 | 4daa09d99422e2583b5234228f76f7b7 |
|
BLAKE2b-256 | 1e47d3280fefbe922ddc4d4eb10c391ed00efe8bd1da97835685bdb7adfe432b |