Text to Video synthesis
Project description
Gen1
My Implementation of " Structure and Content-Guided Video Synthesis with Diffusion Models" by RunwayML
The flow:
image => midas => clip => spacetime unet => diffusion
Install
pip3 install gen1
Usage
import torch
from gen1.model import Gen1
model = Gen1()
images = torch.randn(1, 3, 128, 128)
video = torch.randn(1, 3, 16, 128, 128)
run_out = model.forward(images, video)
Usage
- Help us implement it we need help with the Midas, Clip, and modified Unet blocks
Citation
@misc{2302.03011,
Author = {Patrick Esser and Johnathan Chiu and Parmida Atighehchian and Jonathan Granskog and Anastasis Germanidis},
Title = {Structure and Content-Guided Video Synthesis with Diffusion Models},
Year = {2023},
Eprint = {arXiv:2302.03011},
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
gen1-0.0.2.tar.gz
(12.8 kB
view details)
Built Distribution
gen1-0.0.2-py3-none-any.whl
(13.4 kB
view details)
File details
Details for the file gen1-0.0.2.tar.gz.
File metadata
- Download URL: gen1-0.0.2.tar.gz
- Upload date:
- Size: 12.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 0aff0f1e723aae22415be8bc6ec730db71887d2c65c87cf5af3bd524bc40ac1c |
|
| MD5 | 1628505e5b0eedae8c32fdf9f8f943f0 |
|
| BLAKE2b-256 | 4e72a123ca3349cdd93167f563477036b2287bf705c8d8041aed620b8c56142d |
File details
Details for the file gen1-0.0.2-py3-none-any.whl.
File metadata
- Download URL: gen1-0.0.2-py3-none-any.whl
- Upload date:
- Size: 13.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 8ec76d94f38417051aaa55fb29b0ba0d06322fe85eb755e93697ccfc03fd7756 |
|
| MD5 | 46ea92d05aa36c2c6f7c72dfe057a4a0 |
|
| BLAKE2b-256 | 17c823c338f1b74d0c99ba2278e46c4485cf6c275fcf2d634f4f193d10528cf1 |
