VapourSynth GAN Implementation using RRDBNet, based on ESRGAN's implementation
VapourSynth Single Image Super-Resolution Generative Adversarial Network (GAN)
This is a single image super-resolution generative adversarial network handler for VapourSynth. Since VapourSynth will take the frames from a video, and feed it to VSGAN, it is essentially a single video super-resolution gan. It is a direct port of ESRGAN by xinntao, so all results, accomplishments, and such that ESRGAN does, VSGAN will do too.
Using the right pre-trained model, on the right image, can have tremendous results.
Here's an example from a US Region 1 (NTSC) DVD of American Dad running with VSGAN (model not public)
Qualitive Comparisons against other Super-Resolution Strategies
Following comparisons were taken from ESRGAN's repo
- NVIDIA GPU that has support for CUDA 9.2+. A CUDA Compute Capability score of 6 or higher is recommended, and a score <= 2 will be incredibly slow, if it works at all.
- CPU that isn't from the stone age. While this is going to do 90% of stuff on the GPU, a super bottle-knecking CPU could limit you're GPU's potential.
- An ESRGAN model file to use. Either train one or get an already trained one. There's new models being trained every day in all kinds of communities, with all kinds of specific purposes for each model, like denoising, upscaling, cleaning, inpainting, b&w to color, e.t.c. You can find models on the Game Upscale Discord or their Upscale.wiki Model Database. The model database may not be as active as the Discord though.
Install dependencies in the listed order:
- Python 3.6+ and pip. The required pip packages are listed in the requirements.txt file.
- VapourSynth. Ensure the Python version you have installed is supported by the version you are installing. The supported Python versions may differ per OS.
- NVIDIA CUDA.
- PyTorch 1.6.0+, latest version is always recommended.
Important information when Installing Python, VapourSynth, and PyTorch
- Ensure the Python version you have installed (or are going to install) is supported by the version of VapourSynth and PyTorch you are installing. The supported Python versions in correlation to a VapourSynth or PyTorch version may differ per OS, noticeably on Windows due to it's Python environment in general.
- When installing Python and VapourSynth, you will be given the option to "Install for all users" by both. Make sure your chosen answer matches for both installations or VapourSynth and Python wont be able to find each other.
Important note for Windows users: It is very important for you to tick the checkbox "Add Python X.X to PATH" while installing. The Python installer's checkbox that states "Install launcher for all users" is not referring to the Python binaries. To install for all users you must click "Customize installation" and in there, after "Optional Features" section, it will have a checkbox titled "Install for all users" unticked by default so tick it.
Tips on Installing PyTorch
Go to the Get Started Locally page and choose the following options:
- PyTorch Build:
- CUDA: Latest available version, must match the installed version.
Then run the command provided by the
Run this Command: text field.
Tips on Installing NVIDIA CUDA
Go to the NVIDIA CUDA Downloads page and download and install the version you selected on the PyTorch page earlier.
If you chose for example
11.0 and >=
11.0 versions should work, but if you chose for example
10.2, then chances are you need specifically version
10.2, and not >
10.2. However, I cannot confirm if this is the case.
Finally, Installing VSGAN
It's as simple as running
pip install vsgan
Usage (Quick Example)
from vapoursynth import RGB24 from vsgan import VSGAN # ... # Create a VSGAN instance, which creates a pytorch device instance vsgan = VSGAN("cuda") # available options: `"cuda"`, `0`, `1`, ..., e.t.c # Load an ESRGAN model into the VSGAN instance # Tip: You can run load_model() at any point to change the model vsgan.load_model(r"C:\Users\PHOENiX\Documents\ESRGAN Models\PSNR_x4_DB.pth") # Convert the clip to RGB24 as ESRGAN can only work with linear RGB data clip = core.resize.Point(clip, format=RGB24) # RGB24 is a int constant that was imported earlier # Use the VSGAN instance (with its loaded model) on a clip clip = vsgan.run(clip) # Convert back to any other color space if you wish. # ... # don't forget to set the output in your vapoursynth script clip.set_output()
Create a PyTorch Device instance using VSGAN for the provided device
device: Acceptable values are
"cuda", and a device id number (e.g.
"cpu"is not allowed as it's simply too slow and I don't want people hurting their CPU's.
Load a model into the VSGAN Device instance
model: A path to an ESRGAN .pth model file.
VSGAN.run(clip: VideoNode[, chunk: bool=False])
Executes VSGAN on the provided clip, returning the resulting in a new clip.
clip: Clip to use as the input frames. It must be RGB. It will also return as RGB.
chunk: If your system is running out of memory, try enable
chunkas it will split the image into smaller sub-images and render them one by one, then finally merging them back together. Trading memory requirements for speed and accuracy. WARNING: Since the images will be processed separately, the result may have issues on the edges of the chunks, an example of this issue.
VSGAN.execute(n: int, clip: VideoNode)
Executes the GAN model on
nth frame from
n: Frame number.
clip: Clip to get the frame from.
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size vsgan-1.2.1-py3-none-any.whl (12.9 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size vsgan-1.2.1.tar.gz (14.5 kB)||File type Source||Python version None||Upload date||Hashes View|