vsgan 1.2.0

VapourSynth GAN Implementation using RRDBNet, based on ESRGAN's implementation

VSGAN

VapourSynth Single Image Super-Resolution Generative Adversarial Network (GAN)

Introduction

This is a single image super-resolution generative adversarial network handler for VapourSynth. Since VapourSynth will take the frames from a video, and feed it to VSGAN, it is essentially a single video super-resolution gan. It is a direct port of ESRGAN by xinntao, so all results, accomplishments, and such that ESRGAN does, VSGAN will do too.

Using the right pre-trained model, on the right image, can have tremendous results.
Here's an example from a US Region 1 (NTSC) DVD of American Dad running with VSGAN (model not public)

Qualitive Comparisons against other Super-Resolution Strategies

Following comparisons were taken from ESRGAN's repo

Installation

Requirements

1. NVIDIA GPU that has support for CUDA 9.2+. A CUDA Compute Capability score of 6 or higher is recommended, and a score <= 2 will be incredibly slow, if it works at all.
2. CPU that isn't from the stone age. While this is going to do 90% of stuff on the GPU, a super bottle-knecking CPU could limit you're GPU's potential.
3. An ESRGAN model file to use. Either train one or get an already trained one. There's new models being trained every day in all kinds of communities, with all kinds of specific purposes for each model, like denoising, upscaling, cleaning, inpainting, b&w to color, e.t.c. You can find models on the Game Upscale Discord or their Upscale.wiki Model Database. The model database may not be as active as the Discord though.

Dependencies

Install dependencies in the listed order:

1. Python 3.6+ and pip. The required pip packages are listed in the requirements.txt file.
2. VapourSynth. Ensure the Python version you have installed is supported by the version you are installing. The supported Python versions may differ per OS.
3. NVIDIA CUDA.
4. PyTorch 1.6.0+, latest version is always recommended.

Important information when Installing Python, VapourSynth, and PyTorch

1. Ensure the Python version you have installed (or are going to install) is supported by the version of VapourSynth and PyTorch you are installing. The supported Python versions in correlation to a VapourSynth or PyTorch version may differ per OS, noticeably on Windows due to it's Python environment in general.
2. When installing Python and VapourSynth, you will be given the option to "Install for all users" by both. Make sure your chosen answer matches for both installations or VapourSynth and Python wont be able to find each other.

Important note for Windows users: It is very important for you to tick the checkbox "Add Python X.X to PATH" while installing. The Python installer's checkbox that states "Install launcher for all users" is not referring to the Python binaries. To install for all users you must click "Customize installation" and in there, after "Optional Features" section, it will have a checkbox titled "Install for all users" unticked by default so tick it.

Tips on Installing PyTorch

Go to the Get Started Locally page and choose the following options:

• PyTorch Build: Stable
• Package: Pip
• Language: Python
• CUDA: Latest available version, must match the installed version.

Then run the command provided by the Run this Command: text field.

Tips on Installing NVIDIA CUDA

Go to the NVIDIA CUDA Downloads page and download and install the version you selected on the PyTorch page earlier.

If you chose for example 11.0 then 11.0 and >= 11.0 versions should work, but if you chose for example 10.2, then chances are you need specifically version 10.2, and not > 10.2. However, I cannot confirm if this is the case.

Finally, Installing VSGAN

It's as simple as running pip install vsgan

Usage (Quick Example)

from vapoursynth import RGB24
from vsgan import VSGAN

# ...

# Create a VSGAN instance, which creates a pytorch device instance
vsgan = VSGAN("cuda")  # available options: `"cuda"`, `0`, `1`, ..., e.t.c
# Load an ESRGAN model into the VSGAN instance
# Tip: You can run load_model() at any point to change the model
vsgan.load_model(r"C:\Users\PHOENiX\Documents\ESRGAN Models\PSNR_x4_DB.pth")
# Convert the clip to RGB24 as ESRGAN can only work with linear RGB data
clip = core.resize.Point(clip, format=RGB24)  # RGB24 is a int constant that was imported earlier
# Use the VSGAN instance (with its loaded model) on a clip
clip = vsgan.run(clip)
# Convert back to any other color space if you wish.

# ...

# don't forget to set the output in your vapoursynth script
clip.set_output()

Documentation

VSGAN([device: int/str="cuda"])

Create a PyTorch Device instance using VSGAN for the provided device

• device: Acceptable values are "cuda", and a device id number (e.g. 0, 1). "cpu" is not allowed as it's simply too slow and I don't want people hurting their CPU's.

VSGAN.load_model(model: str)

Load a model into the VSGAN Device instance

• model: A path to an ESRGAN .pth model file.

VSGAN.run(clip: VideoNode[, chunk: bool=False])

Executes VSGAN on the provided clip, returning the resulting in a new clip.

• clip: Clip to use as the input frames. It must be RGB. It will also return as RGB.
• chunk: If your system is running out of memory, try enable chunk as it will split the image into smaller sub-images and render them one by one, then finally merging them back together. Trading memory requirements for speed and accuracy. WARNING: Since the images will be processed separately, the result may have issues on the edges of the chunks, an example of this issue.

VSGAN.execute(n: int, clip: VideoNode)

Executes the GAN model on nth frame from clip.

• n: Frame number.
• clip: Clip to get the frame from.