spandrel-foss 0.2.2

Give your project support for a variety of PyTorch model architectures, including auto-detecting model architecture from just .pth files. This version of Spandrel is FOSS compliant as it remove support for model architectures that are under a non-commercial license.

Spandrel

This version of Spandrel is FOSS compliant as it remove support for model architectures that are under a non-commercial license.

Spandrel is a library for loading and running pre-trained PyTorch models. It automatically detects the model architecture and hyperparameters from model files, and provides a unified interface for running models.

After seeing many projects extract out chaiNNer's model support into their own projects, I decided to create this PyPi package for the architecture support and model loading functionality. I'm also hoping that by having a central package anyone can use, the community will be encouraged to help add support for more models.

This package does not yet have easy inference code, but porting that code is planned as well.

Installation

Spandrel is available through pip:

pip install spandrel

Basic Usage

While Spandrel supports different kinds of models, this is how you would run a super resolution model (e.g. ESRGAN, SwinIR, HAT, etc.):

from spandrel import ImageModelDescriptor, ModelLoader
import torch

# load a model from disk
model = ModelLoader().load_from_file(r"path/to/model.pth")

# make sure it's an image to image model
assert isinstance(model, ImageModelDescriptor)

# send it to the GPU and put it in inference mode
model.cuda().eval()

# use the model
def process(image: torch.Tensor) -> torch.Tensor:
    with torch.no_grad():
        return model(image)

Note that model is a ModelDescriptor object, which is a wrapper around the actual PyTorch model. This wrapper provides a unified interface for running models, and also contains metadata about the model. See ImageModelDescriptor for more details about the metadata contained and how to call the model.

NOTE: ImageModelDescriptor will NOT convert an image to a tensor for you. It is purely making the forward passes of these models more convenient to use, since the actual forward passes are not always as simple as image in/image out.

Supported File Types

Spandrel mainly supports loading .pth files for all supported architectures. This is what you will typically find from official repos and community trained models. However, Spandrel also supports loading TorchScript traced models (.pt), certain types of .ckpt files, and .safetensors files for any supported architecture saved in one of these formats.

Model Architecture Support

NOTE: By its very nature, Spandrel will never be able to support every model architecture. The goal is just to support as many as is realistically possible.

Spandrel currently supports a limited amount of network architectures. If the architecture you need is not supported, feel free to request it or try adding it.

Single Image Super Resolution

• ESRGAN (RRDBNet)
  • This includes regular ESRGAN, ESRGAN+, "new-arch ESRGAN" (RealSR, BSRGAN), SPSR, and Real-ESRGAN
  • Models: Community ESRGAN | ESRGAN+ | BSRGAN | RealSR | Real-ESRGAN
• Real-ESRGAN Compact (SRVGGNet) | Models
• Swift-SRGAN | Models
• SwinIR | Models
• Swin2SR | Models
• HAT | Models
• Omni-SR | Models
• SRFormer | Models
• DAT | Models
• FeMaSR | Models
• GRL | Models
• DITN | Models
• MM-RealSR | Models
• SPAN | Models
• Real-CUGAN | Models, Pro Models
• CRAFT | Models
• SAFMN | Models

Face Restoration

• GFPGAN | 1.2, 1.3, 1.4
• RestoreFormer | Model
• CodeFormer | Model

Inpainting

• LaMa | Model
• MAT | Model

Denoising

• SCUNet | GAN Model | PSNR Model
• Uformer | Denoise SIDD Model | Deblur GoPro Model
• KBNet | Models
• NAFNet | Models

DeJPEG

• FBCNN | Models

Colorization

• DDColor | Models

Security

Use .safetensors files for guaranteed security.

As you may know, loading .pth files poses a security risk due to python's pickle module being inherently unsafe and vulnerable to arbitrary code execution (ACE). To mitigate this, Spandrel only allows deserializing certain types of data. This helps to improve security, but it still doesn't fully solve the issue of ACE.

Used By

Here are some cool projects that use Spandrel:

• AUTOMATIC1111's SD WebUI
• chaiNNer
• dgenerate

License Notice

This repo is bounded by GPLv3 license. However, the code of implemented architectures (everything inside an arch/ directory) is bound by their original respective licenses. See src/spandrel/architectures/<name>/arch/LICENSE for the license of a specific architecture.

The original architecture code has also been slightly modified and formatted to fit the needs of this repo. If you want to use these architectures in your own codebase (but why would you if you have this package 😉), I recommend grabbing them from their original sources.