Pytorch implementation of the CLIP guided loss.
Project description
pytorch_clip_guided_loss: Pytorch implementation of the CLIP guided loss for Text-To-Image, Image-To-Image, or Image-To-Text generation.
A simple library that implements CLIP guided loss in PyTorch.
Install package
pip install pytorch_clip_guided_loss
Install the latest version
pip install --upgrade git+https://github.com/bes-dev/pytorch_clip_guided_loss.git
Features
- The library supports multiple prompts (images or texts) as targets for optimization.
- The library automatically detects the language of the input text, and multilingual translate it via google translate.
- The library supports the original CLIP model by OpenAI and ruCLIP model by SberAI.
Usage
Simple code
import torch
from pytorch_clip_guided_loss import get_clip_guided_loss
loss_fn = get_clip_guided_loss(clip_type="ruclip", input_range = (-1, 1)).eval().requires_grad_(False)
# text prompt
loss_fn.add_prompt(text="text description of the what we would like to generate")
# image prompt
loss_fn.add_prompt(image=torch.randn(1, 3, 224, 224))
# variable
var = torch.randn(1, 3, 224, 224).requires_grad_(True)
loss = loss_fn.image_loss(image=var)["loss"]
loss.backward()
print(var.grad)
VQGAN-CLIP
We provide our tiny implementation of the VQGAN-CLIP pipeline for image generation as an example of the usage of our library. To start using our implementation of the VQGAN-CLIP please follow by documentation.
Zero-shot Object Detection
We provide our tiny implementation of the object detector based on Selective Search region proposals and CLIP guided loss. To start using our implementation of the ClipRCNN please follow by documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
File details
Details for the file pytorch_clip_guided_loss-2021.12.25.0-py3-none-any.whl
.
File metadata
- Download URL: pytorch_clip_guided_loss-2021.12.25.0-py3-none-any.whl
- Upload date:
- Size: 10.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.7.0 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.10
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 307a056490a3a2ff3b523a5f2e1936f851af7b164524c578ef8b786c9e1c2e68 |
|
MD5 | 8c4289000f4d6b4ccbe5404740fb773d |
|
BLAKE2b-256 | 8df67471c8fe659245ca8b4d70304d5d347c9168e30191ef20d9a8c759afc2c1 |