Skip to main content

Pytorch implementation of the CLIP guided loss.

Project description

pytorch_clip_guided_loss: Pytorch implementation of the CLIP guided loss for Text-To-Image, Image-To-Image, or Image-To-Text generation.

A simple library that implements CLIP guided loss in PyTorch.

Downloads Downloads Downloads

Install package

pip install pytorch_clip_guided_loss

Install the latest version

pip install --upgrade git+https://github.com/bes-dev/pytorch_clip_guided_loss.git

Features

  • The library supports multiple prompts (images or texts) as targets for optimization.
  • The library automatically detects the language of the input text, and multilingual translate it via google translate.
  • The library supports the original CLIP model by OpenAI and ruCLIP model by SberAI.

Usage

Simple code

import torch
from pytorch_clip_guided_loss import get_clip_guided_loss

loss_fn = get_clip_guided_loss(clip_type="ruclip", input_range = (-1, 1)).eval().requires_grad_(False)
# text prompt
loss_fn.add_prompt(text="text description of the what we would like to generate")
# image prompt
loss_fn.add_prompt(image=torch.randn(1, 3, 224, 224))

# variable
var = torch.randn(1, 3, 224, 224).requires_grad_(True)
loss = loss_fn.image_loss(image=var)["loss"]
loss.backward()
print(var.grad)

VQGAN-CLIP

We provide our tiny implementation of the VQGAN-CLIP pipeline for image generation as an example of the usage of our library. To start using our implementation of the VQGAN-CLIP please follow by documentation.

Zero-shot Object Detection

We provide our tiny implementation of the object detector based on Selective Search region proposals and CLIP guided loss. To start using our implementation of the ClipRCNN please follow by documentation.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

File details

Details for the file pytorch_clip_guided_loss-2021.12.25.0-py3-none-any.whl.

File metadata

  • Download URL: pytorch_clip_guided_loss-2021.12.25.0-py3-none-any.whl
  • Upload date:
  • Size: 10.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.0 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.10

File hashes

Hashes for pytorch_clip_guided_loss-2021.12.25.0-py3-none-any.whl
Algorithm Hash digest
SHA256 307a056490a3a2ff3b523a5f2e1936f851af7b164524c578ef8b786c9e1c2e68
MD5 8c4289000f4d6b4ccbe5404740fb773d
BLAKE2b-256 8df67471c8fe659245ca8b4d70304d5d347c9168e30191ef20d9a8c759afc2c1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page