LiT: Zero-Shot Transfer with Locked-image text Tuning
Project description
pytorch-zero-lit
Converted official JAX models for LiT: Zero-Shot Transfer with Locked-image text Tuning to pytorch.
JAX -> Tensorflow -> ONNX -> Pytorch.
- Image encoder is loaded into pytorch and supports gradients
- Text encoder is not loaded into pytorch and runs via ONNX on cpu
Install
poetry add pytorch-zero-lit
or
pip install pytorch-zero-lit
Usage
from lit import LiT
model = LiT()
images = TF.to_tensor(
Image.open("cat.png").convert("RGB").resize((224, 224))
)[None]
texts = [
"a photo of a cat",
"a photo of a dog",
"a photo of a bird",
"a photo of a fish",
]
image_encodings = model.encode_images(images)
text_encodings = model.encode_texts(texts)
cosine_similarity = model.cosine_similarity(image_encodings, text_encodings)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pytorch-zero-lit-0.2.3.tar.gz
(4.1 kB
view hashes)
Built Distribution
Close
Hashes for pytorch_zero_lit-0.2.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | eefa460f4e6ff0053c9ab308cfbe4ff442f9ecc718cdd02eefecfe7dfb1b416e |
|
MD5 | 713443cfeb45f05dcadbccb6f68fce78 |
|
BLAKE2b-256 | 5a630c8319ca80537e862dd335663af05213977b8465bc16e2f8111e4901abbc |