TiTok - Pytorch (wip)
Implementation of TiTok, proposed by Bytedance in An Image is Worth 32 Tokens for Reconstruction and Generation
Install
$ pip install titok-pytorch
Usage
import torch
from titok_pytorch import TiTokTokenizer
images = torch.randn(2, 3, 256, 256)
titok = TiTokTokenizer(
dim = 1024,
patch_size = 32,
num_latent_tokens = 32, # they claim only 32 tokens needed
codebook_size = 4096 # codebook size 4096
)
loss = titok(images)
loss.backward()
# after much training
# extract codes for gpt, maskgit, whatever
codes = titok.tokenize(images) # (2, 32)
# reconstructing images from codes
recon_images = titok.codebook_ids_to_images(codes)
assert recon_images.shape == images.shape
Todo
- add multi-resolution patches
Citations
@article{yu2024an,
author = {Qihang Yu and Mark Weber and Xueqing Deng and Xiaohui Shen and Daniel Cremers and Liang-Chieh Chen},
title = {An Image is Worth 32 Tokens for Reconstruction and Generation},
journal = {arxiv: 2406.07550},
year = {2024}
}
1 maintainer
Metadata
1 maintainer
Release files for titok-pytorch 0.0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| titok_pytorch-0.0.5.tar.gz | 1.3 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| titok_pytorch-0.0.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.3 MB
Release files / titok_pytorch-0.0.5.tar.gz
| Download URL | titok_pytorch-0.0.5.tar.gz |
|---|---|
| Size | 1.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
49ffa0c92faf03b5cc536f84243c42c277a4b001b04adce42c4b3537c9238f1f
|
|
BLAKE2b-256 checksum How to use checksums |
c4d5c23e3ae618858f2c760b160555efb18ace98eda8e485379c38cf9e295e2d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.0 CPython/3.9.19
|
Release files / titok_pytorch-0.0.5-py3-none-any.whl
| Download URL | titok_pytorch-0.0.5-py3-none-any.whl |
|---|---|
| Size | 5.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
bf7e2c69c848c8386dc578ac2063191b2ffe757ac250f2b623013bc66d78f791
|
|
BLAKE2b-256 checksum How to use checksums |
f7597098442f96023cd13ed9c9101148e069b5e3d1bfa8bbb86bac27278ae406
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.1.0 CPython/3.9.19
|
