Tiny GPTV - Pytorch
Project description
TinyGPTV
Simple Implementation of TinyGPTV in super simple Zeta lego blocks. Here all the modules from figure 2 are implemented in Zeta and Pytorch
Usage
import torch
from tiny_gptv import TinyGPTVBlock
x = torch.rand(2, 8, 512)
lora_mha = TinyGPTVBlock(512, 8)
out = lora_mha(x)
print(out.shape)
Citation
@misc{yuan2023tinygptv,
title={TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones},
author={Zhengqing Yuan and Zhaoxu Li and Lichao Sun},
year={2023},
eprint={2312.16862},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
License
MIT
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tiny_gptv-0.0.1.tar.gz
(3.4 kB
view details)
Built Distribution
File details
Details for the file tiny_gptv-0.0.1.tar.gz.
File metadata
- Download URL: tiny_gptv-0.0.1.tar.gz
- Upload date:
- Size: 3.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 7cf0d92c22157361e75446bea097991f4dbb39df9095f64e741394edc5b5b00d |
|
| MD5 | 805a6a10160ebb7cadee93e2fed18d6d |
|
| BLAKE2b-256 | d531f1e41180600778e21eaf10ab94397fc08be15156ae46fe099220fd150f7b |
File details
Details for the file tiny_gptv-0.0.1-py3-none-any.whl.
File metadata
- Download URL: tiny_gptv-0.0.1-py3-none-any.whl
- Upload date:
- Size: 3.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 4b5b9fbbd2583794e2f166589cd1a98179074663fc9cf97a2c4e25778a1a77dd |
|
| MD5 | c0ea0637169f7d9312abbb8050a9b781 |
|
| BLAKE2b-256 | 5f5a7cb376029e1b1fc165933324e756671ec65d286ef4de6788b2abe8b4e980 |
