Tiny GPTV - Pytorch
Project description
TinyGPTV
Simple Implementation of TinyGPTV in super simple Zeta lego blocks. Here all the modules from figure 2 are implemented in Zeta and Pytorch
Usage
import torch
from tiny_gptv import TinyGPTVBlock
x = torch.rand(2, 8, 512)
lora_mha = TinyGPTVBlock(512, 8)
out = lora_mha(x)
print(out.shape)
Citation
@misc{yuan2023tinygptv,
title={TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones},
author={Zhengqing Yuan and Zhaoxu Li and Lichao Sun},
year={2023},
eprint={2312.16862},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
License
MIT
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tiny_gptv-0.0.2.tar.gz
(3.4 kB
view hashes)
Built Distribution
Close
Hashes for tiny_gptv-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f3cac6f97519584aad92a6819960b58bbb5047f387ee9f86c01539db8db7b9c5 |
|
MD5 | 19dfea756c625d4ed673e48357e27f17 |
|
BLAKE2b-256 | b64b94bbe3c428a6062d5bb693b7aa65528ca8a9f3940261016261ea4a897f61 |