Vision LLama

Implementation of VisionLLaMA from the paper: "VisionLLaMA: A Unified LLaMA Interface for Vision Tasks" in PyTorch and Zeta. PAPER LINK

install

$ pip install vision-llama

usage

import torch
from vision_llama.main import VisionLlama

# Forward Tensor
x = torch.randn(1, 3, 224, 224)

# Create an instance of the VisionLlamaBlock model with the specified parameters
model = VisionLlama(
    dim=768, depth=12, channels=3, heads=12, num_classes=1000
)


# Print the shape of the output tensor when x is passed through the model
print(model(x))

License

MIT

Citation

@misc{chu2024visionllama,
    title={VisionLLaMA: A Unified LLaMA Interface for Vision Tasks}, 
    author={Xiangxiang Chu and Jianlin Su and Bo Zhang and Chunhua Shen},
    year={2024},
    eprint={2403.00522},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

todo

Implement the AS2DRoPE rope, might just use axial rotary embeddings instead, my implementation is really bad
Implement the GSA attention, i implemented it but's bad
Add imagenet training script with distributed

Release files for vision-llama 0.0.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for vision-llama 0.0.8
File	Size	Uploaded
vision_llama-0.0.8.tar.gz	7.3 kB	Mar 6, 2024	Details

Built distribution (wheel)

Table of built distributions (wheels) for vision-llama 0.0.8
File	Interpreter	ABI	Platform	Reset
vision_llama-0.0.8-py3-none-any.whl 7.5 kB Mar 6, 2024	Python 3	none	any	Details

Total release size: 14.8 kB

Release files / vision_llama-0.0.8.tar.gz

Download URL	vision_llama-0.0.8.tar.gz
Size	7.3 kB
Tags	Source
SHA-256 checksum How to use checksums	`5adc93a897c33fed5db0f4fa05f7ec6254986990f4f4691b38e39f2d9d02cb6a`
BLAKE2b-256 checksum How to use checksums	`5168d3bd820836cfb702b873d7af9adc2eda4300ebf9758abe5e90f1a076ff98`
Upload date	Mar 6, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0`

Release files / vision_llama-0.0.8-py3-none-any.whl

Download URL	vision_llama-0.0.8-py3-none-any.whl
Size	7.5 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`e9ba5d07001b8115eff47e07bfbca15838d75c1f53065dfef5da0ccd2ffa7e28`
BLAKE2b-256 checksum How to use checksums	`a97d3bdcd336d5f261367182e6f8c3549476c78238bc56b11a12be4f8e6ffa20`
Upload date	Mar 6, 2024
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0`