BRAVE or Swarms of Vision Transformers
Implementation of the paper: "BRAVE : Broadening the visual encoding of vision-language models". BRAVE achieves state-of-the-art performance on a broad range of captioning and VQA benchmarks and significantly reduces the aforementioned issues of VLMs, while requiring a smaller number of trainable parameters than existing methods and having a more compressed representation.
install
pip3 install brave-torch
usage
import torch
from brave_torch.main import SwarmOfViTs
# IMG Tensor
x = torch.randn(1, 3, 224, 224)
# Model
model = SwarmOfViTs(
image_size=224,
patch_size=32,
encoder_dim=512,
encoder_depth=6,
encoder_heads=8,
num_of_vits=4
)
# Forward
out = model(x)
print(out)
Citations
Todo
- Citation link
- Citation Bibtex
- Diagram photo
- Implement Andromeda Base LLM architecture
- Provide multi-modal tokenizer
- Train and release the model
1 maintainer
Metadata
1 maintainer
Release files for brave-torch 4.7.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| brave_torch-4.7.9.tar.gz | 6.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| brave_torch-4.7.9-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 12.7 kB
Release files / brave_torch-4.7.9.tar.gz
| Download URL | brave_torch-4.7.9.tar.gz |
|---|---|
| Size | 6.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ad2f4b63ef35520a08e8968016cc6b8870c3173b2008ba8a2655760ea0b310e6
|
|
BLAKE2b-256 checksum How to use checksums |
bb128b9ebe67ad9f7c4d74e31638e2fcbd5eb65f95c35c77a41bf3c42208dcb1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0
|
Release files / brave_torch-4.7.9-py3-none-any.whl
| Download URL | brave_torch-4.7.9-py3-none-any.whl |
|---|---|
| Size | 6.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1ecd965fe9ef4ae554a31075cde450dc3f342bc29b44f0d4d4b591f388db7c83
|
|
BLAKE2b-256 checksum How to use checksums |
eb7fe260d2829f894d9a9ab35c77dd76606566f9fc836a329393d3d0e3c3cba4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0
|
