MultiModalCrossAttn
The open source implementation of the cross attention mechanism from the paper: "JOINTLY TRAINING LARGE AUTOREGRESSIVE MULTIMODAL MODELS"
Appreciation
- Lucidrains
- Agorians
Install
pip install cross-attn
Usage
import torch
from cross_attn.main import MultiModalCrossAttention
# Test the MultiModalCrossAttention module
dim = 512 # For example
num_heads = 8
cross_attn = MultiModalCrossAttention(dim, num_heads)
Hllm_sample = torch.randn(32, dim, dim) # Batch size = 32, Sequence length = 10
Himg_sample = torch.randn(32, dim, dim)
output = cross_attn(Hllm_sample, Himg_sample)
print(output)
print(output.shape) # Expected: [32, 10, 512]
License
MIT
Citations
@misc{2309.15564,
Author = {Emanuele Aiello and Lili Yu and Yixin Nie and Armen Aghajanyan and Barlas Oguz},
Title = {Jointly Training Large Autoregressive Multimodal Models},
Year = {2023},
Eprint = {arXiv:2309.15564},
}
1 maintainer
Metadata
1 maintainer
Release files for cross-attn 0.0.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| cross_attn-0.0.5.tar.gz | 8.9 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| cross_attn-0.0.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 21.1 kB
Release files / cross_attn-0.0.5.tar.gz
| Download URL | cross_attn-0.0.5.tar.gz |
|---|---|
| Size | 8.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
588b54dc7ea9e1515286977956ceadf0b81711c2ab2b0e8acb797c01d331ceba
|
|
BLAKE2b-256 checksum How to use checksums |
264511e7662ce13482e0d36438abbaea96e841af173d2c30a07a9072aaf405db
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
Release files / cross_attn-0.0.5-py3-none-any.whl
| Download URL | cross_attn-0.0.5-py3-none-any.whl |
|---|---|
| Size | 12.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ec2023b77e2be8ee15f40fe521f08d6b96700e4c443c751c33b3df9bf031baa8
|
|
BLAKE2b-256 checksum How to use checksums |
7b07a59c7c5fb4a38df1d13d98d4178f37ee8c650c97fbb7b92d72c8c1bd85b8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.3.2 CPython/3.11.0 Darwin/22.4.0
|
