MultiModalCrossAttn - Pytorch
Project description
MultiModalCrossAttn
The open source implementation of the cross attention mechanism from the paper: "JOINTLY TRAINING LARGE AUTOREGRESSIVE MULTIMODAL MODELS"
Appreciation
- Lucidrains
- Agorians
Install
pip install cross-attn
Usage
import torch
from cross_attn.main import MultiModalCrossAttention
# Test the MultiModalCrossAttention module
dim = 512 # For example
num_heads = 8
cross_attn = MultiModalCrossAttention(dim, num_heads)
Hllm_sample = torch.randn(32, dim, dim) # Batch size = 32, Sequence length = 10
Himg_sample = torch.randn(32, dim, dim)
output = cross_attn(Hllm_sample, Himg_sample)
print(output)
print(output.shape) # Expected: [32, 10, 512]
License
MIT
Citations
@misc{2309.15564,
Author = {Emanuele Aiello and Lili Yu and Yixin Nie and Armen Aghajanyan and Barlas Oguz},
Title = {Jointly Training Large Autoregressive Multimodal Models},
Year = {2023},
Eprint = {arXiv:2309.15564},
}
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
cross_attn-0.0.5.tar.gz
(8.9 kB
view hashes)
Built Distribution
cross_attn-0.0.5-py3-none-any.whl
(12.1 kB
view hashes)
Close
Hashes for cross_attn-0.0.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec2023b77e2be8ee15f40fe521f08d6b96700e4c443c751c33b3df9bf031baa8 |
|
MD5 | 252ee3b763979eae73d7c443ea4ca5e9 |
|
BLAKE2b-256 | 7b07a59c7c5fb4a38df1d13d98d4178f37ee8c650c97fbb7b92d72c8c1bd85b8 |