RuDOLPH: One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP
Project description
RUDOLPH 🦌🎄☃️
One Hyper-Tasking Transformer can be creative as DALL-E and GPT-3 and smart as CLIP
RUssian Decoder On Language Picture Hyper-tasking (RUDOLPH) is a text-image-text transformer designed for an easy fine-tuning for a range of tasks: from generating images by text description and image classification to visual question answering and more. This model demonstrates the power of Hyper-tasking Transformers.
Hyper-tasking model is a generalized multi-tasking model, i.e., the model that can solve almost all tasks within supported modalities, mandatory including mutual pairwise translations between modalities (two modalities in case of RUDOLPH: images and Russian texts).
Models
The following table shows the values of the parameters corresponding to different RUDOLPH versions.
350M | 1.3B | 2.7B | |
---|---|---|---|
l | 64 | 128 | 384 |
r | 64 | 128 | 128 |
m | 16 | 32 | 24 |
n | 16 | 32 | 24 |
Sparse Attention Mask
350M
row - col - row - [last] conv
1.3B
row - col - row - [last] conv
2.7B
row - col - row - [last] conv
Installing
pip install rudolph==0.0.1rc10
Usage and Fine-Tuning
Usage and fine-tuning examples for different versions of RUDOLPH can be found in jupyters folder.
Citation
@misc{github2022ruDolph,
title = {RUDOLPH: One Hyper-Tasking Transformer can be creative as DALL-E and GPT-3 and smart as CLIP},
author = {AIRI},
year = {2022},
howpublished = {\url{https://github.com/ai-forever/ru-dolph}},
}
Supported by
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for rudolph-0.0.1rc10-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e861d9a4a9fe080a62b8d054e9fc4bde19a091a3f0844d91d200a513779aa658 |
|
MD5 | 873192501c9ee450fe4a468c5cf3fc50 |
|
BLAKE2b-256 | d52e14860b23c36426cc18ca827fbe505b54ea171f3420dd8f82c6e8e6721ad4 |