RuDOLPH: One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP
Project description
RUDOLPH 🦌🎄☃️
One Hyper-Tasking Transformer can be creative as DALL-E and GPT-3 and smart as CLIP
RUssian Decoder On Language Picture Hyper-tasking (RUDOLPH) is a text-image-text transformer designed for an easy fine-tuning for a range of tasks: from generating images by text description and image classification to visual question answering and more. This model demonstrates the power of Hyper-tasking Transformers.
Hyper-tasking model is a generalized multi-tasking model, i.e., the model that can solve almost all tasks within supported modalities, mandatory including mutual pairwise translations between modalities (two modalities in case of RUDOLPH: images and Russian texts).
Models
The following table shows the values of the parameters corresponding to different RUDOLPH versions.
350M | 1.3B | 2.7B | |
---|---|---|---|
l | 64 | 128 | 384 |
r | 64 | 128 | 128 |
m | 16 | 32 | 24 |
n | 16 | 32 | 24 |
Sparse Attention Mask
350M
row - col - row - [last] conv
1.3B
row - col - row - [last] conv
2.7B
row - col - row - [last] conv
Installing
pip install rudolph==0.0.1rc10
Usage and Fine-Tuning
Usage and fine-tuning examples for different versions of RUDOLPH can be found in jupyters folder.
Citation
@misc{github2022ruDolph,
title = {RUDOLPH: One Hyper-Tasking Transformer can be creative as DALL-E and GPT-3 and smart as CLIP},
author = {AIRI},
year = {2022},
howpublished = {\url{https://github.com/ai-forever/ru-dolph}},
}
Supported by
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file rudolph-0.0.1rc10.tar.gz
.
File metadata
- Download URL: rudolph-0.0.1rc10.tar.gz
- Upload date:
- Size: 22.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.9.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | d5bc6c20c9bd38c61a536077243ef9f760025b8b730fd37d7678f2524e2f47ab |
|
MD5 | 1608bb20152b0071f87900ff50ebb9ae |
|
BLAKE2b-256 | 0512258e75c7bef66c69ef7ffb9707dfdcc9786ba081c1478721143a458e8a2a |
File details
Details for the file rudolph-0.0.1rc10-py3-none-any.whl
.
File metadata
- Download URL: rudolph-0.0.1rc10-py3-none-any.whl
- Upload date:
- Size: 22.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.1 CPython/3.9.13
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e861d9a4a9fe080a62b8d054e9fc4bde19a091a3f0844d91d200a513779aa658 |
|
MD5 | 873192501c9ee450fe4a468c5cf3fc50 |
|
BLAKE2b-256 | d52e14860b23c36426cc18ca827fbe505b54ea171f3420dd8f82c6e8e6721ad4 |