Skip to main content

RuDOLPH: One Hyper-Modal Transformer can be creative as DALL-E and smart as CLIP

Project description

[Colab]

RUDOLPH 🦌🎄☃️

One Hyper-Tasking Transformer can be creative as DALL-E and GPT-3 and smart as CLIP


RUssian Decoder On Language Picture Hyper-tasking (RUDOLPH) is a text-image-text transformer designed for an easy fine-tuning for a range of tasks: from generating images by text description and image classification to visual question answering and more. This model demonstrates the power of Hyper-tasking Transformers.

Hyper-tasking model is a generalized multi-tasking model, i.e., the model that can solve almost all tasks within supported modalities, mandatory including mutual pairwise translations between modalities (two modalities in case of RUDOLPH: images and Russian texts).

Models

The following table shows the values of the parameters corresponding to different RUDOLPH versions.

350M 1.3B 2.7B
l 64 128 384
r 64 128 128
m 16 32 24
n 16 32 24

Sparse Attention Mask

350M

row - col - row - [last] conv

1.3B

row - col - row - [last] conv

2.7B

row - col - row - [last] conv

Installing

pip install rudolph==0.0.1rc10

Usage and Fine-Tuning

Usage and fine-tuning examples for different versions of RUDOLPH can be found in jupyters folder.

Citation

@misc{github2022ruDolph,
  title         = {RUDOLPH: One Hyper-Tasking Transformer can be creative as DALL-E and GPT-3 and smart as CLIP},
  author        = {AIRI},
  year          = {2022},
  howpublished  = {\url{https://github.com/ai-forever/ru-dolph}},
}

Supported by

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rudolph-0.0.1rc10.tar.gz (22.1 kB view details)

Uploaded Source

Built Distribution

rudolph-0.0.1rc10-py3-none-any.whl (22.8 kB view details)

Uploaded Python 3

File details

Details for the file rudolph-0.0.1rc10.tar.gz.

File metadata

  • Download URL: rudolph-0.0.1rc10.tar.gz
  • Upload date:
  • Size: 22.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for rudolph-0.0.1rc10.tar.gz
Algorithm Hash digest
SHA256 d5bc6c20c9bd38c61a536077243ef9f760025b8b730fd37d7678f2524e2f47ab
MD5 1608bb20152b0071f87900ff50ebb9ae
BLAKE2b-256 0512258e75c7bef66c69ef7ffb9707dfdcc9786ba081c1478721143a458e8a2a

See more details on using hashes here.

File details

Details for the file rudolph-0.0.1rc10-py3-none-any.whl.

File metadata

  • Download URL: rudolph-0.0.1rc10-py3-none-any.whl
  • Upload date:
  • Size: 22.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for rudolph-0.0.1rc10-py3-none-any.whl
Algorithm Hash digest
SHA256 e861d9a4a9fe080a62b8d054e9fc4bde19a091a3f0844d91d200a513779aa658
MD5 873192501c9ee450fe4a468c5cf3fc50
BLAKE2b-256 d52e14860b23c36426cc18ca827fbe505b54ea171f3420dd8f82c6e8e6721ad4

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page