130 projects
dalle2-pytorch
DALL-E 2
dalle-pytorch
DALL-E - Pytorch
x-transformers
X-Transformers - Pytorch
denoising-diffusion-pytorch
Denoising Diffusion Probabilistic Models - Pytorch
video-diffusion-pytorch
Video Diffusion - Pytorch
compositional-attention-pytorch
Compositional Attention - Pytorch
anymal-belief-state-encoder-decoder-pytorch
Anymal Belief-state Encoder Decoder - Pytorch
CoCa-pytorch
CoCa, Contrastive Captioners are Image-Text Foundation Models - Pytorch
retro-pytorch
RETRO - Retrieval Enhanced Transformer - Pytorch
resize-right
Resize Right
vit-pytorch
Vision Transformer (ViT) - Pytorch
flamingo-pytorch
Flamingo - Pytorch
x-clip
X-CLIP
nuwa-pytorch
NÜWA - Pytorch
lightweight-gan
Lightweight GAN
memorizing-transformers-pytorch
Memorizing Transformer - Pytorch
PaLM-jax
PaLM: Scaling Language Modeling with Pathways - Jax
PaLM-pytorch
PaLM: Scaling Language Modeling with Pathways - Pytorch
uniformer-pytorch
Uniformer - Pytorch
deformable-attention
Deformable Attention - from the paper "Vision Transformer with Deformable Attention"
ddpm-proteins
Denoising Diffusion Probabilistic Models - for Proteins - Pytorch
protein-glm
Protein generative model with General Language Model PreTraining (GLM)
enformer-pytorch
Enformer - Pytorch
RQ-transformer
RQ Transformer - Autoregressive Transformer for Residual Quantized Codes
einops-exts
Einops Extensions
FLASH-pytorch
FLASH - Transformer Quality in Linear Time - Pytorch
rela-transformer
ReLA Transformer
byol-pytorch
Self-supervised contrastive learning made simple
triton-transformer
Transformer in Triton
ITTR-pytorch
ITTR - Implementation of the Hybrid Perception Block and Dual-Pruned Self-Attention block
rotary-embedding-torch
Rotary Embedding - Pytorch
tf-bind-transformer
Transformer for Transcription Factor Binding
logavgexp-pytorch
LogAvgExp - Pytorch
bidirectional-cross-attention
Bidirectional Cross Attention
ETSformer-pytorch
ETSTransformer - Exponential Smoothing Transformer for Time-Series Forecasting - Pytorch
memory-efficient-attention-pytorch
Memory Efficient Attention - Pytorch
nwt-pytorch
NWT - Pytorch
deep-daze
Deep Daze
vector-quantize-pytorch
Vector Quantization - Pytorch
En-transformer
E(n)-Equivariant Transformer
mlp-mixer-pytorch
MLP Mixer - Pytorch
point-transformer-pytorch
Point Transformer - Pytorch
electra-pytorch
Electra - Pytorch
performer-pytorch
Performer - Pytorch
mlm-pytorch
MLM (Masked Language Modeling) - Pytorch
big-sleep
Big Sleep
stylegan2-pytorch
StyleGan2 in Pytorch
perceiver-pytorch
Perceiver - Pytorch
graph-transformer-pytorch
Graph Transformer - Pytorch
transformer-in-transformer
Transformer in Transformer - Pytorch
mixture-of-experts
Sparsely-Gated Mixture of Experts for Pytorch
n-grammer-pytorch
N-Grammer - Pytorch
siren-pytorch
Implicit Neural Representations with Periodic Activation Functions
h-transformer-1d
H-Transformer 1D - Pytorch
hourglass-transformer-pytorch
Hourglass Transformer
routing-transformer
Routing Transformer (Pytorch)
reformer-pytorch
Reformer, the Efficient Transformer, Pytorch
ponder-transformer
Ponder Transformer - Pytorch
uformer-pytorch
Uformer - Pytorch
jax2torch
Jax 2 Torch
compressive-transformer-pytorch
Implementation of Compressive Transformer in Pytorch
remixer-pytorch
Remixer - Pytorch
bottleneck-transformer-pytorch
Bottleneck Transformer - Pytorch
se3-transformer-pytorch
SE3 Transformer - Pytorch
htm-pytorch
Hierarchical Transformer Memory - Pytorch
linear-attention-transformer
Linear Attention Transformer
tab-transformer-pytorch
Tab Transformer - Pytorch
alphafold2-pytorch
AlphaFold2 - Pytorch
tr-rosetta-pytorch
trRosetta - Pytorch
axial-attention
Axial Attention
fast-transformer-pytorch
Fast Transformer - Pytorch
timesformer-pytorch
TimeSformer - Pytorch
segformer-pytorch
Segformer - Pytorch
invariant-point-attention
Invariant Point Attention
token-shift-gpt
Token Shift GPT - Pytorch
progen-transformer
Protein Generation (ProGen)
g-mlp-pytorch
gMLP - Pytorch
protein-bert-pytorch
ProteinBERT - Pytorch
sinkhorn-transformer
Sinkhorn Transformer - Sparse Sinkhorn Attention
long-short-transformer
Long Short Transformer - Pytorch
triangle-multiplicative-module
Triangle Multiplicative Module
multistream-transformers
Multistream Transformers - Pytorch
charformer-pytorch
Charformer - Pytorch
local-attention
Local windowed attention, for language modeling
mlp-gpt-jax
MLP GPT - Jax
egnn-pytorch
E(n)-Equivariant Graph Neural Network - Pytorch
geometric-vector-perceptron
Geometric Vector Perceptron - Pytorch
res-mlp-pytorch
ResMLP - Pytorch
local-attention-flax
Local Attention - Flax Module in Jax
g-mlp-gpt
gMLP - GPT
poolformer
Poolformer
conformer
The convolutional module from the Conformer paper
transganformer
TransGanFormer
nystrom-attention
Nystrom Attention - Pytorch
stam-pytorch
Space Time Attention Model (STAM) - Pytorch
cross-transformers-pytorch
Cross Transformers - Pytorch
glom-pytorch
Glom - Pytorch
halonet-pytorch
HaloNet - Pytorch
omninet-pytorch
Omninet - Pytorch
contrastive-learner
Self-supervised contrastive learning made simple
coco-lm-pytorch
COCO - Pytorch
feedback-transformer-pytorch
Implementation of Feedback Transformer in Pytorch
pi-gan-pytorch
π-GAN - Pytorch
lie-transformer-pytorch
Lie Transformer - Pytorch
memory-compressed-attention
Memory-Compressed Self Attention
pixel-level-contrastive-learning
Pixel-Level Contrastive Learning
slot-attention
Implementation of Slot Attention in Pytorch
marge-pytorch
Marge - Pytorch
dalle-pytorch-dev
DALL-E - Pytorch
esbn-pytorch
Emergent Symbol Binding Network - Pytorch
adjacent-attention-pytorch
Adjacent Attention Network - Pytorch
isab-pytorch
Induced Set Attention Block - Pytorch
linformer
Linformer implementation in Pytorch
molecule-attention-transformer
Molecule Attention Transformer - Pytorch
gsa-pytorch
Global Self-attention Network (GSA) - Pytorch
lambda-networks
Lambda Networks - Pytorch
memformer
Memformer - Pytorch
hamburger-pytorch
Hamburger - Pytorch
unet-stylegan2
StyleGan2 with UNet Discriminator, in Pytorch
aoa-pytorch
Attention on Attention - Pytorch
deep-linear-network
Deep Linear Network - Pytorch
kronecker-attention-pytorch
Kronecker Attention - Pytorch
attention-tensorflow-mesh
A bunch of attention related functions, for constructing transformers in tensorflow mesh
memory-transformer-xl
Memory Transformer-XL, a variant of Transformer-XL that uses linear attention update long term memory
scattering-transform
Scattering Transform module from the paper Scattering Compositional Learner
relay-transformer
Relay Transformer, a long-range transformer
mogrifier
Implementation of Mogrifier circuit from Deepmind
axial-positional-embedding
Axial Positional Embedding
product-key-memory
Product Key Memory
linear-attention
Linear Attention Transformer