171 projects
x-transformers
X-Transformers - Pytorch
TPDNE-utils
TPDNE
vector-quantize-pytorch
Vector Quantization - Pytorch
audiolm-pytorch
AudioLM - Language Modeling Approach to Audio Generation from Google Research - Pytorch
MEGABYTE-pytorch
MEGABYTE - Pytorch
CoLT5-attention
Conditionally Routed Attention
enformer-pytorch
Enformer - Pytorch
naturalspeech2-pytorch
Natural Speech 2 - Pytorch
soundstorm-pytorch
SoundStorm - Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
recurrent-memory-transformer-pytorch
Recurrent Memory Transformer - Pytorch
mixture-of-attention
Mixture of Attention
dalle-pytorch
DALL-E - Pytorch
imagen-pytorch
Imagen - unprecedented photorealism × deep level of language understanding
parti-pytorch
Parti - Pathways Autoregressive Text-to-Image Model - Pytorch
coordinate-descent-attention
Coordinate Descent Attention - Pytorch
classifier-free-guidance-pytorch
Classifier Free Guidance - Pytorch
denoising-diffusion-pytorch
Denoising Diffusion Probabilistic Models - Pytorch
vit-pytorch
Vision Transformer (ViT) - Pytorch
conformer
The convolutional module from the Conformer paper
FLASH-pytorch
FLASH - Transformer Quality in Linear Time - Pytorch
MaMMUT-pytorch
MaMMUT - Pytorch
lion-pytorch
Lion Optimizer - Pytorch
tab-transformer-pytorch
Tab Transformer - Pytorch
rotary-embedding-torch
Rotary Embedding - Pytorch
local-attention
Local attention, window with lookback, for language modeling
gigagan-pytorch
GigaGAN - Pytorch
block-recurrent-transformer-pytorch
Block Recurrent Transformer - Pytorch
dalle2-pytorch
DALL-E 2
ema-pytorch
Easy way to keep track of exponential moving average version of your pytorch module
simple-hierarchical-transformer
Simple Hierarchical Transformer
rvq-vae-gpt
Yet another attempt at GPT in quantized latent space
memory-compressed-attention
Memory-Compressed Self Attention
muse-maskgit-pytorch
MUSE - Text-to-Image Generation via Masked Generative Transformers, in Pytorch
musiclm-pytorch
MusicLM - AudioLM + Audio CLIP to text to music synthesis
equiformer-pytorch
Equiformer - SE3/E3 Graph Attention Transformer for Molecules and Proteins
toolformer-pytorch
Toolformer - Pytorch
perceiver-ar-pytorch
Perceiver AR
PaLM-rlhf-pytorch
PaLM + Reinforcement Learning with Human Feedback - Pytorch
med-seg-diff-pytorch
MedSegDiff - SOTA medical image segmentation - Pytorch
memorizing-transformers-pytorch
Memorizing Transformer - Pytorch
phenaki-pytorch
Phenaki - Pytorch
RIN-pytorch
RIN - Recurrent Interface Network - Pytorch
make-a-video-pytorch
Make-A-Video - Pytorch
En-transformer
E(n)-Equivariant Transformer
robotic-transformer-pytorch
Robotic Transformer - Pytorch
memory-efficient-attention-pytorch
Memory Efficient Attention - Pytorch
gated-state-spaces-pytorch
Gated State Spaces - GSS - Pytorch
zorro-pytorch
Zorro - Pytorch
Mega-pytorch
Mega - Pytorch
x-clip
X-CLIP
bit-diffusion
Bit Diffusion - Pytorch
nuwa-pytorch
NÜWA - Pytorch
slot-attention
Implementation of Slot Attention in Pytorch
discrete-key-value-bottleneck-pytorch
Discrete Key / Value Bottleneck - Pytorch
isab-pytorch
Induced Set Attention Block - Pytorch
einops-exts
Einops Extensions
x-unet
X-Unet
perceiver-pytorch
Perceiver - Pytorch
adjacent-attention-pytorch
Adjacent Attention Network - Pytorch
CoCa-pytorch
CoCa, Contrastive Captioners are Image-Text Foundation Models - Pytorch
n-grammer-pytorch
N-Grammer - Pytorch
se3-transformer-pytorch
SE3 Transformer - Pytorch
invariant-point-attention
Invariant Point Attention
flash-cosine-sim-attention
Flash Cosine Similarity Attention
flash-attention-jax
Flash Attention - in Jax
flamingo-pytorch
Flamingo - Pytorch
adan-pytorch
Adan - (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch
lightweight-gan
Lightweight GAN
PaLM-pytorch
PaLM: Scaling Language Modeling with Pathways - Pytorch
video-diffusion-pytorch
Video Diffusion - Pytorch
stylegan2-pytorch
StyleGan2 in Pytorch
alphafold2-pytorch
AlphaFold2 - Pytorch
retro-pytorch
RETRO - Retrieval Enhanced Transformer - Pytorch
metaformer-gpt
Metaformer - GPT
tranception-pytorch
Tranception - Pytorch
PaLM-jax
PaLM: Scaling Language Modeling with Pathways - Jax
tf-bind-transformer
Transformer for Transcription Factor Binding
compositional-attention-pytorch
Compositional Attention - Pytorch
anymal-belief-state-encoder-decoder-pytorch
Anymal Belief-state Encoder Decoder - Pytorch
resize-right
Resize Right
uniformer-pytorch
Uniformer - Pytorch
deformable-attention
Deformable Attention - from the paper "Vision Transformer with Deformable Attention"
ddpm-proteins
Denoising Diffusion Probabilistic Models - for Proteins - Pytorch
protein-glm
Protein generative model with General Language Model PreTraining (GLM)
RQ-transformer
RQ Transformer - Autoregressive Transformer for Residual Quantized Codes
rela-transformer
ReLA Transformer
byol-pytorch
Self-supervised contrastive learning made simple
triton-transformer
Transformer in Triton
ITTR-pytorch
ITTR - Implementation of the Hybrid Perception Block and Dual-Pruned Self-Attention block
logavgexp-pytorch
LogAvgExp - Pytorch
bidirectional-cross-attention
Bidirectional Cross Attention
ETSformer-pytorch
ETSTransformer - Exponential Smoothing Transformer for Time-Series Forecasting - Pytorch
nwt-pytorch
NWT - Pytorch
deep-daze
Deep Daze
mlp-mixer-pytorch
MLP Mixer - Pytorch
point-transformer-pytorch
Point Transformer - Pytorch
electra-pytorch
Electra - Pytorch
performer-pytorch
Performer - Pytorch
mlm-pytorch
MLM (Masked Language Modeling) - Pytorch
big-sleep
Big Sleep
graph-transformer-pytorch
Graph Transformer - Pytorch
transformer-in-transformer
Transformer in Transformer - Pytorch
mixture-of-experts
Sparsely-Gated Mixture of Experts for Pytorch
siren-pytorch
Implicit Neural Representations with Periodic Activation Functions
h-transformer-1d
H-Transformer 1D - Pytorch
hourglass-transformer-pytorch
Hourglass Transformer
routing-transformer
Routing Transformer (Pytorch)
reformer-pytorch
Reformer, the Efficient Transformer, Pytorch
ponder-transformer
Ponder Transformer - Pytorch
uformer-pytorch
Uformer - Pytorch
jax2torch
Jax 2 Torch
compressive-transformer-pytorch
Implementation of Compressive Transformer in Pytorch
remixer-pytorch
Remixer - Pytorch
bottleneck-transformer-pytorch
Bottleneck Transformer - Pytorch
htm-pytorch
Hierarchical Transformer Memory - Pytorch
linear-attention-transformer
Linear Attention Transformer
tr-rosetta-pytorch
trRosetta - Pytorch
axial-attention
Axial Attention
fast-transformer-pytorch
Fast Transformer - Pytorch
timesformer-pytorch
TimeSformer - Pytorch
segformer-pytorch
Segformer - Pytorch
token-shift-gpt
Token Shift GPT - Pytorch
progen-transformer
Protein Generation (ProGen)
g-mlp-pytorch
gMLP - Pytorch
protein-bert-pytorch
ProteinBERT - Pytorch
sinkhorn-transformer
Sinkhorn Transformer - Sparse Sinkhorn Attention
long-short-transformer
Long Short Transformer - Pytorch
triangle-multiplicative-module
Triangle Multiplicative Module
multistream-transformers
Multistream Transformers - Pytorch
charformer-pytorch
Charformer - Pytorch
mlp-gpt-jax
MLP GPT - Jax
egnn-pytorch
E(n)-Equivariant Graph Neural Network - Pytorch
geometric-vector-perceptron
Geometric Vector Perceptron - Pytorch
res-mlp-pytorch
ResMLP - Pytorch
local-attention-flax
Local Attention - Flax Module in Jax
g-mlp-gpt
gMLP - GPT
poolformer
Poolformer
transganformer
TransGanFormer
nystrom-attention
Nystrom Attention - Pytorch
stam-pytorch
Space Time Attention Model (STAM) - Pytorch
cross-transformers-pytorch
Cross Transformers - Pytorch
glom-pytorch
Glom - Pytorch
halonet-pytorch
HaloNet - Pytorch
omninet-pytorch
Omninet - Pytorch
contrastive-learner
Self-supervised contrastive learning made simple
coco-lm-pytorch
COCO - Pytorch
feedback-transformer-pytorch
Implementation of Feedback Transformer in Pytorch
pi-gan-pytorch
π-GAN - Pytorch
lie-transformer-pytorch
Lie Transformer - Pytorch
pixel-level-contrastive-learning
Pixel-Level Contrastive Learning
marge-pytorch
Marge - Pytorch
dalle-pytorch-dev
DALL-E - Pytorch
esbn-pytorch
Emergent Symbol Binding Network - Pytorch
linformer
Linformer implementation in Pytorch
molecule-attention-transformer
Molecule Attention Transformer - Pytorch
gsa-pytorch
Global Self-attention Network (GSA) - Pytorch
lambda-networks
Lambda Networks - Pytorch
memformer
Memformer - Pytorch
hamburger-pytorch
Hamburger - Pytorch
unet-stylegan2
StyleGan2 with UNet Discriminator, in Pytorch
aoa-pytorch
Attention on Attention - Pytorch
deep-linear-network
Deep Linear Network - Pytorch
kronecker-attention-pytorch
Kronecker Attention - Pytorch
attention-tensorflow-mesh
A bunch of attention related functions, for constructing transformers in tensorflow mesh
memory-transformer-xl
Memory Transformer-XL, a variant of Transformer-XL that uses linear attention update long term memory
scattering-transform
Scattering Transform module from the paper Scattering Compositional Learner
relay-transformer
Relay Transformer, a long-range transformer
mogrifier
Implementation of Mogrifier circuit from Deepmind
axial-positional-embedding
Axial Positional Embedding
product-key-memory
Product Key Memory
linear-attention
Linear Attention Transformer