274 projects
dreamer4
Dreamer 4
locoformer
LocoFormer
discrete-continuous-embed-readout
Discrete Continuous Embed Readout
value-network
Value networks
metacontroller-pytorch
Transformer Metacontroller
mimic-video
Mimic Video
RISE-pytorch
Implementation of RISE, Self-Improving Robot Policy with Compositional World Model
pi-zero-pytorch
π0 in Pytorch
rectified-flow-pytorch
Rectified Flow in Pytorch
memmap-replay-buffer
Simple Replay Buffer for RL
torch-einops-utils
Personal utility functions
vit-pytorch
Vision Transformer (ViT) - Pytorch
x-evolution
x-evolution
kl-div-attention
Attention with QK distance using KL divergence
vector-quantize-pytorch
Vector Quantization - Pytorch
x-transformers
X-Transformers
fast-weight-product-key-memory
Fast Weight Product Key Memory
denoising-diffusion-pytorch
Denoising Diffusion Probabilistic Models - Pytorch
titans-pytorch
Titans
h-net-dynamic-chunking
H-Net Dynamic Chunking Modules
hippoformer
hippoformer
sdft-pytorch
SDFT - Pytorch
hyper-connections
Hyper-Connections
BS-RoFormer
BS-RoFormer - Band-Split Rotary Transformer for SOTA Music Source Separation
contrastive-rl-pytorch
Contrastive RL
x-mlps-pytorch
A collection of MLPs / Feedforwards for Pytorch
PoPE-pytorch
PoPE
transfusion-pytorch
Transfusion in Pytorch
mmdit
MMDiT
discrete-distribution-network
Discrete Distribution Network
tab-transformer-pytorch
Tab Transformer - Pytorch
assoc-scan
Associative Scan
SAC-pytorch
Soft Actor Critic - Pytorch
tiny-recursive-model
Tiny Recursive Model
e2-tts-pytorch
E2-TTS in Pytorch
ema-pytorch
Easy way to keep track of exponential moving average version of your pytorch module
nim-mmcif
mmCIF parser written in Nim with Python bindings
transformer-lm-gan
Explorations into Transformer Language Model with Adversarial Loss
autoregressive-diffusion-pytorch
Autoregressive Diffusion - Pytorch
HS-TasNet
HS TasNet
hl-gauss-pytorch
HL Gauss - Pytorch
ultra-mem
UltraMem
PEER-pytorch
PEER - Pytorch
product-key-memory
Product Key Memory
DASH-pytorch
DASH
adam-atan2-pytorch
Adam-atan2 for Pytorch
SRT-H
SRT-H
x-transformers-rl
X-Transformer for RL
lookahead-keys-attention
Lookahead Keys Attention
TRI-LBM
Large Behavioral Model from Toyota Research
PaLM-rlhf-pytorch
PaLM + Reinforcement Learning with Human Feedback - Pytorch
alphafold3-pytorch
Alphafold 3 - Pytorch
simplicial-attention
(2) - Simplicial Attention
ViLLa-X
ViLLa-X
gradnorm-pytorch
GradNorm - Pytorch
amplify-pytorch
Amplify
native-sparse-attention-pytorch
Native Sparse Attention
rewind-reward-pytorch
Rewind Reward
Dex1B
MMDiT
HRM-pytorch
The proposal from a Singaporean AGI company
evolutionary-policy-optimization
EPO - Pytorch
bidirectional-cross-attention
Bidirectional Cross Attention
rotary-embedding-torch
Rotary Embedding - Pytorch
local-attention
Local attention, window with lookback, for language modeling
strassen-attention
Strassen Attention
enformer-pytorch
Enformer - Pytorch
mlp-mixer-pytorch
MLP Mixer - Pytorch
gaia2-pytorch
Gaia2 - Pytorch
iTransformer
iTransformer - Inverted Transformer Are Effective for Time Series Forecasting
blackbox-gradient-sensing
Blackbox Gradient Sensing
nGPT-pytorch
nGPT
q-transformer
Q-Transformer
transformer-directed-evolution
Directed Evolution with Transformer
ring-attention-pytorch
Ring Attention - Pytorch
HoST-pytorch
Humanoid Standing Up
improving-transformers-world-model
Improving Transformers World Model for RL
soundstorm-pytorch
SoundStorm - Efficient Parallel Audio Generation from Google Deepmind, in Pytorch
anymal-belief-state-encoder-decoder-pytorch
Anymal Belief-state Encoder Decoder - Pytorch
genetic-algorithm-pytorch
MMDiT
logavgexp-pytorch
LogAvgExp - Pytorch
gotennet-pytorch
GotenNet in Pytorch
soft-moe-pytorch
Soft MoE - Pytorch
quartic-transformer
Quartic Transformer
nystrom-attention
Nystrom Attention - Pytorch
minGRU-pytorch
minGRU
axial-positional-embedding
Axial Positional Embedding
lvsm-pytorch
LVSM - Pytorch
deep-cross-attention
Deep Cross Attention Language Model
deformable-attention
Deformable Attention - from the paper "Vision Transformer with Deformable Attention"
GAF-microbatch-pytorch
Gradient Agreement Filtering
audiolm-pytorch
AudioLM - Language Modeling Approach to Audio Generation from Google Research - Pytorch
gigagan-pytorch
GigaGAN - Pytorch
stylegan2-pytorch
StyleGan2 in Pytorch
lightweight-gan
Lightweight GAN
magvit2-pytorch
MagViT2 - Pytorch
genie2-pytorch
Genie2
recurrent-memory-transformer-pytorch
Recurrent Memory Transformer - Pytorch
infini-transformer-pytorch
Infini-Transformer in Pytorch
coconut-pytorch
Coconut in Pytorch
MEGABYTE-pytorch
MEGABYTE - Pytorch
meshgpt-pytorch
MeshGPT Pytorch
grokfast-pytorch
Grokfast
speculative-decoding
Speculative Decoding
egnn-pytorch
E(n)-Equivariant Graph Neural Network - Pytorch
lion-pytorch
Lion Optimizer - Pytorch
equiformer-pytorch
Equiformer - SE3/E3 Graph Attention Transformer for Molecules and Proteins
streaming-deep-rl
Streaming Deep Reinforcement Learning
maskbit-pytorch
MaskBit
spline-based-transformer
Spline Based Transformer
mixture-of-attention
Mixture of Attention
imagen-pytorch
Imagen - unprecedented photorealism × deep level of language understanding
robotic-transformer-pytorch
Robotic Transformer - Pytorch
classifier-free-guidance-pytorch
Classifier Free Guidance - Pytorch
scaling-vin-pytorch
Scaling Value Iteration Networks
CALM-Pytorch
CALM - Pytorch
CoLT5-attention
Conditionally Routed Attention
light-recurrent-unit-pytorch
Light Recurrent Unit
sinkhorn-router-pytorch
Sinkhorn Router - Pytorch
slot-attention
Implementation of Slot Attention in Pytorch
block-recurrent-transformer-pytorch
Block Recurrent Transformer - Pytorch
taylor-series-linear-attention
Taylor Series Linear Attention
phenaki-pytorch
Phenaki - Pytorch
pytorch-custom-utils
Pytorch Custom Utils
frame-averaging-pytorch
Frame Averaging
lumiere-pytorch
Lumiere
byol-pytorch
Self-supervised contrastive learning made simple
titok-pytorch
TiTok - Pytorch
gateloop-transformer
GateLoop Transformer
mogrifier
Implementation of Mogrifier circuit from Deepmind
st-moe-pytorch
ST - Mixture of Experts - Pytorch
En-transformer
E(n)-Equivariant Transformer
self-reasoning-tokens-pytorch
Self Reasoning Tokens
make-a-video-pytorch
Make-A-Video - Pytorch
x-unet
X-Unet
video-diffusion-pytorch
Video Diffusion - Pytorch
self-rewarding-lm-pytorch
Self Rewarding LM - Pytorch
muse-maskgit-pytorch
MUSE - Text-to-Image Generation via Masked Generative Transformers, in Pytorch
RIN-pytorch
RIN - Recurrent Interface Network - Pytorch
h-transformer-1d
H-Transformer 1D - Pytorch
voicebox-pytorch
Voicebox - Pytorch
linformer
Linformer implementation in Pytorch
agent-attention-pytorch
Agent Attention - Pytorch
mirasol-pytorch
Mirasol - Pytorch
toolformer-pytorch
Toolformer - Pytorch
parti-pytorch
Parti - Pathways Autoregressive Text-to-Image Model - Pytorch
med-seg-diff-pytorch
MedSegDiff - SOTA medical image segmentation - Pytorch
simple-hierarchical-transformer
Simple Hierarchical Transformer
metnet3-pytorch
MetNet 3 - Pytorch
spear-tts-pytorch
Spear-TTS - Pytorch
retro-pytorch
RETRO - Retrieval Enhanced Transformer - Pytorch
pause-transformer
Pause Transformer
zorro-pytorch
Zorro - Pytorch
dalle2-pytorch
DALL-E 2
x-clip
X-CLIP
bit-diffusion
Bit Diffusion - Pytorch
complex-valued-transformer
Complex Valued Transformer / Attention
MaMMUT-pytorch
MaMMUT - Pytorch
CoCa-pytorch
CoCa, Contrastive Captioners are Image-Text Foundation Models - Pytorch
FLASH-pytorch
FLASH - Transformer Quality in Linear Time - Pytorch
naturalspeech2-pytorch
Natural Speech 2 - Pytorch
perfusion-pytorch
Perfusion - Pytorch
musiclm-pytorch
MusicLM - AudioLM + Audio CLIP to text to music synthesis
TPDNE-utils
TPDNE
ETSformer-pytorch
ETSTransformer - Exponential Smoothing Transformer for Time-Series Forecasting - Pytorch
Mega-pytorch
Mega - Pytorch
perceiver-pytorch
Perceiver - Pytorch
mixture-of-experts
Sparsely-Gated Mixture of Experts for Pytorch
VN-transformer
Vector Neuron Transformer (VN-Transformer)
siren-pytorch
Implicit Neural Representations with Periodic Activation Functions
flash-attention-jax
Flash Attention - in Jax
memory-efficient-attention-pytorch
Memory Efficient Attention - Pytorch
memorizing-transformers-pytorch
Memorizing Transformer - Pytorch
discrete-key-value-bottleneck-pytorch
Discrete Key / Value Bottleneck - Pytorch
graph-transformer-pytorch
Graph Transformer - Pytorch
dalle-pytorch
DALL-E - Pytorch
coordinate-descent-attention
Coordinate Descent Attention - Pytorch
conformer
The convolutional module from the Conformer paper
rvq-vae-gpt
Yet another attempt at GPT in quantized latent space
memory-compressed-attention
Memory-Compressed Self Attention
perceiver-ar-pytorch
Perceiver AR
gated-state-spaces-pytorch
Gated State Spaces - GSS - Pytorch
nuwa-pytorch
NÜWA - Pytorch
isab-pytorch
Induced Set Attention Block - Pytorch
einops-exts
Einops Extensions
adjacent-attention-pytorch
Adjacent Attention Network - Pytorch
n-grammer-pytorch
N-Grammer - Pytorch
se3-transformer-pytorch
SE3 Transformer - Pytorch
invariant-point-attention
Invariant Point Attention
flash-cosine-sim-attention
Flash Cosine Similarity Attention
flamingo-pytorch
Flamingo - Pytorch
adan-pytorch
Adan - (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch
PaLM-pytorch
PaLM: Scaling Language Modeling with Pathways - Pytorch
alphafold2-pytorch
AlphaFold2 - Pytorch
metaformer-gpt
Metaformer - GPT
tranception-pytorch
Tranception - Pytorch
PaLM-jax
PaLM: Scaling Language Modeling with Pathways - Jax
tf-bind-transformer
Transformer for Transcription Factor Binding
compositional-attention-pytorch
Compositional Attention - Pytorch
resize-right
Resize Right
uniformer-pytorch
Uniformer - Pytorch
ddpm-proteins
Denoising Diffusion Probabilistic Models - for Proteins - Pytorch
protein-glm
Protein generative model with General Language Model PreTraining (GLM)
RQ-transformer
RQ Transformer - Autoregressive Transformer for Residual Quantized Codes
rela-transformer
ReLA Transformer
triton-transformer
Transformer in Triton
ITTR-pytorch
ITTR - Implementation of the Hybrid Perception Block and Dual-Pruned Self-Attention block
nwt-pytorch
NWT - Pytorch
deep-daze
Deep Daze
point-transformer-pytorch
Point Transformer - Pytorch
electra-pytorch
Electra - Pytorch
performer-pytorch
Performer - Pytorch
mlm-pytorch
MLM (Masked Language Modeling) - Pytorch
big-sleep
Big Sleep
transformer-in-transformer
Transformer in Transformer - Pytorch
hourglass-transformer-pytorch
Hourglass Transformer
routing-transformer
Routing Transformer (Pytorch)
reformer-pytorch
Reformer, the Efficient Transformer, Pytorch
ponder-transformer
Ponder Transformer - Pytorch
uformer-pytorch
Uformer - Pytorch
jax2torch
Jax 2 Torch
compressive-transformer-pytorch
Implementation of Compressive Transformer in Pytorch
remixer-pytorch
Remixer - Pytorch
bottleneck-transformer-pytorch
Bottleneck Transformer - Pytorch
htm-pytorch
Hierarchical Transformer Memory - Pytorch
linear-attention-transformer
Linear Attention Transformer
tr-rosetta-pytorch
trRosetta - Pytorch
axial-attention
Axial Attention
fast-transformer-pytorch
Fast Transformer - Pytorch
timesformer-pytorch
TimeSformer - Pytorch
segformer-pytorch
Segformer - Pytorch
token-shift-gpt
Token Shift GPT - Pytorch
progen-transformer
Protein Generation (ProGen)
g-mlp-pytorch
gMLP - Pytorch
protein-bert-pytorch
ProteinBERT - Pytorch
sinkhorn-transformer
Sinkhorn Transformer - Sparse Sinkhorn Attention
long-short-transformer
Long Short Transformer - Pytorch
triangle-multiplicative-module
Triangle Multiplicative Module
multistream-transformers
Multistream Transformers - Pytorch
charformer-pytorch
Charformer - Pytorch
mlp-gpt-jax
MLP GPT - Jax
geometric-vector-perceptron
Geometric Vector Perceptron - Pytorch
res-mlp-pytorch
ResMLP - Pytorch
local-attention-flax
Local Attention - Flax Module in Jax
g-mlp-gpt
gMLP - GPT
poolformer
Poolformer
transganformer
TransGanFormer
stam-pytorch
Space Time Attention Model (STAM) - Pytorch
cross-transformers-pytorch
Cross Transformers - Pytorch
glom-pytorch
Glom - Pytorch
halonet-pytorch
HaloNet - Pytorch
omninet-pytorch
Omninet - Pytorch
contrastive-learner
Self-supervised contrastive learning made simple
coco-lm-pytorch
COCO - Pytorch
feedback-transformer-pytorch
Implementation of Feedback Transformer in Pytorch
pi-gan-pytorch
π-GAN - Pytorch
lie-transformer-pytorch
Lie Transformer - Pytorch
pixel-level-contrastive-learning
Pixel-Level Contrastive Learning
marge-pytorch
Marge - Pytorch
dalle-pytorch-dev
DALL-E - Pytorch
esbn-pytorch
Emergent Symbol Binding Network - Pytorch
molecule-attention-transformer
Molecule Attention Transformer - Pytorch
gsa-pytorch
Global Self-attention Network (GSA) - Pytorch
lambda-networks
Lambda Networks - Pytorch
memformer
Memformer - Pytorch
hamburger-pytorch
Hamburger - Pytorch
unet-stylegan2
StyleGan2 with UNet Discriminator, in Pytorch
aoa-pytorch
Attention on Attention - Pytorch
deep-linear-network
Deep Linear Network - Pytorch
kronecker-attention-pytorch
Kronecker Attention - Pytorch
attention-tensorflow-mesh
A bunch of attention related functions, for constructing transformers in tensorflow mesh
memory-transformer-xl
Memory Transformer-XL, a variant of Transformer-XL that uses linear attention update long term memory
scattering-transform
Scattering Transform module from the paper Scattering Compositional Learner
relay-transformer
Relay Transformer, a long-range transformer
linear-attention
Linear Attention Transformer
