Skip to main content
Avatar for Phil from gravatar.com

Phil

Username    lucidrains
Date joined   Joined

197 projects

denoising-diffusion-pytorch

Last released

Denoising Diffusion Probabilistic Models - Pytorch

ring-attention-pytorch

Last released

Ring Attention - Pytorch

self-rewarding-lm-pytorch

Last released

Self Rewarding LM - Pytorch

imagen-pytorch

Last released

Imagen - unprecedented photorealism × deep level of language understanding

lion-pytorch

Last released

Lion Optimizer - Pytorch

ema-pytorch

Last released

Easy way to keep track of exponential moving average version of your pytorch module

vector-quantize-pytorch

Last released

Vector Quantization - Pytorch

meshgpt-pytorch

Last released

MeshGPT Pytorch

soft-moe-pytorch

Last released

Soft MoE - Pytorch

st-moe-pytorch

Last released

ST - Mixture of Experts - Pytorch

x-transformers

Last released

X-Transformers - Pytorch

muse-maskgit-pytorch

Last released

MUSE - Text-to-Image Generation via Masked Generative Transformers, in Pytorch

soundstorm-pytorch

Last released

SoundStorm - Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

CoLT5-attention

Last released

Conditionally Routed Attention

lumiere-pytorch

Last released

Lumiere

quartic-transformer

Last released

Quartic Transformer

RIN-pytorch

Last released

RIN - Recurrent Interface Network - Pytorch

h-transformer-1d

Last released

H-Transformer 1D - Pytorch

recurrent-memory-transformer-pytorch

Last released

Recurrent Memory Transformer - Pytorch

q-transformer

Last released

Q-Transformer

pytorch-custom-utils

Last released

Pytorch Custom Utils

classifier-free-guidance-pytorch

Last released

Classifier Free Guidance - Pytorch

CALM-Pytorch

Last released

CALM - Pytorch

gateloop-transformer

Last released

GateLoop Transformer

voicebox-pytorch

Last released

Voicebox - Pytorch

iTransformer

Last released

iTransformer - Inverted Transformer Are Effective for Time Series Forecasting

audiolm-pytorch

Last released

AudioLM - Language Modeling Approach to Audio Generation from Google Research - Pytorch

gradnorm-pytorch

Last released

GradNorm - Pytorch

taylor-series-linear-attention

Last released

Taylor Series Linear Attention

nystrom-attention

Last released

Nystrom Attention - Pytorch

equiformer-pytorch

Last released

Equiformer - SE3/E3 Graph Attention Transformer for Molecules and Proteins

magvit2-pytorch

Last released

MagViT2 - Pytorch

En-transformer

Last released

E(n)-Equivariant Transformer

linformer

Last released

Linformer implementation in Pytorch

rotary-embedding-torch

Last released

Rotary Embedding - Pytorch

vit-pytorch

Last released

Vision Transformer (ViT) - Pytorch

agent-attention-pytorch

Last released

Agent Attention - Pytorch

mirasol-pytorch

Last released

Mirasol - Pytorch

toolformer-pytorch

Last released

Toolformer - Pytorch

enformer-pytorch

Last released

Enformer - Pytorch

BS-RoFormer

Last released

BS-RoFormer - Band-Split Rotary Transformer for SOTA Music Source Separation

bidirectional-cross-attention

Last released

Bidirectional Cross Attention

robotic-transformer-pytorch

Last released

Robotic Transformer - Pytorch

parti-pytorch

Last released

Parti - Pathways Autoregressive Text-to-Image Model - Pytorch

tab-transformer-pytorch

Last released

Tab Transformer - Pytorch

med-seg-diff-pytorch

Last released

MedSegDiff - SOTA medical image segmentation - Pytorch

phenaki-pytorch

Last released

Phenaki - Pytorch

simple-hierarchical-transformer

Last released

Simple Hierarchical Transformer

video-diffusion-pytorch

Last released

Video Diffusion - Pytorch

egnn-pytorch

Last released

E(n)-Equivariant Graph Neural Network - Pytorch

byol-pytorch

Last released

Self-supervised contrastive learning made simple

metnet3-pytorch

Last released

MetNet 3 - Pytorch

gigagan-pytorch

Last released

GigaGAN - Pytorch

spear-tts-pytorch

Last released

Spear-TTS - Pytorch

retro-pytorch

Last released

RETRO - Retrieval Enhanced Transformer - Pytorch

local-attention

Last released

Local attention, window with lookback, for language modeling

pause-transformer

Last released

Pause Transformer

zorro-pytorch

Last released

Zorro - Pytorch

dalle2-pytorch

Last released

DALL-E 2

x-clip

Last released

X-CLIP

bit-diffusion

Last released

Bit Diffusion - Pytorch

complex-valued-transformer

Last released

Complex Valued Transformer / Attention

MaMMUT-pytorch

Last released

MaMMUT - Pytorch

CoCa-pytorch

Last released

CoCa, Contrastive Captioners are Image-Text Foundation Models - Pytorch

speculative-decoding

Last released

Speculative Decoding

FLASH-pytorch

Last released

FLASH - Transformer Quality in Linear Time - Pytorch

naturalspeech2-pytorch

Last released

Natural Speech 2 - Pytorch

perfusion-pytorch

Last released

Perfusion - Pytorch

musiclm-pytorch

Last released

MusicLM - AudioLM + Audio CLIP to text to music synthesis

deformable-attention

Last released

Deformable Attention - from the paper "Vision Transformer with Deformable Attention"

product-key-memory

Last released

Product Key Memory

TPDNE-utils

Last released

TPDNE

ETSformer-pytorch

Last released

ETSTransformer - Exponential Smoothing Transformer for Time-Series Forecasting - Pytorch

Mega-pytorch

Last released

Mega - Pytorch

perceiver-pytorch

Last released

Perceiver - Pytorch

mixture-of-experts

Last released

Sparsely-Gated Mixture of Experts for Pytorch

VN-transformer

Last released

Vector Neuron Transformer (VN-Transformer)

siren-pytorch

Last released

Implicit Neural Representations with Periodic Activation Functions

flash-attention-jax

Last released

Flash Attention - in Jax

memory-efficient-attention-pytorch

Last released

Memory Efficient Attention - Pytorch

memorizing-transformers-pytorch

Last released

Memorizing Transformer - Pytorch

x-unet

Last released

X-Unet

discrete-key-value-bottleneck-pytorch

Last released

Discrete Key / Value Bottleneck - Pytorch

block-recurrent-transformer-pytorch

Last released

Block Recurrent Transformer - Pytorch

mixture-of-attention

Last released

Mixture of Attention

graph-transformer-pytorch

Last released

Graph Transformer - Pytorch

make-a-video-pytorch

Last released

Make-A-Video - Pytorch

MEGABYTE-pytorch

Last released

MEGABYTE - Pytorch

dalle-pytorch

Last released

DALL-E - Pytorch

coordinate-descent-attention

Last released

Coordinate Descent Attention - Pytorch

conformer

Last released

The convolutional module from the Conformer paper

rvq-vae-gpt

Last released

Yet another attempt at GPT in quantized latent space

memory-compressed-attention

Last released

Memory-Compressed Self Attention

perceiver-ar-pytorch

Last released

Perceiver AR

PaLM-rlhf-pytorch

Last released

PaLM + Reinforcement Learning with Human Feedback - Pytorch

gated-state-spaces-pytorch

Last released

Gated State Spaces - GSS - Pytorch

nuwa-pytorch

Last released

NÜWA - Pytorch

slot-attention

Last released

Implementation of Slot Attention in Pytorch

isab-pytorch

Last released

Induced Set Attention Block - Pytorch

einops-exts

Last released

Einops Extensions

adjacent-attention-pytorch

Last released

Adjacent Attention Network - Pytorch

n-grammer-pytorch

Last released

N-Grammer - Pytorch

se3-transformer-pytorch

Last released

SE3 Transformer - Pytorch

invariant-point-attention

Last released

Invariant Point Attention

flash-cosine-sim-attention

Last released

Flash Cosine Similarity Attention

flamingo-pytorch

Last released

Flamingo - Pytorch

adan-pytorch

Last released

Adan - (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch

lightweight-gan

Last released

Lightweight GAN

PaLM-pytorch

Last released

PaLM: Scaling Language Modeling with Pathways - Pytorch

stylegan2-pytorch

Last released

StyleGan2 in Pytorch

alphafold2-pytorch

Last released

AlphaFold2 - Pytorch

metaformer-gpt

Last released

Metaformer - GPT

tranception-pytorch

Last released

Tranception - Pytorch

PaLM-jax

Last released

PaLM: Scaling Language Modeling with Pathways - Jax

tf-bind-transformer

Last released

Transformer for Transcription Factor Binding

compositional-attention-pytorch

Last released

Compositional Attention - Pytorch

anymal-belief-state-encoder-decoder-pytorch

Last released

Anymal Belief-state Encoder Decoder - Pytorch

resize-right

Last released

Resize Right

uniformer-pytorch

Last released

Uniformer - Pytorch

ddpm-proteins

Last released

Denoising Diffusion Probabilistic Models - for Proteins - Pytorch

protein-glm

Last released

Protein generative model with General Language Model PreTraining (GLM)

RQ-transformer

Last released

RQ Transformer - Autoregressive Transformer for Residual Quantized Codes

rela-transformer

Last released

ReLA Transformer

triton-transformer

Last released

Transformer in Triton

ITTR-pytorch

Last released

ITTR - Implementation of the Hybrid Perception Block and Dual-Pruned Self-Attention block

logavgexp-pytorch

Last released

LogAvgExp - Pytorch

nwt-pytorch

Last released

NWT - Pytorch

deep-daze

Last released

Deep Daze

mlp-mixer-pytorch

Last released

MLP Mixer - Pytorch

point-transformer-pytorch

Last released

Point Transformer - Pytorch

electra-pytorch

Last released

Electra - Pytorch

performer-pytorch

Last released

Performer - Pytorch

mlm-pytorch

Last released

MLM (Masked Language Modeling) - Pytorch

big-sleep

Last released

Big Sleep

transformer-in-transformer

Last released

Transformer in Transformer - Pytorch

hourglass-transformer-pytorch

Last released

Hourglass Transformer

routing-transformer

Last released

Routing Transformer (Pytorch)

reformer-pytorch

Last released

Reformer, the Efficient Transformer, Pytorch

ponder-transformer

Last released

Ponder Transformer - Pytorch

uformer-pytorch

Last released

Uformer - Pytorch

jax2torch

Last released

Jax 2 Torch

compressive-transformer-pytorch

Last released

Implementation of Compressive Transformer in Pytorch

remixer-pytorch

Last released

Remixer - Pytorch

bottleneck-transformer-pytorch

Last released

Bottleneck Transformer - Pytorch

htm-pytorch

Last released

Hierarchical Transformer Memory - Pytorch

linear-attention-transformer

Last released

Linear Attention Transformer

tr-rosetta-pytorch

Last released

trRosetta - Pytorch

axial-attention

Last released

Axial Attention

fast-transformer-pytorch

Last released

Fast Transformer - Pytorch

timesformer-pytorch

Last released

TimeSformer - Pytorch

segformer-pytorch

Last released

Segformer - Pytorch

token-shift-gpt

Last released

Token Shift GPT - Pytorch

progen-transformer

Last released

Protein Generation (ProGen)

g-mlp-pytorch

Last released

gMLP - Pytorch

protein-bert-pytorch

Last released

ProteinBERT - Pytorch

sinkhorn-transformer

Last released

Sinkhorn Transformer - Sparse Sinkhorn Attention

long-short-transformer

Last released

Long Short Transformer - Pytorch

triangle-multiplicative-module

Last released

Triangle Multiplicative Module

multistream-transformers

Last released

Multistream Transformers - Pytorch

charformer-pytorch

Last released

Charformer - Pytorch

mlp-gpt-jax

Last released

MLP GPT - Jax

geometric-vector-perceptron

Last released

Geometric Vector Perceptron - Pytorch

res-mlp-pytorch

Last released

ResMLP - Pytorch

local-attention-flax

Last released

Local Attention - Flax Module in Jax

g-mlp-gpt

Last released

gMLP - GPT

poolformer

Last released

Poolformer

transganformer

Last released

TransGanFormer

stam-pytorch

Last released

Space Time Attention Model (STAM) - Pytorch

cross-transformers-pytorch

Last released

Cross Transformers - Pytorch

glom-pytorch

Last released

Glom - Pytorch

halonet-pytorch

Last released

HaloNet - Pytorch

omninet-pytorch

Last released

Omninet - Pytorch

contrastive-learner

Last released

Self-supervised contrastive learning made simple

coco-lm-pytorch

Last released

COCO - Pytorch

feedback-transformer-pytorch

Last released

Implementation of Feedback Transformer in Pytorch

pi-gan-pytorch

Last released

π-GAN - Pytorch

lie-transformer-pytorch

Last released

Lie Transformer - Pytorch

pixel-level-contrastive-learning

Last released

Pixel-Level Contrastive Learning

marge-pytorch

Last released

Marge - Pytorch

dalle-pytorch-dev

Last released

DALL-E - Pytorch

esbn-pytorch

Last released

Emergent Symbol Binding Network - Pytorch

molecule-attention-transformer

Last released

Molecule Attention Transformer - Pytorch

gsa-pytorch

Last released

Global Self-attention Network (GSA) - Pytorch

lambda-networks

Last released

Lambda Networks - Pytorch

memformer

Last released

Memformer - Pytorch

hamburger-pytorch

Last released

Hamburger - Pytorch

unet-stylegan2

Last released

StyleGan2 with UNet Discriminator, in Pytorch

aoa-pytorch

Last released

Attention on Attention - Pytorch

deep-linear-network

Last released

Deep Linear Network - Pytorch

kronecker-attention-pytorch

Last released

Kronecker Attention - Pytorch

attention-tensorflow-mesh

Last released

A bunch of attention related functions, for constructing transformers in tensorflow mesh

memory-transformer-xl

Last released

Memory Transformer-XL, a variant of Transformer-XL that uses linear attention update long term memory

scattering-transform

Last released

Scattering Transform module from the paper Scattering Compositional Learner

relay-transformer

Last released

Relay Transformer, a long-range transformer

mogrifier

Last released

Implementation of Mogrifier circuit from Deepmind

axial-positional-embedding

Last released

Axial Positional Embedding

linear-attention

Last released

Linear Attention Transformer

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page