Skip to main content
Avatar for Phil from gravatar.com

Phil

Username    lucidrains
Date joined   Joined

171 projects

x-transformers

Last released

X-Transformers - Pytorch

TPDNE-utils

Last released

TPDNE

vector-quantize-pytorch

Last released

Vector Quantization - Pytorch

audiolm-pytorch

Last released

AudioLM - Language Modeling Approach to Audio Generation from Google Research - Pytorch

MEGABYTE-pytorch

Last released

MEGABYTE - Pytorch

CoLT5-attention

Last released

Conditionally Routed Attention

enformer-pytorch

Last released

Enformer - Pytorch

naturalspeech2-pytorch

Last released

Natural Speech 2 - Pytorch

soundstorm-pytorch

Last released

SoundStorm - Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

recurrent-memory-transformer-pytorch

Last released

Recurrent Memory Transformer - Pytorch

mixture-of-attention

Last released

Mixture of Attention

dalle-pytorch

Last released

DALL-E - Pytorch

imagen-pytorch

Last released

Imagen - unprecedented photorealism × deep level of language understanding

parti-pytorch

Last released

Parti - Pathways Autoregressive Text-to-Image Model - Pytorch

coordinate-descent-attention

Last released

Coordinate Descent Attention - Pytorch

classifier-free-guidance-pytorch

Last released

Classifier Free Guidance - Pytorch

denoising-diffusion-pytorch

Last released

Denoising Diffusion Probabilistic Models - Pytorch

vit-pytorch

Last released

Vision Transformer (ViT) - Pytorch

conformer

Last released

The convolutional module from the Conformer paper

FLASH-pytorch

Last released

FLASH - Transformer Quality in Linear Time - Pytorch

MaMMUT-pytorch

Last released

MaMMUT - Pytorch

lion-pytorch

Last released

Lion Optimizer - Pytorch

tab-transformer-pytorch

Last released

Tab Transformer - Pytorch

rotary-embedding-torch

Last released

Rotary Embedding - Pytorch

local-attention

Last released

Local attention, window with lookback, for language modeling

gigagan-pytorch

Last released

GigaGAN - Pytorch

block-recurrent-transformer-pytorch

Last released

Block Recurrent Transformer - Pytorch

dalle2-pytorch

Last released

DALL-E 2

ema-pytorch

Last released

Easy way to keep track of exponential moving average version of your pytorch module

simple-hierarchical-transformer

Last released

Simple Hierarchical Transformer

rvq-vae-gpt

Last released

Yet another attempt at GPT in quantized latent space

memory-compressed-attention

Last released

Memory-Compressed Self Attention

muse-maskgit-pytorch

Last released

MUSE - Text-to-Image Generation via Masked Generative Transformers, in Pytorch

musiclm-pytorch

Last released

MusicLM - AudioLM + Audio CLIP to text to music synthesis

equiformer-pytorch

Last released

Equiformer - SE3/E3 Graph Attention Transformer for Molecules and Proteins

toolformer-pytorch

Last released

Toolformer - Pytorch

perceiver-ar-pytorch

Last released

Perceiver AR

PaLM-rlhf-pytorch

Last released

PaLM + Reinforcement Learning with Human Feedback - Pytorch

med-seg-diff-pytorch

Last released

MedSegDiff - SOTA medical image segmentation - Pytorch

memorizing-transformers-pytorch

Last released

Memorizing Transformer - Pytorch

phenaki-pytorch

Last released

Phenaki - Pytorch

RIN-pytorch

Last released

RIN - Recurrent Interface Network - Pytorch

make-a-video-pytorch

Last released

Make-A-Video - Pytorch

En-transformer

Last released

E(n)-Equivariant Transformer

robotic-transformer-pytorch

Last released

Robotic Transformer - Pytorch

memory-efficient-attention-pytorch

Last released

Memory Efficient Attention - Pytorch

gated-state-spaces-pytorch

Last released

Gated State Spaces - GSS - Pytorch

zorro-pytorch

Last released

Zorro - Pytorch

Mega-pytorch

Last released

Mega - Pytorch

x-clip

Last released

X-CLIP

bit-diffusion

Last released

Bit Diffusion - Pytorch

nuwa-pytorch

Last released

NÜWA - Pytorch

slot-attention

Last released

Implementation of Slot Attention in Pytorch

discrete-key-value-bottleneck-pytorch

Last released

Discrete Key / Value Bottleneck - Pytorch

isab-pytorch

Last released

Induced Set Attention Block - Pytorch

einops-exts

Last released

Einops Extensions

x-unet

Last released

X-Unet

perceiver-pytorch

Last released

Perceiver - Pytorch

adjacent-attention-pytorch

Last released

Adjacent Attention Network - Pytorch

CoCa-pytorch

Last released

CoCa, Contrastive Captioners are Image-Text Foundation Models - Pytorch

n-grammer-pytorch

Last released

N-Grammer - Pytorch

se3-transformer-pytorch

Last released

SE3 Transformer - Pytorch

invariant-point-attention

Last released

Invariant Point Attention

flash-cosine-sim-attention

Last released

Flash Cosine Similarity Attention

flash-attention-jax

Last released

Flash Attention - in Jax

flamingo-pytorch

Last released

Flamingo - Pytorch

adan-pytorch

Last released

Adan - (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch

lightweight-gan

Last released

Lightweight GAN

PaLM-pytorch

Last released

PaLM: Scaling Language Modeling with Pathways - Pytorch

video-diffusion-pytorch

Last released

Video Diffusion - Pytorch

stylegan2-pytorch

Last released

StyleGan2 in Pytorch

alphafold2-pytorch

Last released

AlphaFold2 - Pytorch

retro-pytorch

Last released

RETRO - Retrieval Enhanced Transformer - Pytorch

metaformer-gpt

Last released

Metaformer - GPT

tranception-pytorch

Last released

Tranception - Pytorch

PaLM-jax

Last released

PaLM: Scaling Language Modeling with Pathways - Jax

tf-bind-transformer

Last released

Transformer for Transcription Factor Binding

compositional-attention-pytorch

Last released

Compositional Attention - Pytorch

anymal-belief-state-encoder-decoder-pytorch

Last released

Anymal Belief-state Encoder Decoder - Pytorch

resize-right

Last released

Resize Right

uniformer-pytorch

Last released

Uniformer - Pytorch

deformable-attention

Last released

Deformable Attention - from the paper "Vision Transformer with Deformable Attention"

ddpm-proteins

Last released

Denoising Diffusion Probabilistic Models - for Proteins - Pytorch

protein-glm

Last released

Protein generative model with General Language Model PreTraining (GLM)

RQ-transformer

Last released

RQ Transformer - Autoregressive Transformer for Residual Quantized Codes

rela-transformer

Last released

ReLA Transformer

byol-pytorch

Last released

Self-supervised contrastive learning made simple

triton-transformer

Last released

Transformer in Triton

ITTR-pytorch

Last released

ITTR - Implementation of the Hybrid Perception Block and Dual-Pruned Self-Attention block

logavgexp-pytorch

Last released

LogAvgExp - Pytorch

bidirectional-cross-attention

Last released

Bidirectional Cross Attention

ETSformer-pytorch

Last released

ETSTransformer - Exponential Smoothing Transformer for Time-Series Forecasting - Pytorch

nwt-pytorch

Last released

NWT - Pytorch

deep-daze

Last released

Deep Daze

mlp-mixer-pytorch

Last released

MLP Mixer - Pytorch

point-transformer-pytorch

Last released

Point Transformer - Pytorch

electra-pytorch

Last released

Electra - Pytorch

performer-pytorch

Last released

Performer - Pytorch

mlm-pytorch

Last released

MLM (Masked Language Modeling) - Pytorch

big-sleep

Last released

Big Sleep

graph-transformer-pytorch

Last released

Graph Transformer - Pytorch

transformer-in-transformer

Last released

Transformer in Transformer - Pytorch

mixture-of-experts

Last released

Sparsely-Gated Mixture of Experts for Pytorch

siren-pytorch

Last released

Implicit Neural Representations with Periodic Activation Functions

h-transformer-1d

Last released

H-Transformer 1D - Pytorch

hourglass-transformer-pytorch

Last released

Hourglass Transformer

routing-transformer

Last released

Routing Transformer (Pytorch)

reformer-pytorch

Last released

Reformer, the Efficient Transformer, Pytorch

ponder-transformer

Last released

Ponder Transformer - Pytorch

uformer-pytorch

Last released

Uformer - Pytorch

jax2torch

Last released

Jax 2 Torch

compressive-transformer-pytorch

Last released

Implementation of Compressive Transformer in Pytorch

remixer-pytorch

Last released

Remixer - Pytorch

bottleneck-transformer-pytorch

Last released

Bottleneck Transformer - Pytorch

htm-pytorch

Last released

Hierarchical Transformer Memory - Pytorch

linear-attention-transformer

Last released

Linear Attention Transformer

tr-rosetta-pytorch

Last released

trRosetta - Pytorch

axial-attention

Last released

Axial Attention

fast-transformer-pytorch

Last released

Fast Transformer - Pytorch

timesformer-pytorch

Last released

TimeSformer - Pytorch

segformer-pytorch

Last released

Segformer - Pytorch

token-shift-gpt

Last released

Token Shift GPT - Pytorch

progen-transformer

Last released

Protein Generation (ProGen)

g-mlp-pytorch

Last released

gMLP - Pytorch

protein-bert-pytorch

Last released

ProteinBERT - Pytorch

sinkhorn-transformer

Last released

Sinkhorn Transformer - Sparse Sinkhorn Attention

long-short-transformer

Last released

Long Short Transformer - Pytorch

triangle-multiplicative-module

Last released

Triangle Multiplicative Module

multistream-transformers

Last released

Multistream Transformers - Pytorch

charformer-pytorch

Last released

Charformer - Pytorch

mlp-gpt-jax

Last released

MLP GPT - Jax

egnn-pytorch

Last released

E(n)-Equivariant Graph Neural Network - Pytorch

geometric-vector-perceptron

Last released

Geometric Vector Perceptron - Pytorch

res-mlp-pytorch

Last released

ResMLP - Pytorch

local-attention-flax

Last released

Local Attention - Flax Module in Jax

g-mlp-gpt

Last released

gMLP - GPT

poolformer

Last released

Poolformer

transganformer

Last released

TransGanFormer

nystrom-attention

Last released

Nystrom Attention - Pytorch

stam-pytorch

Last released

Space Time Attention Model (STAM) - Pytorch

cross-transformers-pytorch

Last released

Cross Transformers - Pytorch

glom-pytorch

Last released

Glom - Pytorch

halonet-pytorch

Last released

HaloNet - Pytorch

omninet-pytorch

Last released

Omninet - Pytorch

contrastive-learner

Last released

Self-supervised contrastive learning made simple

coco-lm-pytorch

Last released

COCO - Pytorch

feedback-transformer-pytorch

Last released

Implementation of Feedback Transformer in Pytorch

pi-gan-pytorch

Last released

π-GAN - Pytorch

lie-transformer-pytorch

Last released

Lie Transformer - Pytorch

pixel-level-contrastive-learning

Last released

Pixel-Level Contrastive Learning

marge-pytorch

Last released

Marge - Pytorch

dalle-pytorch-dev

Last released

DALL-E - Pytorch

esbn-pytorch

Last released

Emergent Symbol Binding Network - Pytorch

linformer

Last released

Linformer implementation in Pytorch

molecule-attention-transformer

Last released

Molecule Attention Transformer - Pytorch

gsa-pytorch

Last released

Global Self-attention Network (GSA) - Pytorch

lambda-networks

Last released

Lambda Networks - Pytorch

memformer

Last released

Memformer - Pytorch

hamburger-pytorch

Last released

Hamburger - Pytorch

unet-stylegan2

Last released

StyleGan2 with UNet Discriminator, in Pytorch

aoa-pytorch

Last released

Attention on Attention - Pytorch

deep-linear-network

Last released

Deep Linear Network - Pytorch

kronecker-attention-pytorch

Last released

Kronecker Attention - Pytorch

attention-tensorflow-mesh

Last released

A bunch of attention related functions, for constructing transformers in tensorflow mesh

memory-transformer-xl

Last released

Memory Transformer-XL, a variant of Transformer-XL that uses linear attention update long term memory

scattering-transform

Last released

Scattering Transform module from the paper Scattering Compositional Learner

relay-transformer

Last released

Relay Transformer, a long-range transformer

mogrifier

Last released

Implementation of Mogrifier circuit from Deepmind

axial-positional-embedding

Last released

Axial Positional Embedding

product-key-memory

Last released

Product Key Memory

linear-attention

Last released

Linear Attention Transformer

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page