Skip to main content
Avatar for Phil from gravatar.com

Phil

Username    lucidrains
Date joined   Joined

130 projects

dalle2-pytorch

Last released

DALL-E 2

dalle-pytorch

Last released

DALL-E - Pytorch

x-transformers

Last released

X-Transformers - Pytorch

denoising-diffusion-pytorch

Last released

Denoising Diffusion Probabilistic Models - Pytorch

video-diffusion-pytorch

Last released

Video Diffusion - Pytorch

compositional-attention-pytorch

Last released

Compositional Attention - Pytorch

anymal-belief-state-encoder-decoder-pytorch

Last released

Anymal Belief-state Encoder Decoder - Pytorch

CoCa-pytorch

Last released

CoCa, Contrastive Captioners are Image-Text Foundation Models - Pytorch

retro-pytorch

Last released

RETRO - Retrieval Enhanced Transformer - Pytorch

resize-right

Last released

Resize Right

vit-pytorch

Last released

Vision Transformer (ViT) - Pytorch

flamingo-pytorch

Last released

Flamingo - Pytorch

x-clip

Last released

X-CLIP

nuwa-pytorch

Last released

NÜWA - Pytorch

lightweight-gan

Last released

Lightweight GAN

memorizing-transformers-pytorch

Last released

Memorizing Transformer - Pytorch

PaLM-jax

Last released

PaLM: Scaling Language Modeling with Pathways - Jax

PaLM-pytorch

Last released

PaLM: Scaling Language Modeling with Pathways - Pytorch

uniformer-pytorch

Last released

Uniformer - Pytorch

deformable-attention

Last released

Deformable Attention - from the paper "Vision Transformer with Deformable Attention"

ddpm-proteins

Last released

Denoising Diffusion Probabilistic Models - for Proteins - Pytorch

protein-glm

Last released

Protein generative model with General Language Model PreTraining (GLM)

enformer-pytorch

Last released

Enformer - Pytorch

RQ-transformer

Last released

RQ Transformer - Autoregressive Transformer for Residual Quantized Codes

einops-exts

Last released

Einops Extensions

FLASH-pytorch

Last released

FLASH - Transformer Quality in Linear Time - Pytorch

rela-transformer

Last released

ReLA Transformer

byol-pytorch

Last released

Self-supervised contrastive learning made simple

triton-transformer

Last released

Transformer in Triton

ITTR-pytorch

Last released

ITTR - Implementation of the Hybrid Perception Block and Dual-Pruned Self-Attention block

rotary-embedding-torch

Last released

Rotary Embedding - Pytorch

tf-bind-transformer

Last released

Transformer for Transcription Factor Binding

logavgexp-pytorch

Last released

LogAvgExp - Pytorch

bidirectional-cross-attention

Last released

Bidirectional Cross Attention

ETSformer-pytorch

Last released

ETSTransformer - Exponential Smoothing Transformer for Time-Series Forecasting - Pytorch

memory-efficient-attention-pytorch

Last released

Memory Efficient Attention - Pytorch

nwt-pytorch

Last released

NWT - Pytorch

deep-daze

Last released

Deep Daze

vector-quantize-pytorch

Last released

Vector Quantization - Pytorch

En-transformer

Last released

E(n)-Equivariant Transformer

mlp-mixer-pytorch

Last released

MLP Mixer - Pytorch

point-transformer-pytorch

Last released

Point Transformer - Pytorch

electra-pytorch

Last released

Electra - Pytorch

performer-pytorch

Last released

Performer - Pytorch

mlm-pytorch

Last released

MLM (Masked Language Modeling) - Pytorch

big-sleep

Last released

Big Sleep

stylegan2-pytorch

Last released

StyleGan2 in Pytorch

perceiver-pytorch

Last released

Perceiver - Pytorch

graph-transformer-pytorch

Last released

Graph Transformer - Pytorch

transformer-in-transformer

Last released

Transformer in Transformer - Pytorch

mixture-of-experts

Last released

Sparsely-Gated Mixture of Experts for Pytorch

n-grammer-pytorch

Last released

N-Grammer - Pytorch

siren-pytorch

Last released

Implicit Neural Representations with Periodic Activation Functions

h-transformer-1d

Last released

H-Transformer 1D - Pytorch

hourglass-transformer-pytorch

Last released

Hourglass Transformer

routing-transformer

Last released

Routing Transformer (Pytorch)

reformer-pytorch

Last released

Reformer, the Efficient Transformer, Pytorch

ponder-transformer

Last released

Ponder Transformer - Pytorch

uformer-pytorch

Last released

Uformer - Pytorch

jax2torch

Last released

Jax 2 Torch

compressive-transformer-pytorch

Last released

Implementation of Compressive Transformer in Pytorch

remixer-pytorch

Last released

Remixer - Pytorch

bottleneck-transformer-pytorch

Last released

Bottleneck Transformer - Pytorch

se3-transformer-pytorch

Last released

SE3 Transformer - Pytorch

htm-pytorch

Last released

Hierarchical Transformer Memory - Pytorch

linear-attention-transformer

Last released

Linear Attention Transformer

tab-transformer-pytorch

Last released

Tab Transformer - Pytorch

alphafold2-pytorch

Last released

AlphaFold2 - Pytorch

tr-rosetta-pytorch

Last released

trRosetta - Pytorch

axial-attention

Last released

Axial Attention

fast-transformer-pytorch

Last released

Fast Transformer - Pytorch

timesformer-pytorch

Last released

TimeSformer - Pytorch

segformer-pytorch

Last released

Segformer - Pytorch

invariant-point-attention

Last released

Invariant Point Attention

token-shift-gpt

Last released

Token Shift GPT - Pytorch

progen-transformer

Last released

Protein Generation (ProGen)

g-mlp-pytorch

Last released

gMLP - Pytorch

protein-bert-pytorch

Last released

ProteinBERT - Pytorch

sinkhorn-transformer

Last released

Sinkhorn Transformer - Sparse Sinkhorn Attention

long-short-transformer

Last released

Long Short Transformer - Pytorch

triangle-multiplicative-module

Last released

Triangle Multiplicative Module

multistream-transformers

Last released

Multistream Transformers - Pytorch

charformer-pytorch

Last released

Charformer - Pytorch

local-attention

Last released

Local windowed attention, for language modeling

mlp-gpt-jax

Last released

MLP GPT - Jax

egnn-pytorch

Last released

E(n)-Equivariant Graph Neural Network - Pytorch

geometric-vector-perceptron

Last released

Geometric Vector Perceptron - Pytorch

res-mlp-pytorch

Last released

ResMLP - Pytorch

local-attention-flax

Last released

Local Attention - Flax Module in Jax

g-mlp-gpt

Last released

gMLP - GPT

poolformer

Last released

Poolformer

conformer

Last released

The convolutional module from the Conformer paper

transganformer

Last released

TransGanFormer

nystrom-attention

Last released

Nystrom Attention - Pytorch

stam-pytorch

Last released

Space Time Attention Model (STAM) - Pytorch

cross-transformers-pytorch

Last released

Cross Transformers - Pytorch

glom-pytorch

Last released

Glom - Pytorch

halonet-pytorch

Last released

HaloNet - Pytorch

omninet-pytorch

Last released

Omninet - Pytorch

contrastive-learner

Last released

Self-supervised contrastive learning made simple

coco-lm-pytorch

Last released

COCO - Pytorch

feedback-transformer-pytorch

Last released

Implementation of Feedback Transformer in Pytorch

pi-gan-pytorch

Last released

π-GAN - Pytorch

lie-transformer-pytorch

Last released

Lie Transformer - Pytorch

memory-compressed-attention

Last released

Memory-Compressed Self Attention

pixel-level-contrastive-learning

Last released

Pixel-Level Contrastive Learning

slot-attention

Last released

Implementation of Slot Attention in Pytorch

marge-pytorch

Last released

Marge - Pytorch

dalle-pytorch-dev

Last released

DALL-E - Pytorch

esbn-pytorch

Last released

Emergent Symbol Binding Network - Pytorch

adjacent-attention-pytorch

Last released

Adjacent Attention Network - Pytorch

isab-pytorch

Last released

Induced Set Attention Block - Pytorch

linformer

Last released

Linformer implementation in Pytorch

molecule-attention-transformer

Last released

Molecule Attention Transformer - Pytorch

gsa-pytorch

Last released

Global Self-attention Network (GSA) - Pytorch

lambda-networks

Last released

Lambda Networks - Pytorch

memformer

Last released

Memformer - Pytorch

hamburger-pytorch

Last released

Hamburger - Pytorch

unet-stylegan2

Last released

StyleGan2 with UNet Discriminator, in Pytorch

aoa-pytorch

Last released

Attention on Attention - Pytorch

deep-linear-network

Last released

Deep Linear Network - Pytorch

kronecker-attention-pytorch

Last released

Kronecker Attention - Pytorch

attention-tensorflow-mesh

Last released

A bunch of attention related functions, for constructing transformers in tensorflow mesh

memory-transformer-xl

Last released

Memory Transformer-XL, a variant of Transformer-XL that uses linear attention update long term memory

scattering-transform

Last released

Scattering Transform module from the paper Scattering Compositional Learner

relay-transformer

Last released

Relay Transformer, a long-range transformer

mogrifier

Last released

Implementation of Mogrifier circuit from Deepmind

axial-positional-embedding

Last released

Axial Positional Embedding

product-key-memory

Last released

Product Key Memory

linear-attention

Last released

Linear Attention Transformer

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page