🧑🏫 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit), optimizers (adam, radam, adabelief), gans(dcgan, cyclegan, stylegan2), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, diffusion, etc. 🧠
Project description
labml.ai Deep Learning Paper Implementations
This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations,
The website renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better.
We are actively maintaining this repo and adding new implementations almost weekly. for updates.
Paper Implementations
✨ Transformers
- Multi-headed attention
- Transformer building blocks
- Transformer XL
- Rotary Positional Embeddings
- Attention with Linear Biases (ALiBi)
- RETRO
- Compressive Transformer
- GPT Architecture
- GLU Variants
- kNN-LM: Generalization through Memorization
- Feedback Transformer
- Switch Transformer
- Fast Weights Transformer
- FNet
- Attention Free Transformer
- Masked Language Model
- MLP-Mixer: An all-MLP Architecture for Vision
- Pay Attention to MLPs (gMLP)
- Vision Transformer (ViT)
- Primer EZ
- Hourglass
✨ Low-Rank Adaptation (LoRA)
✨ Eleuther GPT-NeoX
✨ Diffusion models
- Denoising Diffusion Probabilistic Models (DDPM)
- Denoising Diffusion Implicit Models (DDIM)
- Latent Diffusion Models
- Stable Diffusion
✨ Generative Adversarial Networks
- Original GAN
- GAN with deep convolutional network
- Cycle GAN
- Wasserstein GAN
- Wasserstein GAN with Gradient Penalty
- StyleGAN 2
✨ Recurrent Highway Networks
✨ LSTM
✨ HyperNetworks - HyperLSTM
✨ ResNet
✨ ConvMixer
✨ Capsule Networks
✨ U-Net
✨ Sketch RNN
✨ Graph Neural Networks
✨ Counterfactual Regret Minimization (CFR)
Solving games with incomplete information such as poker with CFR.
✨ Reinforcement Learning
- Proximal Policy Optimization with Generalized Advantage Estimation
- Deep Q Networks with with Dueling Network, Prioritized Replay and Double Q Network.
✨ Optimizers
- Adam
- AMSGrad
- Adam Optimizer with warmup
- Noam Optimizer
- Rectified Adam Optimizer
- AdaBelief Optimizer
- Sophia-G Optimizer
✨ Normalization Layers
- Batch Normalization
- Layer Normalization
- Instance Normalization
- Group Normalization
- Weight Standardization
- Batch-Channel Normalization
- DeepNorm
✨ Distillation
✨ Adaptive Computation
✨ Uncertainty
✨ Activations
✨ Langauge Model Sampling Techniques
✨ Scalable Training/Inference
Installation
pip install labml-nn
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file labml-nn-0.4.137.tar.gz
.
File metadata
- Download URL: labml-nn-0.4.137.tar.gz
- Upload date:
- Size: 316.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.11
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8369f2c129829e49bd14a0919260a6ea38453fa1caa5aaf991fa11c38bac482c |
|
MD5 | 491dfa2003b81ae1edd6e85d1681bb63 |
|
BLAKE2b-256 | f205b026334bfce4072b061bd7775bd7eefa1ab3386f116cd8ad9bec3ec7afa4 |
File details
Details for the file labml_nn-0.4.137-py3-none-any.whl
.
File metadata
- Download URL: labml_nn-0.4.137-py3-none-any.whl
- Upload date:
- Size: 443.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.11
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6f03605a939dfe162dbd50de984e6d84c52c9e72c29973594e571f5426b50100 |
|
MD5 | 3fd460d184ca7d171dbe88db8e4cfcfd |
|
BLAKE2b-256 | dd553e8e1415f07cab7ffcab38cece074c9026168721d52bda44c2dc9fec8f35 |