labml.ai Deep Learning Paper Implementations
This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations,
The website renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better.
We are actively maintaining this repo and adding new
implementations almost weekly.
Paper Implementations
✨ Transformers
- Multi-headed attention
- Triton Flash Attention
- Transformer building blocks
- Transformer XL
- Rotary Positional Embeddings
- Attention with Linear Biases (ALiBi)
- RETRO
- Compressive Transformer
- GPT Architecture
- GLU Variants
- kNN-LM: Generalization through Memorization
- Feedback Transformer
- Switch Transformer
- Fast Weights Transformer
- FNet
- Attention Free Transformer
- Masked Language Model
- MLP-Mixer: An all-MLP Architecture for Vision
- Pay Attention to MLPs (gMLP)
- Vision Transformer (ViT)
- Primer EZ
- Hourglass
✨ Low-Rank Adaptation (LoRA)
✨ Eleuther GPT-NeoX
✨ Diffusion models
- Denoising Diffusion Probabilistic Models (DDPM)
- Denoising Diffusion Implicit Models (DDIM)
- Latent Diffusion Models
- Stable Diffusion
✨ Generative Adversarial Networks
- Original GAN
- GAN with deep convolutional network
- Cycle GAN
- Wasserstein GAN
- Wasserstein GAN with Gradient Penalty
- StyleGAN 2
✨ Recurrent Highway Networks
✨ LSTM
✨ HyperNetworks - HyperLSTM
✨ ResNet
✨ ConvMixer
✨ Capsule Networks
✨ U-Net
✨ Sketch RNN
✨ Graph Neural Networks
✨ Counterfactual Regret Minimization (CFR)
Solving games with incomplete information such as poker with CFR.
✨ Reinforcement Learning
- Proximal Policy Optimization with Generalized Advantage Estimation
- Deep Q Networks with with Dueling Network, Prioritized Replay and Double Q Network.
✨ Optimizers
- Adam
- AMSGrad
- Adam Optimizer with warmup
- Noam Optimizer
- Rectified Adam Optimizer
- AdaBelief Optimizer
- Sophia-G Optimizer
✨ Normalization Layers
- Batch Normalization
- Layer Normalization
- Instance Normalization
- Group Normalization
- Weight Standardization
- Batch-Channel Normalization
- DeepNorm
✨ Distillation
✨ Adaptive Computation
✨ Uncertainty
✨ Activations
✨ Langauge Model Sampling Techniques
✨ Scalable Training/Inference
Installation
pip install labml-nn
1 maintainer
Release files for labml-nn 0.5.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| labml_nn-0.5.1.tar.gz | 334.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| labml_nn-0.5.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 796.1 kB
Release files / labml_nn-0.5.1.tar.gz
| Download URL | labml_nn-0.5.1.tar.gz |
|---|---|
| Size | 334.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
14a23a126e3da62ddb38a2a4e3fd082581b42ab4b82f270882e0a6378be1546b
|
|
BLAKE2b-256 checksum How to use checksums |
07e64f9eeb84fb5a37d5efa7ae637e337f12e3e59acf3359ea2d0985dc874740
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.10.11
|
Release files / labml_nn-0.5.1-py3-none-any.whl
| Download URL | labml_nn-0.5.1-py3-none-any.whl |
|---|---|
| Size | 461.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ba6b6d4efb2590636f237aa6204ffe789b3ec484088226248890393e225a5761
|
|
BLAKE2b-256 checksum How to use checksums |
a1ed058a18295819037c4ae8ad2b273aa287d4afacc324a883e3ed95ae890bfc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.10.11
|
