deep-learning-toolkit 0.4.0

Deep Learning Toolkit — Reusable PyTorch building blocks for artificial intelligence & scientific machine learning: networks, losses, training loops, and utilities.

Deep Learning Toolkit

Reusable PyTorch building blocks for artificial intelligence & scientific machine learning: networks, losses, training loops, and utilities.

---

Installing the deep-learning-toolkit

Requirements

• Python >=3.11

Runtime dependencies

• prettytable>=3,<4
• pyyaml>=6,<7
• torch>=2,<3
• tqdm>=4,<5

Install in regular mode

pip install deep-learning-toolkit

Install the package in editable mode

When using a clone of the Git repository, run this command from inside the cloned directory:

pip install -e .

Install with optional extras

Dependencies for kernel density estimation:

pip install -e ".[kde]"

Dependencies for running tests:

pip install -e ".[test]"

Install with optional extras for development

pip install -e ".[dev]"

Using [dev] includes extras from [test].

---

Importing and using dlk

To use the toolkit, import its modules in your Python code like this:

from dlk.nets.mlp import MLPNet
from dlk.opt.train import train_epochs

# load your data
...

# create the model
net = MLPNet(input_size=784, output_size=10)

# train the model
train_epochs(n_epochs=100, net=net, dataloader=..., optimizer=..., loss_fn=...)

# evaluate
...

---

Architecture

Neural network architectures → dlk/nets/

• mlp.py: Multilayer Perceptron (MLPNet, MLPNet_MultIn, MLPResNet with residual and attention blocks)
• autoencoder.py: Generic autoencoder wrapper for encoder/decoder pairs
• conv1d.py, conv2d.py: 1D/2D convolutional networks and UNet components (Downsample, Upsample)
• unet.py: Complete UNet implementations (older UNet1D/UNet2D and newer UNetXd_2025 architecture)
• transformer1d.py: 1D transformer networks with patch embeddings and multi-head attention
• efficientnet.py: EfficientNet architecture

Network initialization

All network modules follow a consistent pattern:

• Constructor calls self.init_parameters() at the end
• init_parameters() uses Xavier initialization with gain calculated from activation functions
• Utility functions _get_gain() and _set_init_parameters() handle activation-aware initialization

Training and optimization → dlk/opt/

• train.py: Training loops (train_epochs, train_batches) with checkpointing and validation hooks
• train_gan.py: GAN-specific training loops
• scheduler.py: Learning rate schedulers (multi-stage: linear warmup, constant, cosine annealing)

Logging of the training progress

Training functions return detailed logging dictionaries (dlog) containing:

• Per-epoch loss statistics (loss_mean, loss_std)
• Batch-level logs nested in batch_dlog
• Total training time in time_train
• Checkpointing saves model and optimizer states at specified intervals

Additional components of the package

• dlk/mgmt/: Management of configuration parameter loading/saving, logging, etc.
• dlk/loss/: Loss functions
• dlk/metrics/: Metrics for evaluating trained nets

---

Development

Commands for development

• make format: run isort and black on dlk/ and tests/
• make format-check: check isort and black formatting without modifying files
• make compile: run python -m compileall -q -f on dlk/ and tests/
• make lint: run basedpyright on dlk/ and tests/
• make test: run pytest (after make compile)
• make testq: run pytest -q (after make compile)
• make testv: run pytest -v (after make compile)
• make testvv: run pytest -sv (after make compile)