sdhd 0.1.0

Stochastic-depth ResNet training library extracted from a thesis project

sdhd

sdhd stands for Stochastic Depth for High-Depth residual networks.

It is a reusable Python library extracted from my thesis codebase on stochastic-depth regularization for residual CNNs.

What the library provides

• ResNet models without stochastic depth;
• ResNet models with stochastic depth in two modes:
  • per-batch gating,
  • per-sample gating;
• CIFAR-style residual networks with 3x3 basic blocks;
• bottleneck residual networks (ResNet-50 / ResNet-101 style);
• folder-based image loading;
• CIFAR-100 loading;
• LMDB-backed datasets for large-scale training;
• a generic PyTorch training loop with checkpoint save/load support;
• a small command-line interface for training experiments.

Installation

Local editable install

pip install -e ./sdhd

With LMDB support

pip install -e './sdhd[lmdb]'

Quick examples

Build a CIFAR-style model

from sdhd import StochasticDepthConfig, CIFARResNetConfig, build_cifar_resnet

sd = StochasticDepthConfig(enabled=True, mode="sample", final_survival_prob=0.5)
config = CIFARResNetConfig(depth=110, num_classes=100, stochastic_depth=sd)
model = build_cifar_resnet(config)

Build a bottleneck ResNet

from sdhd import StochasticDepthConfig, BottleneckResNetConfig, build_bottleneck_resnet

sd = StochasticDepthConfig(enabled=True, mode="batch", final_survival_prob=0.5)
config = BottleneckResNetConfig(depth=50, num_classes=365, stochastic_depth=sd)
model = build_bottleneck_resnet(config)

Create CIFAR-100 dataloaders

from sdhd import create_cifar100_dataloaders

train_loader, test_loader = create_cifar100_dataloaders("/path/to/cifar100", batch_size=256)

Train a built-in ResNet in one function call

from sdhd import TrainerConfig, train_resnet

trainer = TrainerConfig(epochs=5, lr=0.01, output_dir="runs/example", compile_model=False)
result = train_resnet(
    architecture="bottleneck",
    dataset_type="folder",
    data_dir="/path/to/dataset",
    depth=50,
    batch_size=8,
    trainer_config=trainer,
    enable_sd=True,
    sd_mode="sample",
)
print(result.best_val_accuracy)

Train your own custom PyTorch model

import torch.nn as nn
from sdhd import TrainerConfig, train_custom_model

model = nn.Sequential(
    nn.Flatten(),
    nn.Linear(3 * 224 * 224, 102),
)

trainer = TrainerConfig(epochs=5, lr=0.01, output_dir="runs/custom", compile_model=False)
result = train_custom_model(
    model,
    dataset_type="folder",
    data_dir="/path/to/dataset",
    batch_size=8,
    trainer_config=trainer,
)
print(result.best_val_accuracy)

Lower-level training API

from sdhd import TrainerConfig, train_classifier

trainer = TrainerConfig(epochs=100, lr=0.1, output_dir="runs/example")
result = train_classifier(model, train_loader, test_loader, trainer)
print(result.best_val_accuracy)

CLI

After installation, you can run:

sdhd-train --help

Or:

python -m sdhd --help

Thesis background

This package comes from a thesis project on stochastic depth for regularizing residual CNNs. The main thesis-specific contributions preserved in this library are:

• explicit support for per-sample vs per-batch stochastic depth;
• stochastic depth for bottleneck residual blocks;
• support for large-scale LMDB datasets.

Full documentation

The package keeps its public-facing documentation concentrated in this README, in-code docstrings, and the printable LaTeX manual:

• docs/tex/sdhd_manual.tex

Additional private/helper Markdown notes for workspace management are intentionally kept outside the standalone sdhd repository.

Release and maintenance

Use GitHub Releases together with Trusted Publishing to publish to PyPI.

Recommendation for the LaTeX manual

Keep the LaTeX source in the repository and optionally publish the compiled PDF through GitHub Releases. PyPI should stay focused on the concise package README.