dwn_cpu: Lightweight CPU Implementation of Differentiable Weightless Neural Networks
dwn_cpu is a lightweight C++/PyTorch implementation of Differentiable Weightless Neural Networks (DWN) for small CPU models.
It is intended primarily for the Weightless Neural Networks course at UFRJ and for experiments where the model is small enough that a fused CPU implementation can avoid much of the Python and PyTorch autograd overhead.
Training and inference are implemented in optimized C++ extensions and exposed through two main functions:
train_dwninference_dwn
Installation
pip install dwn_cpu
Requirements: PyTorch and a C++17-compatible compiler.
Quick Docs
train_dwn
train_dwn(
x, # Binary input tensor [num_samples, input_size].
y, # Integer class labels.
n, # Number of inputs per LUT. LUT size is 2^n.
tau, # GroupSum temperature.
learning_rate, # SGD learning rate.
epochs, # Number of training epochs.
num_luts, # LUTs per layer. Int for 1 layer, list for multiple layers.
num_classes, # Number of output classes.
batch_size, # Training batch size.
lr_decay_steps, # Number of epochs between learning-rate decays.
gamma, # Learning-rate decay factor.
mapping="learnable", # First-layer mapping: "learnable", "random", or "fixed".
initial_luts=None, # Optional initial LUT values.
initial_mapping=None, # Optional initial learnable weights or fixed mapping.
verbose=True, # Print training statistics once per epoch.
)
For a single-layer model:
num_luts = 40
For a multilayer model:
num_luts = [128, 64, 40]
Only the first layer can use learnable mapping in this implementation. Later layers always use fixed random mappings.
The final layer LUT count must be divisible by num_classes.
Returns:
binary_luts, mapping
binary_luts: Trained LUT values binarized to0or1. For a single-layer model this is a tensor; for a multilayer model it is a list of tensors, one per layer.mapping: Final discrete LUT input mappings. For a single-layer model this is a tensor; for a multilayer model it is a list of tensors, one per layer.
These outputs can be passed directly to inference_dwn.
inference_dwn
inference_dwn(
x, # Binary input tensor [num_samples, input_size].
binary_luts, # Binarized LUTs returned by train_dwn.
mapping, # Final discrete mappings returned by train_dwn.
n, # Number of inputs per LUT.
num_classes, # Number of output classes.
num_luts, # LUT count, or list of LUT counts for multiple layers.
)
Returns:
predictions
predictions: Integer tensor with shape[num_samples]containing the predicted class index for each sample.
DistributiveThermometer
DistributiveThermometer(
num_bits, # Number of thermometer bits per input feature.
feature_wise=True, # Learn thresholds independently for each feature.
)
Fit the encoder on the training data:
thermometer = DistributiveThermometer(8).fit(x_train)
Binarize data:
x_train = thermometer.binarize(x_train)
x_test = thermometer.binarize(x_test)
For use with dwn_cpu, flatten the thermometer dimension and convert to uint8:
x_train = (
thermometer.binarize(x_train)
.flatten(start_dim=1)
.to(torch.uint8)
.contiguous()
)
x_test = (
thermometer.binarize(x_test)
.flatten(start_dim=1)
.to(torch.uint8)
.contiguous()
)
Methods:
.fit(x): Fits the thermometer thresholds from the training data and returns the fitted encoder..binarize(x): Converts the input features into their binary thermometer representation.
Returns from .binarize(x):
A binary PyTorch tensor containing num_bits thermometer bits for each original input feature.
Metadata
Release files for dwn-cpu 0.6.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| dwn_cpu-0.6.0.tar.gz | 20.2 kB | Without provenance | Details for dwn_cpu-0.6.0.tar.gz |
Release files / dwn_cpu-0.6.0.tar.gz
| Download URL | dwn_cpu-0.6.0.tar.gz |
|---|---|
| Size | 20.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
0e03e84262ed55d0eea6aa1594354100dc38921bc2271940505bc6cb2aa65cd9
|
|
BLAKE2b-256 checksum How to use checksums |
6290d419f27740379abeba515320da5c9fe3cac96b3faa275064cf491b5bcb70
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.2
|
Security for dwn-cpu 0.6.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.