Skip to main content

dwn_cpu: Lightweight CPU Implementation of Differentiable Weightless Neural Networks

dwn_cpu is a lightweight C++/PyTorch implementation of Differentiable Weightless Neural Networks (DWN) for small CPU models.

It is intended primarily for the Weightless Neural Networks course at UFRJ and for experiments where the model is small enough that a fused CPU implementation can avoid much of the Python and PyTorch autograd overhead.

Training and inference are implemented in optimized C++ extensions and exposed through two main functions:

  • train_dwn
  • inference_dwn

Installation

pip install dwn_cpu

Requirements: PyTorch and a C++17-compatible compiler.

Quick Docs

train_dwn

train_dwn(
    x,                       # Binary input tensor [num_samples, input_size].
    y,                       # Integer class labels.
    n,                       # Number of inputs per LUT. LUT size is 2^n.
    tau,                     # GroupSum temperature.
    learning_rate,           # SGD learning rate.
    epochs,                  # Number of training epochs.
    num_luts,                # LUTs per layer. Int for 1 layer, list for multiple layers.
    num_classes,             # Number of output classes.
    batch_size,              # Training batch size.
    lr_decay_steps,          # Number of epochs between learning-rate decays.
    gamma,                   # Learning-rate decay factor.
    mapping="learnable",     # First-layer mapping: "learnable", "random", or "fixed".
    initial_luts=None,       # Optional initial LUT values.
    initial_mapping=None,    # Optional initial learnable weights or fixed mapping.
    verbose=True,            # Print training statistics once per epoch.
)

For a single-layer model:

num_luts = 40

For a multilayer model:

num_luts = [128, 64, 40]

Only the first layer can use learnable mapping in this implementation. Later layers always use fixed random mappings.

The final layer LUT count must be divisible by num_classes.

Returns:

binary_luts, mapping
  • binary_luts: Trained LUT values binarized to 0 or 1. For a single-layer model this is a tensor; for a multilayer model it is a list of tensors, one per layer.
  • mapping: Final discrete LUT input mappings. For a single-layer model this is a tensor; for a multilayer model it is a list of tensors, one per layer.

These outputs can be passed directly to inference_dwn.

inference_dwn

inference_dwn(
    x,                       # Binary input tensor [num_samples, input_size].
    binary_luts,             # Binarized LUTs returned by train_dwn.
    mapping,                 # Final discrete mappings returned by train_dwn.
    n,                       # Number of inputs per LUT.
    num_classes,             # Number of output classes.
    num_luts,                # LUT count, or list of LUT counts for multiple layers.
)

Returns:

predictions
  • predictions: Integer tensor with shape [num_samples] containing the predicted class index for each sample.

DistributiveThermometer

DistributiveThermometer(
    num_bits,                # Number of thermometer bits per input feature.
    feature_wise=True,       # Learn thresholds independently for each feature.
)

Fit the encoder on the training data:

thermometer = DistributiveThermometer(8).fit(x_train)

Binarize data:

x_train = thermometer.binarize(x_train)
x_test = thermometer.binarize(x_test)

For use with dwn_cpu, flatten the thermometer dimension and convert to uint8:

x_train = (
    thermometer.binarize(x_train)
    .flatten(start_dim=1)
    .to(torch.uint8)
    .contiguous()
)

x_test = (
    thermometer.binarize(x_test)
    .flatten(start_dim=1)
    .to(torch.uint8)
    .contiguous()
)

Methods:

  • .fit(x): Fits the thermometer thresholds from the training data and returns the fitted encoder.
  • .binarize(x): Converts the input features into their binary thermometer representation.

Returns from .binarize(x):

A binary PyTorch tensor containing num_bits thermometer bits for each original input feature.

Metadata

Release files for dwn-cpu 0.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dwn-cpu 0.6.0
File Size Uploaded Provenance Actions
dwn_cpu-0.6.0.tar.gz 20.2 kB Without provenance Details for dwn_cpu-0.6.0.tar.gz

Release files / dwn_cpu-0.6.0.tar.gz

Download URL dwn_cpu-0.6.0.tar.gz
Size 20.2 kB
Tags Source
SHA-256 checksum
How to use checksums
0e03e84262ed55d0eea6aa1594354100dc38921bc2271940505bc6cb2aa65cd9
BLAKE2b-256 checksum
How to use checksums
6290d419f27740379abeba515320da5c9fe3cac96b3faa275064cf491b5bcb70
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.2

Security for dwn-cpu 0.6.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.6.0 This release

1 release file No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page