Skip to main content

Bregman Neural Networks

Project description

BregmaNet : Bregman Neural Networks

License: MIT

BregmaNet is a PyTorch library providing multiple Bregman Neural Networks. To date, implemented models cover Bregman variants of multi-layer perceptrons and various residual networks.

Contributor: Jordan Frécon (INSA Rouen Normandy, France)

Table of Contents

  1. Requirements and Installation
  2. Getting Started
  3. List of Supported Models
  4. Citation
  5. Contribution and Acknowledgments

Requirements and Installation

:clipboard: Requirements

  • PyTorch version >=1.4.0
  • Python version >=3.6
  • Torchvision version >=0.8.2

:hammer: Installation

pip install bregmanet

In development versions can be found here.

Getting Started

:warning: Precautions

  • All images should be scaled within the domain range of the activation function.
  • MLP models provided work only for 1-dimensional data inputs.
  • MLP models are designed without a softmax final layer.
  • All models need to be trained first. If you wish to provide your pretrained models, please contribute.

:rocket: Demos

Multiple demo files can be found there in the demos folder. It contains:

  • demo_toy_mlp.py: training of MLP on the Two-spiral toy dataset.
  • demo_mnist_mlp.py: training of MLP on the MNIST dataset.
  • demo_cifar10_resnet.py: training of ResNet20 on the CIFAR-10 dataset.
  • demo_cifar100_resnet.py: training of ResNet20 on the CIFAR-100 dataset.
  • demo_imagenet_resnet.py: training of ResNet18 on the ImageNet dataset.

:page_with_curl: Loading a Model

To date, all Bregman neural models provided are not trained. If needed, a training procedure is made available there in the demos/utils folder. In order to load a model, proceed as follows.

Multi-Layer Perceptrons

For a sigmoid-based MLP with

  • a linear input accepting 1d tensors of size 1024
  • 3 hidden layers of size (1024, 1024, 512)
  • a linear output layer mapping to 1d tensors of size 10
import bregmanet
model = bregmanet.MLP(activation='sigmoid', num_neurons=[1024, 1024, 512], input_dim=1024, output_dim=10)

ResNet

For a BregmanResNet20 with SoftPlus activation function:

import bregmanet
model = bregmanet.bresnet20(activation='softplus')

List of Supported Models

The following list reports all models currently supporting a Bregman variant. If you have any issue with one of them or wish to provide your own, please contact us.

  • MLP
  • ResNet18
  • ResNet20
  • ResNet32
  • ResNet34
  • ResNet44
  • ResNet56
  • ResNet101
  • Resnet110
  • ResNet152
  • Resnet1202
  • ResNeXt50_32x4d
  • ResNeXt101_32x8d
  • WideResNet50_2
  • WideResnet101_2

Citation

If you use this package, please cite the following work:

@inproceedings{2022_Frecon_J_p-icml_bregmanet,
  title = {{Bregman Neural Networks}},
  author = {Frecon, Jordan and Gasso, Gilles and Pontil, Massimiliano and Salzo, Saverio},
  url = {https://hal.archives-ouvertes.fr/hal-03132512},
  series    = {Proceedings of Machine Learning Research},
  booktitle = {Proceedings of the 39th International Conference on Machine Learning,
               {ICML} 2022, 17-23 July 2022, Baltimore, USA},
  year = {2022},
}

Contribution and Acknowledgments

The proposed BregmanResNets for CIFAR-10 are based on a rework of the ResNet implementation of Yerlan Idelbayev. Other models are devised by hinging upon the official PyTorch/TorchVision repository. For more information, please refer to:

All kind of contributions are welcome, do not hesitate to contact us!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bregmanet-0.1.1.tar.gz (14.2 kB view details)

Uploaded Source

Built Distribution

bregmanet-0.1.1-py3-none-any.whl (14.3 kB view details)

Uploaded Python 3

File details

Details for the file bregmanet-0.1.1.tar.gz.

File metadata

  • Download URL: bregmanet-0.1.1.tar.gz
  • Upload date:
  • Size: 14.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.8.5

File hashes

Hashes for bregmanet-0.1.1.tar.gz
Algorithm Hash digest
SHA256 674f58648a7bd75db4c67ad6dd3904aca18d7cbc09cb3e5196b8401be3800e8c
MD5 b4e2f37deff87ecf7e179a3124448286
BLAKE2b-256 b22b17d3a15229c35b06f4e6f103273dec6b65b228913a69a5607ff5375b7e97

See more details on using hashes here.

File details

Details for the file bregmanet-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: bregmanet-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 14.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.8.5

File hashes

Hashes for bregmanet-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 aa710149b418ab658756425a6c820fce755493d940acae88003210be4641d566
MD5 68c509f427db93fdd56b99b0cc00351f
BLAKE2b-256 9c3cce28051937c671bfa4c9d5d018160d73460ab5f7032983529dec3dde3040

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page