Skip to main content

EDEN compression written in PyTorch

Project description

EDEN compression

PyPI Changelog License

A PyTorch implementation of 'EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning', presented at ICML 2022.

Installation

Install this library using pip:

pip install cmprs-eden

Usage

from cmprs_eden import eden
import torch

from dataclasses import astuple

compression = eden(bits=1)

x = torch.randn(2 ** 12)

compressed_x, context, bitrate = astuple(compression.forward(x))
reconstructed_x = compression.backward(compressed_x, context)

Development

To contribute to this library, first checkout the code. Then create a new virtual environment:

cd <project-checkout-folder>
python -m venv venv
source venv/bin/activate

Now install the development dependencies:

pip install -e ".[test,preprocess]"

To run the tests:

pytest

Citation

If you find this useful, please cite us:

@InProceedings{pmlr-v162-vargaftik22a,
  title = 	 {{EDEN}: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learning},
  author =       {Vargaftik, Shay and Basat, Ran Ben and Portnoy, Amit and Mendelson, Gal and Itzhak, Yaniv Ben and Mitzenmacher, Michael},
  booktitle = 	 {Proceedings of the 39th International Conference on Machine Learning},
  pages = 	 {21984--22014},
  year = 	 {2022},
  editor = 	 {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
  volume = 	 {162},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {17--23 Jul},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v162/vargaftik22a/vargaftik22a.pdf},
  url = 	 {https://proceedings.mlr.press/v162/vargaftik22a.html},
  abstract = 	 {Distributed Mean Estimation (DME) is a central building block in federated learning, where clients send local gradients to a parameter server for averaging and updating the model. Due to communication constraints, clients often use lossy compression techniques to compress the gradients, resulting in estimation inaccuracies. DME is more challenging when clients have diverse network conditions, such as constrained communication budgets and packet losses. In such settings, DME techniques often incur a significant increase in the estimation error leading to degraded learning performance. In this work, we propose a robust DME technique named EDEN that naturally handles heterogeneous communication budgets and packet losses. We derive appealing theoretical guarantees for EDEN and evaluate it empirically. Our results demonstrate that EDEN consistently improves over state-of-the-art DME techniques.}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cmprs-eden-0.1a3.tar.gz (16.7 kB view details)

Uploaded Source

Built Distribution

cmprs_eden-0.1a3-py3-none-any.whl (20.4 kB view details)

Uploaded Python 3

File details

Details for the file cmprs-eden-0.1a3.tar.gz.

File metadata

  • Download URL: cmprs-eden-0.1a3.tar.gz
  • Upload date:
  • Size: 16.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for cmprs-eden-0.1a3.tar.gz
Algorithm Hash digest
SHA256 6230196b4e448bd2c48822d2939b8d665a1899229ea005c77380bf7a26be4337
MD5 c8676513d85d5be90afe42aaa4e8cc85
BLAKE2b-256 65294ea3f3a75f7307fdcdf19f8d3673a30618167786c8441085918dac7841c2

See more details on using hashes here.

File details

Details for the file cmprs_eden-0.1a3-py3-none-any.whl.

File metadata

  • Download URL: cmprs_eden-0.1a3-py3-none-any.whl
  • Upload date:
  • Size: 20.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for cmprs_eden-0.1a3-py3-none-any.whl
Algorithm Hash digest
SHA256 66a8983bcc81bfbace9d0baca9fdd542f9ea32459da98232d5d4d37ac46a647b
MD5 5fd7a50dcd9a17bdf52b794caa55a22a
BLAKE2b-256 6d12a401b6467476787b4557544317792ad572ea19d6e0601bca427b2bc4c094

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page