Zero-config distributed ML training for CPU clusters
Project description
ColonyML
Zero-config distributed ML training for CPU clusters.
Pool any laptops into a training cluster — no IP addresses, no config files, no GPU required. Just:
pip install colonyml
colonyml join
The Problem
Training ML models on one laptop is slow. You might have multiple machines sitting idle. Existing distributed training tools require manual IP configuration, Linux, or expensive GPUs.
ColonyML fixes this. Any laptop. Any OS. Zero setup.
How It Works
Laptop 1 → colonyml join (announces itself on network)
Laptop 2 → colonyml join (finds Laptop 1 automatically)
Both laptops train together:
- Auto-discovery via mDNS (like AirDrop for ML)
- Ring AllReduce for gradient synchronization
- Adaptive batch sizing based on CPU power
- Gradient compression over WiFi (up to 50x)
Features
- Zero config — no IP addresses, no config files
- Auto-discovery — machines find each other via mDNS
- Ring AllReduce — efficient gradient averaging across nodes
- Adaptive scheduling — faster machines get bigger batches
- Gradient compression — up to 50x smaller over WiFi
- Any hardware — works on any CPU laptop, any OS
Installation
pip install colonyml
Status
Active Development — v0.1.0 is the initial release. Core features being built now. Star the repo to follow progress.
Roadmap
- v0.1.0 — PyPI release, project structure
- v0.2.0 — mDNS auto-discovery between machines
- v0.3.0 — gradient compression
- v0.4.0 — adaptive CPU-aware batch scheduler
- v0.5.0 — Ring AllReduce gradient synchronization
- v1.0.0 — stable release with full test coverage
Author
Jeevana Sai Gogineni MS Data Science · University of Maryland · GPA 4.0
GitHub: https://github.com/gjeevana27 LinkedIn: https://linkedin.com/in/jeevana-gogineni
License
MIT License — free to use, modify, and distribute.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file colonyml-0.1.1.tar.gz.
File metadata
- Download URL: colonyml-0.1.1.tar.gz
- Upload date:
- Size: 2.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ea543acdd7dd6e6274d8859a7994a63f89a57bb1cf5e6d437ee0382ebe2af833
|
|
| MD5 |
6d8e4379746f6d92dbbe157fc8bdd17c
|
|
| BLAKE2b-256 |
9780c826a754a924a4a7e2164c7cf3b35204a2838aa51124bedfc441355dcceb
|
File details
Details for the file colonyml-0.1.1-py3-none-any.whl.
File metadata
- Download URL: colonyml-0.1.1-py3-none-any.whl
- Upload date:
- Size: 2.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
19f82f65dc3da61ef975b6efce148f757244f209c64a6389349cc04aa42613c8
|
|
| MD5 |
4aa84a75c55941fc48e7d98858ecf37b
|
|
| BLAKE2b-256 |
ee31160b77a58bc22953cc2a86aad4b732e628c2a7c47a9f0d9437a6d842d69f
|