Kubeflow Pipeline distributed training support
kfp-dist-train contains utilities to use together with Kubeflow Pipeline to enable writing distributed training code directly using Kubeflow Pipeline SDK.
Get Started
- Setup an Kubeflow environment (maybe use https://github.com/alauda/kubeflow-chart).
- Upload the example kfp-dist-train.ipynb into a Notebook instance, or setup local pipeline submit.
- Execute the example to submit a workflow, you can configure the number of workers in the Kubeflow web UI. The job should look like below:
Some Roadmap
- support
kfpdist.component(dist=True)decorator as an wrap ofdsl.component - support parameter server strategy
- support pytorch
1 maintainer
Release files for kfpdist 0.1.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| kfpdist-0.1.8-py3-none-any.whl | Python 3 | none | any | Details |
Release files / kfpdist-0.1.8-py3-none-any.whl
| Download URL | kfpdist-0.1.8-py3-none-any.whl |
|---|---|
| Size | 3.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
14ec778d5cbaf6dded96b7d15df7ef0ca8f76c6503271b4e4a2d18fe038e2f64
|
|
BLAKE2b-256 checksum How to use checksums |
9939589db3838dcb6edf44cffa4d460d212833c04db9329f91a7365cab0a7df1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.10.6
|
