Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.8.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | db4b22e567553845033c4897dc36dd8ac18186b4681e92839b4344a8ed30ef64 |
|
MD5 | e52d20f8e3d5eecc5f5369b2af905483 |
|
BLAKE2b-256 | 4c6f915d3927dec46ec3e60e9870ea26b15bf9a27e87ccae0a95a217c756c0bb |
Hashes for flytekitplugins_dask-1.8.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2ba63bdf6b6d421c0c4ef9ad69c180fc1bf609547b2598560fb4afa5a96c972e |
|
MD5 | 60ce006687ad502901e7a244138a3498 |
|
BLAKE2b-256 | 18da73c406951e00c65d0734889cac8aae7425d02c5c6fb6945724f23f3963f9 |