Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.3.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d7a0129e9d98dc9951667b5cad4ce2f319f452864ac30fcf1f97088621326b54 |
|
MD5 | 04447da03a1effe97972ff8660bd3092 |
|
BLAKE2b-256 | 9de6a82ed5ce4e36d7ffc43713aaf45ffd6ee49acf5bce55c566e770c3c7e71b |
Hashes for flytekitplugins_dask-1.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 67405128f4e4de1699f59c0af4ff16e0b5c8adf854d166f8ff25c154e307bbd1 |
|
MD5 | f2cd3f01b2922bdd4ca7a3ebd08fd2a0 |
|
BLAKE2b-256 | 1c3ff217e490e6579b64e3ad33f6fc11b9011fd639d7467a46428ff769dca886 |