Dask plugin for flytekit
Project description
Flytekit Dask Plugin
Flyte can execute dask
jobs natively on a Kubernetes Cluster, which manages the virtual dask
cluster's lifecycle
(spin-up and tear down). It leverages the open-source Kubernetes Dask Operator and can be enabled without signing up
for any service. This is like running a transient (ephemeral) dask
cluster - a type of cluster spun up for a specific
task and torn down after completion. This helps in making sure that the Python environment is the same on the job-runner
(driver), scheduler and the workers.
To install the plugin, run the following command:
pip install flytekitplugins-dask
To configure Dask in the Flyte deployment's backed, follow step 1 and step 2
An example can be found in the documentation.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for flytekitplugins-dask-1.10.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1f612db5872cd54e9dbb07915b24456d5a91af5946de5c66d7fc2dee3ebe3eac |
|
MD5 | 3190f157cc107f7fae7d03a882ea5930 |
|
BLAKE2b-256 | e22ff1a9080e354b005366a067d8752174b95238f83be6a425b88fbb0de05269 |
Hashes for flytekitplugins_dask-1.10.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cd61df4fcb99fb4dbfe36d5d23335c35b80b6fc168c1e4344233a9e548d87508 |
|
MD5 | f819fd203544dbc59896bf27aa99af37 |
|
BLAKE2b-256 | dd0a4d2bbb2c805da14610863c79bee2d6d2a8a7d1faf67b29bdda47a6742ac6 |