Dask Cluster objects in Saturn Cloud
Project description
dask-saturn
Python library for interacting with Dask clusters in Saturn Cloud.
Dask-Saturn mimics the API of Dask-Kubernetes, but allows the user to interact with clusters created within Saturn Cloud.
Start cluster
In order to interact with a Dask cluster, the cluster must first be created in the Saturn User Interface. Then, from within a Jupyter notebook, you can start the cluster and adjust the number of workers.
from dask_saturn import SaturnCluster
cluster = SaturnCluster
cluster
Adjust number of workers
Once you have a cluster you can interact with it via the jupyter
widget, or using the scale
and adapt
methods.
For example, to manually scale up to 20 workers:
cluster.scale(20)
To create an adaptive cluster that controls its own scaling:
cluster.adapt(minimum=1, maximum=20)
Interact with client
To submit tasks to the cluster, you sometimes need access to the
Client
object. Instantiate this with the cluster as the only argument:
from distributed import Client
client = Client(cluster)
client
Close cluster
To terminate all resources associated with a cluster, use the
close
method:
cluster.close()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dask_saturn-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 289f176fc3041c01f4bd036b9fdf478ba1428c2168c077b476a544cee03aaf2b |
|
MD5 | adb122c865d27653673bfbcd5f3ba6e5 |
|
BLAKE2b-256 | 74ddbf26c7b9ed7115b843847bd6579cd00e1a6290a4e82398c18ae71b6c79c0 |