Ray gen2 node provider
Project description
Gen2-connector enables Ray cluster to be deployed over IBM Gen2 infrastructure
-
Install Ray release 1.4.1 using
pip install ray[default]==1.4.1
-
Install gen2-connector on your machine
pip install gen2-connector
-
Configure ibm vpc
-
Create/Update security group to have SSH, Redis and Ray Dashboard ports open: 22, 8265 and 6379
-
Create cluster config file
- Use interactive
vpc-config
tool to generate cluster.yaml configuration file
vpc-config --iam-api-key ${IAM_API_KEY} --format ray --filename cluster.yaml
-
Select security group from previous step when prompted
-
The output of the
vpc-config
is a cluster config yaml file, e.g
================================================= Cluster config file: /tmp/tmpkf0dztfk.yaml =================================================
- Alternatively, update cluster config manually based on defaults.yaml
- Use interactive
-
Use generated file to bring ray cluster up, e.g
ray up /tmp/tmpkf0dztfk.yaml
- After finished, find cluster head node and worker nodes ips:
ray get-head-ip /tmp/tmpkf0dztfk.yaml
ray get-worker-ips /tmp/tmpkf0dztfk.yaml
- To get status of the cluster
ray status --address PUBLIC_HEAD_IP:6379
-
Use browser to open ray dashboard on PUBLIC_HEAD_IP:8265. Alternatively use
ray dashboard
to forward ray cluster dashboard to your localhost. -
Submit example task
ray submit /tmp/tmpkf0dztfk.yaml templates/example.py
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for gen2_connector-1.0.0b9-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4d9e733f45579e83aa56adfaef66f74c0bdbceebe99a93dc9a956b04cb087a67 |
|
MD5 | 77705201e762d0210147269055c80254 |
|
BLAKE2b-256 | cac03ebe0f8a378aae8e685819e2de2f16096e354aca0f0b219c07726841777c |