Ray gen2 node provider
Project description
Gen2-connector enables Ray cluster to be deployed over IBM Gen2 infrastructure
-
Install Ray release 1.4.1 using
pip install ray[default]==1.4.1
-
Install gen2-connector on your machine
pip install gen2-connector
-
Configure ibm vpc
-
Create/Update security group to have SSH, Redis and Ray Dashboard ports open: 22, 8265 and 6379
-
Create cluster config file
- Use interactive
vpc-config
tool to generate cluster.yaml configuration file
vpc-config --iam-api-key ${IAM_API_KEY} --format ray --filename cluster.yaml
-
Select security group from previous step when prompted
-
The output of the
vpc-config
is a cluster config yaml file, e.g
================================================= Cluster config file: /tmp/tmpkf0dztfk.yaml =================================================
- Alternatively, update cluster config manually based on defaults.yaml
- Use interactive
-
Use generated file to bring ray cluster up, e.g
ray up /tmp/tmpkf0dztfk.yaml
- After finished, find cluster head node and worker nodes ips:
ray get-head-ip /tmp/tmpkf0dztfk.yaml
ray get-worker-ips /tmp/tmpkf0dztfk.yaml
- To get status of the cluster
ray status --address PUBLIC_HEAD_IP:6379
-
Use browser to open ray dashboard on PUBLIC_HEAD_IP:8265. Alternatively use
ray dashboard
to forward ray cluster dashboard to your localhost. -
Submit example task
ray submit /tmp/tmpkf0dztfk.yaml templates/example.py
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for gen2_connector-1.0.0b18-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b2102ac0792494273932e328caa5dbf6b2254dbe94ca9d5e40bb42277ca9fa9c |
|
MD5 | 8ccbea62e0f9ecf6af5740455fdb1561 |
|
BLAKE2b-256 | 9f86f33e1d6958cb6f6267e30323f3160640d6939cf9c6cc295bc5dd9dcfd65a |