Ray gen2 node provider
Project description
Gen2-connector enables Ray cluster to be deployed over IBM Gen2 infrastructure
-
Install Ray release 1.4.1 using
pip install ray[default]==1.4.1
-
Install gen2-connector on your machine
pip install gen2-connector
-
Configure ibm vpc
-
Create/Update security group to have SSH, Redis and Ray Dashboard ports open: 22, 8265 and 6379
-
Create cluster config file
- Use interactive
vpc-config
tool to generate cluster.yaml configuration file
vpc-config --iam-api-key ${IAM_API_KEY} --format ray --filename cluster.yaml
-
Select security group from previous step when prompted
-
The output of the
vpc-config
is a cluster config yaml file, e.g
================================================= Cluster config file: /tmp/tmpkf0dztfk.yaml =================================================
- Alternatively, update cluster config manually based on defaults.yaml
- Use interactive
-
Use generated file to bring ray cluster up, e.g
ray up /tmp/tmpkf0dztfk.yaml
- After finished, find cluster head node and worker nodes ips:
ray get-head-ip /tmp/tmpkf0dztfk.yaml
ray get-worker-ips /tmp/tmpkf0dztfk.yaml
- To get status of the cluster
ray status --address PUBLIC_HEAD_IP:6379
-
Use browser to open ray dashboard on PUBLIC_HEAD_IP:8265. Alternatively use
ray dashboard
to forward ray cluster dashboard to your localhost. -
Submit example task
ray submit /tmp/tmpkf0dztfk.yaml templates/example.py
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for gen2_connector-1.0.0b8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5f5bc11af4ddcc1b432ae87a02957afe315d2e71fcbbcdcbb9c69ea9050d49c0 |
|
MD5 | f27435ce5e720763b1223546365b12a1 |
|
BLAKE2b-256 | ea9df1aa4bca50fc0d7f57488c8cf030792ab2c38d4ed56b0d7761c829ae09ac |