Ray gen2 node provider
Project description
Gen2-connector enables Ray cluster to be deployed over IBM Gen2 infrastructure
-
Install Ray release 1.4.1 using
pip install ray[default]==1.4.1
-
Install gen2-connector on your machine
pip install gen2-connector
-
Configure ibm vpc
-
Create/Update security group to have SSH, Redis and Ray Dashboard ports open: 22, 8265 and 6379
-
Create cluster config file
- Use interactive
vpc-config
tool to generate cluster.yaml configuration file
vpc-config --iam-api-key ${IAM_API_KEY} --format ray --filename cluster.yaml
-
Select security group from previous step when prompted
-
The output of the
vpc-config
is a cluster config yaml file, e.g
================================================= Cluster config file: /tmp/tmpkf0dztfk.yaml =================================================
- Alternatively, update cluster config manually based on defaults.yaml
- Use interactive
-
Use generated file to bring ray cluster up, e.g
ray up /tmp/tmpkf0dztfk.yaml
- After finished, find cluster head node and worker nodes ips:
ray get-head-ip /tmp/tmpkf0dztfk.yaml
ray get-worker-ips /tmp/tmpkf0dztfk.yaml
- To get status of the cluster
ray status --address PUBLIC_HEAD_IP:6379
-
Use browser to open ray dashboard on PUBLIC_HEAD_IP:8265. Alternatively use
ray dashboard
to forward ray cluster dashboard to your localhost. -
Submit example task
ray submit /tmp/tmpkf0dztfk.yaml templates/example.py
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for gen2_connector-1.0.0b33-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec5d0bb9f0f10e691765734447f837c463ac1e711bcc19677ca781d25a7c7d97 |
|
MD5 | 68941e5f27fcac1f450c288000bbdd59 |
|
BLAKE2b-256 | c34fa640b0158c62776a87c96623ac0e6b4256372071a4c3c22f92ddd28ab3e7 |