ray-vast

Run a Ray cluster with the head on AWS EC2 and the workers on vast.ai. You keep the normal Ray workflow. Only the cluster commands change: ray-vast up, ray-vast dashboard and ray-vast down.

Documentation (WIP): https://duc-ph.github.io/ray-vast/

Why

Usually, all nodes of a Ray cluster come from one cloud provider, such as AWS, GCP or Azure. The nodes are in one private network. In this network, they communicate simply and securely.

vast.ai is a market with cheaper GPUs. But vast instances have many constraints:

• An instance is not a VM. It is a container on a host machine. Many renters (co-tenants) can share one host.
• The instances communicate over the internet, not over a private network.
• An instance can stop at any time. Its IP address can change.
• The number of ports and the number of processes and threads have hard limits.

What ray-vast does

• It implements a Ray autoscaler that works with these constraints.
• It solves the communication between the nodes over the internet.
• No critical endpoint is accessible without credentials. All traffic is encrypted. Co-tenants on the same host cannot read it.
• It evaluates the Ray libraries on vast: Core, Data, Train, Tune, Serve and RLlib. The results are in the table below.

The vast host has root access to the worker container. Read the security model before you use private data.

Ray libraries

✅ works. 🟨 works with an extra change. ❌ the main use fails.

Install

pip install ray-vast "ray[default]==2.59.*"

Keep the Ray pin: Ray on your laptop must match Ray in the cluster image. Then follow Getting started.

Documentation

• Getting started: your first cluster, step by step.
• Run your work: submit jobs, Ray libraries, many tasks or actors.
• How-to guides: choose node types and GPU images, control cost, troubleshoot, clean up.
• Reference: CLI, configuration, limits, AWS permissions, changelog.
• How it works: architecture, security model. Read the security model before you use the cluster with real data. A vast host has root access to the worker container.

Status

Tested with Ray 2.59.

Contributing

Read CONTRIBUTING.md for the tests and the docs build.

License

MIT. See LICENSE.