ray-vast

(work in progress)

Run a Ray cluster with the head on AWS EC2 and GPU workers on vast.ai.

• The Ray head runs on one EC2 instance in your AWS account.
• The Ray autoscaler rents GPU workers from vast.ai when your jobs need them, and removes them when they are idle.
• The workers talk to the head over the public internet, across NAT, with TLS and a token.
• You do not need a VPN or an overlay network.

Documentation: https://duc-ph.github.io/ray-vast/

Why

• vast.ai has cheap GPUs. But a vast instance can stop at any time, and its IP can change.
• ray-vast keeps the head on EC2, so the cluster stays up. Only workers come and go.
• You keep the normal Ray workflow. Only the cluster commands change.
• A worker that loses the head destroys its own instance. You do not pay for lost workers.

Install

pip install ray-vast "ray[default]==2.59.*"

The Ray pin keeps Ray on your laptop at the same version as Ray in the cluster image. ray-vast is tested with Ray 2.59.

The CLI is tested on Linux. macOS is not tested. You need Python 3.10 or later, an AWS profile, the IAM policy ray-vast-head-boundary in your AWS account (a one-time step), and a vast.ai API key. Details: Getting started.

Quick start

Copy an example cluster YAML (gpu-24gb.yaml, gpu-80gb.yaml or cpu-heavy.yaml in ray_vast/examples/). Set the region of the head (node_config.region of the head node type) and the budget (max_dollars_per_hour). up makes an SSH key for the cluster.

ray-vast up cluster.yaml --profile my-aws-profile
ray-vast dashboard cluster.yaml   # keep this terminal open; uses the profile that up saved

In a second terminal, run the export line that dashboard printed, and submit a job:

export RAY_AUTH_MODE=token RAY_AUTH_TOKEN_PATH=...     # the line that dashboard printed
ray job submit --address http://127.0.0.1:8265 --working-dir . -- python train.py

When you finish:

ray-vast down cluster.yaml

Some ray cluster commands do not work. Read limits. Use --working-dir and runtime_env to send code to the cluster.

Ray libraries

Each library page shows how to run your work. It also gives the settings that keep a worker in its process limit.

✅ works (the process settings on its page do not lower the mark). 🟨 works with an extra change, or a part fails. ❌ the main use fails.

Documentation

• Getting started: your first cluster, step by step.
• Ray libraries: Core, Jobs, Data, Train, Tune, Serve, RLlib.
• How-to guides: choose node types, use GPU images, control cost, troubleshoot, clean up.
• Reference: CLI, configuration, limits, IAM permissions, changelog.
• Explanation: architecture, enrollment, worker life cycle, design decisions.
• Security model. Read it before you use the cluster with real data. A vast host has root access to the worker container.

Status

Version 0.2.0. Tested with Ray 2.59.

Contributing

Read CONTRIBUTING.md for the tests and the docs build.

License

MIT. See LICENSE.