ray-vast

(work in progress)

Run a Ray cluster with the head on AWS EC2 and GPU workers on vast.ai.

• The Ray head runs on one EC2 instance in your AWS account.
• The Ray autoscaler rents GPU workers from vast.ai when your jobs need them, and removes them when they are idle.
• The workers talk to the head over the public internet, across NAT, with TLS and a token.
• You do not need a VPN or an overlay network.

Documentation: https://duc-ph.github.io/ray-vast/

Why

• vast.ai has cheap GPUs. But a vast instance can stop at any time, and its IP can change.
• ray-vast keeps the head on EC2, so the cluster stays up. Only workers come and go.
• You keep the normal Ray workflow. Only the cluster commands change.
• A worker that loses the head destroys its own instance. You do not pay for lost workers.

Install

pip install ray-vast "ray[default]==2.59.*"

The Ray pin keeps Ray on your laptop at the same version as Ray in the cluster image. ray-vast is tested with Ray 2.59.

You need Python 3.10 or later, an AWS profile, an EC2 key pair, and a vast.ai API key. Details: Getting started.

Quick start

Copy an example cluster YAML (gpu-24gb.yaml, gpu-80gb.yaml or cpu-heavy.yaml in ray_vast/examples/). Set the region, the budget (max_dollars_per_hour) and your EC2 key pair (key_name, key_file).

ray-vast up cluster.yaml --profile my-aws-profile
ray-vast dashboard cluster.yaml --profile my-aws-profile   # keep this terminal open

In a second terminal, run the export line that dashboard printed, and submit a job:

export RAY_AUTH_MODE=token RAY_AUTH_TOKEN_PATH=...     # the line that dashboard printed
ray job submit --address http://127.0.0.1:8265 --working-dir . -- python train.py

When you finish:

ray-vast down cluster.yaml --profile my-aws-profile

Some ray cluster commands do not work. Read limits. Use --working-dir and runtime_env to send code to the cluster.

Documentation

• Getting started: your first cluster, step by step.
• How-to guides: run your Ray workload, choose node types, use GPU images, control cost, troubleshoot, clean up.
• Reference: CLI, configuration, Ray library support, limits, IAM permissions, changelog.
• Explanation: architecture, enrollment, worker life cycle, design decisions.
• Security model. Read it before you use the cluster with real data. A vast host has root access to the worker container.

Status

Version 0.1.0. Tested with Ray 2.59.

Contributing

Read CONTRIBUTING.md for the tests and the docs build.

License

MIT. See LICENSE.