dagster-ray
Ray integration for Dagster.
dagster-ray enables working with distributed Ray compute from Dagster pipelines, combining Dagster's excellent orchestration capabilities and Ray's distributed computing power together.
[!NOTE] This project is ready for production use, but some APIs may change between minor releases.
Learn more in the docs
🚀 Key Features
- 🎯 Run Launchers & Executors: Submit Dagster runs or individual steps by submitting Ray jobs
- 🔧 Ray Resources: Automatically create and destroy ephemeral Ray clusters and connect to them in client mode
- 📡 Dagster Pipes Integration: Submit external scripts as Ray jobs, stream back logs and rich Dagster metadata
- ☸️ KubeRay Support: Utilize
RayJobandRayClustercustom resources in client or job submission mode (tutorial) - 🏭 Production Ready: Tested against a matrix of core dependencies, integrated with Dagster+
- ⚡ Instant Startup: Leverage
RayClusterwith Cluster Sharing for lightning-fast development cycles with zero cold start times (intended for development environments)
📦 Installation
pip install dagster-ray
📚 Docs
- Tutorial: Step-by-step guide with examples
- API Reference: Complete API reference
- Changelog: History of changes and links to docs for older
dagster-rayreleases.
👥 Who is using dagster-ray?
These happy users are running dagster-ray in production:
🤝 Contributing
Contributions are very welcome! See CONTRIBUTING.md for details.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file dagster_ray-0.4.6.tar.gz.
File metadata
- Download URL: dagster_ray-0.4.6.tar.gz
- Upload date:
- Size: 54.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
f20dd71b515b2f40101e1ec796bbc271324182702964bde6a930845e5f93482a
|
|
| MD5 |
c4e8b383e852ead48ea991e090bb8836
|
|
| BLAKE2b-256 |
8200e6e355cdb295dfd376970a8cf7f96c10f143ca93d85c1a96b586d93d01db
|
File details
Details for the file dagster_ray-0.4.6-py3-none-any.whl.
File metadata
- Download URL: dagster_ray-0.4.6-py3-none-any.whl
- Upload date:
- Size: 70.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1c0d859ffc9fdbf01d3462df02c6f814ffeeb1d9f95b093e0ca6dbd6b9bf7787
|
|
| MD5 |
04b1221147fc9cb1bfb252ff60af904c
|
|
| BLAKE2b-256 |
f686459afd6c5c78baccf6aaa70a9d9969435f53ba7c03ab4becb6dd0f80accb
|
