dagster-ray
Ray integration for Dagster.
dagster-ray enables working with distributed Ray compute from Dagster pipelines, combining Dagster's excellent orchestration capabilities and Ray's distributed computing power together.
[!NOTE] This project is ready for production use, but some APIs may change between minor releases.
Learn more in the docs
🚀 Key Features
- 🎯 Run Launchers & Executors: Submit Dagster runs or individual steps by submitting Ray jobs
- 🔧 Ray Resources: Automatically create and destroy ephemeral Ray clusters and connect to them in client mode
- 📡 Dagster Pipes Integration: Submit external scripts as Ray jobs, stream back logs and rich Dagster metadata
- ☸️ KubeRay Support: Utilize
RayJobandRayClustercustom resources in client or job submission mode (tutorial) - 🏭 Production Ready: Tested against a matrix of core dependencies, integrated with Dagster+
- ⚡ Instant Startup: Leverage
RayClusterwith Cluster Sharing for lightning-fast development cycles with zero cold start times (intended for development environments)
📦 Installation
pip install dagster-ray
📚 Docs
- Tutorial: Step-by-step guide with examples
- API Reference: Complete API reference
- Changelog: History of changes and links to docs for older
dagster-rayreleases.
👥 Who is using dagster-ray?
These happy users are running dagster-ray in production:
🤝 Contributing
Contributions are very welcome! See CONTRIBUTING.md for details.
1 maintainer
Release files for dagster-ray 0.4.7
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| dagster_ray-0.4.7.tar.gz | 55.3 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| dagster_ray-0.4.7-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 126.3 kB
Release files / dagster_ray-0.4.7.tar.gz
| Download URL | dagster_ray-0.4.7.tar.gz |
|---|---|
| Size | 55.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bd6f2514a02059e5121a48634b5138921b83d956ab5b52f028c61dba8005df84
|
|
BLAKE2b-256 checksum How to use checksums |
72b9f81f2b9a62195efdfcc76635201882cf8bda6e76ca9ffc75b4535ba4652e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / dagster_ray-0.4.7-py3-none-any.whl
| Download URL | dagster_ray-0.4.7-py3-none-any.whl |
|---|---|
| Size | 71.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9332eb07634fd1c0ee6c6b09246e5c777e0cb80011824844272903ec876f9b5b
|
|
BLAKE2b-256 checksum How to use checksums |
a0635a1816a41dea60c555bcc9dba86c836f88ba93a745a78ff2af69ba7a09e8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
