Project description

Airflow Ray Executor

Airflow executor implemented using ray

Usage

$ pip install airflow-ray-executor

Edit your airflow.cfg to set your executor to class: airflow_ray_executor.RayExecutor and add ray client address to this file, example:

executor = airflow_ray_executor.RayExecutor

[ray]
# ray client address to connect to ray cluster
# Ray Executor will start Ray on a single machine if not provided
client = ray://127.0.0.1:10001

Please note: Airflow not support sqlite database when executor neither DebugExecutor nor SequentialExecutor

Project details

These details have not been verified by PyPI

Project links

Homepage

Release history Release notifications | RSS feed

This version

0.0.1

Jan 18, 2022

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

airflow-ray-executor-0.0.1.tar.gz (12.6 kB view details)

Uploaded Jan 18, 2022 Source

Built Distribution

airflow_ray_executor-0.0.1-py3-none-any.whl (13.5 kB view details)

Uploaded Jan 18, 2022 Python 3

File details

Details for the file airflow-ray-executor-0.0.1.tar.gz.

File metadata

Download URL: airflow-ray-executor-0.0.1.tar.gz
Upload date: Jan 18, 2022
Size: 12.6 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12

File hashes

Hashes for airflow-ray-executor-0.0.1.tar.gz
Algorithm	Hash digest
SHA256	`60f2a56cd0e9ebd70368fc3274d5f22fb54fb5982ae345e071467ed68df52d79`
MD5	`6843d797db4159b1499d7aadc3bf5da1`
BLAKE2b-256	`554a4ef1462c9d88bde1979a47fd4eaddb4cd1754da54c4297f410ab98d8a16c`

See more details on using hashes here.

File details

Details for the file airflow_ray_executor-0.0.1-py3-none-any.whl.

File metadata

Download URL: airflow_ray_executor-0.0.1-py3-none-any.whl
Upload date: Jan 18, 2022
Size: 13.5 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12

File hashes

Hashes for airflow_ray_executor-0.0.1-py3-none-any.whl
Algorithm	Hash digest
SHA256	`470a8aa53cf2704aa479f235b0b56562792768064eda64e7d4492ff3e5b67d29`
MD5	`088449e10f29e37b9ea3fa7436205689`
BLAKE2b-256	`a593a59f049e40ab686d53160583f8bb50667746e2d87f76df493f6817d91b3a`