An opinionated implementation of exclusively using airflow DockerOperators for all Operators
Project description
airflow-docker
Description
An opinionated implementation of exclusively using airflow DockerOperators for all Operators.
Default Operator
from airflow_docker.operator import Operator
task = Operator(
image='some-image:latest',
...
)
Default Sensor
from airflow_docker.operator import Sensor
sensor = Sensor(
image='some-image:latest',
...
)
Task Code
from airflow_docker_helper import client
client.sensor(True)
Branch Operator
Dag Task
from airflow_docker.operator import BranchOperator
branching_task = BranchOperator(
image='some-image:latest',
...
)
Task Code
from airflow_docker_helper import client
client.branch_to_tasks(['task1', 'task2'])
Short Circuit Operator
Dag Task
from airflow_docker.operator import ShortCircuitOperator
short_circuit = ShortCircuitOperator(
image='some-image:latest',
...
)
Task Code
from airflow_docker_helper import client
client.short_circuit() # This task will short circuit if this function gets called
Context Usage
Dag Task
from airflow_docker.operator import Operator
task = Operator(
image='some-image:latest',
provide_context=True,
...
)
Task Code
from airflow_docker_helper import client
context = client.context()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
airflow-docker-0.1.4.tar.gz
(9.6 kB
view hashes)
Built Distribution
Close
Hashes for airflow_docker-0.1.4-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 35bfc3ea85e74b738f3dc1bbad5302efc5b7d126b884e2ea307a8b4aa35308a4 |
|
MD5 | 216d883b5cdd48bb2dd9249540d48bb0 |
|
BLAKE2b-256 | 3a658d9c7e52b03a31d6047200eaf833d775f6b28587f9ca7ca224b324b20725 |