Simple Pipeline Scheduler in Python
Project description
airduct
Simple Pipeline Scheduler in Python
Documentation
Installing
$ pip install airduct
or
$ poetry add airduct
Quickstart
Airduct calls pipelines "flows". A flow is a python file with a very particular definition, which is hopefully self explanatory.
Here is an example flow:
from airduct import schedule, task
schedule(
name='ExampleFlow',
run_at='* * * * *',
flow=[
task('e1f1'),
[task('e1f2'), task('e1f3', can_fail=True)],
[task('e1f4')]
]
)
async def e1f1():
print('e1f1 - An async function!')
def e1f2():
print('e1f2 - Regular functions work too')
async def e1f3():
print('e1f3')
async def e1f4():
print('e1f4')
A flow requires a airduct.scheduling.Schedule
object assigned to a schedule
variable. The Schedule object requires:
name
: A name to identify the flow asrun_at
: A crontab-like scheduling syntax. (Uses this crontab parser)flow
: A list ofairduct.scheduling.task
's. These can be nested lists, but only 2 levels deep.
task()
Requires the name of the function you desire to run during that step. Must be defined in that flow file.
This file is placed in a folder/python-module alongside other flows.
Then to run, there are two commands.
airduct schedule --path /path/to/flow/folder
airduct work --path /path/to/flow/folder
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
airduct-0.1.4.tar.gz
(6.3 kB
view hashes)