A distributed declarative task scheduler built on Celery
thescheduler: A Declarative Task Scheduling Engine built on Celery
This project is in pre-alpha, early development, and subject to fast (read: breaking) changes
A proof-of-concept Distributed Task/Job Scheduler, designed largely for batch-driven environments.
thescheduler exposes built-in functionality of Celery through “tags”, declarative syntax used to create job definitions.
It also implements additional features (e.g. centralized logging, log rotation) to cover extended use-cases.
- Write job definitions using a simple yaml-based syntax.
- Definitions are file based, so that they can be incorporated into existing development workflows (CI/CD). Write/commit/deploy your job definitions alongside your code.
- thescheduler will read and parse valid definitions, add them to the schedule, and send to the pool of workers for execution.
- Build workflows/pipelines by linking jobs using the dependency tag.
$ git clone firstname.lastname@example.org:ehm-are/thescheduler.git $ cd thescheduler $ pip install -e .
or direct from the repository:
pip install git+https://github.com/ehm-are/thescheduler.git
- Configure settings
- Start thescheduler
- Start the workers
- Create a job definition in your configured job path ex: /jobs/test/job_3
runcmd: python3 hello_world.py startat: 13:00 US/Pacific failat: 13:05 US/Pacific runday: mon, wed, fri pool: us-west-1-dev env: PATH="/home/user/.local/bin" dependon: /test/job_1, /test/job_2
- First release on PyPI.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for thescheduler-0.1.0-py2.py3-none-any.whl