thescheduler: A Declarative Task Scheduling Engine built on Celery
This project is in pre-alpha, early development, and subject to fast (read: breaking) changes
A proof-of-concept Distributed Task/Job Scheduler, designed largely for batch-driven environments.
thescheduler exposes built-in functionality of Celery through “tags”, declarative syntax used to create job definitions.
It also implements additional features (e.g. centralized logging, log rotation) to cover extended use-cases.
Features
Write job definitions using a simple yaml-based syntax.
Definitions are file based, so that they can be incorporated into existing development workflows (CI/CD). Write/commit/deploy your job definitions alongside your code.
thescheduler will read and parse valid definitions, add them to the schedule, and send to the pool of workers for execution.
Build workflows/pipelines by linking jobs using the dependency tag.
External Dependencies
redis
Installation
Development version:
$ git clone git@github.com:ehm-are/thescheduler.git $ cd thescheduler $ pip install -e .
or direct from the repository:
pip install git+https://github.com/ehm-are/thescheduler.git
Docker
TODO
Usage
- Configure settings
TODO
- Start thescheduler
TODO
- Start the workers
TODO
- Create a job definition in your configured job path ex: /jobs/test/job_3
runcmd: python3 hello_world.py startat: 13:00 US/Pacific failat: 13:05 US/Pacific runday: mon, wed, fri pool: us-west-1-dev env: PATH="/home/user/.local/bin" dependon: /test/job_1, /test/job_2
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
Free software: Mozilla Public License 2.0 (MPL 2.0)
Documentation: https://thescheduler.readthedocs.io.
History
0.1.0 (2020-01-01)
First release on PyPI.
Metadata
Release files for thescheduler 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| thescheduler-0.1.0.tar.gz | 12.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| thescheduler-0.1.0-py2.py3-none-any.whl | Python 3, Python 2 | none | any | Details |
Total release size: 21.6 kB
Release files / thescheduler-0.1.0.tar.gz
| Download URL | thescheduler-0.1.0.tar.gz |
|---|---|
| Size | 12.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ecda97541c588610a089abd9f8fc13cf08970c4cb7171c7fd6c97430a3eb69f2
|
|
BLAKE2b-256 checksum How to use checksums |
b4124babf44ccb8c70b9d6d1ae266706783f456a25255fc59e4bb7610dc5ee76
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/43.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/2.7.16
|
Release files / thescheduler-0.1.0-py2.py3-none-any.whl
| Download URL | thescheduler-0.1.0-py2.py3-none-any.whl |
|---|---|
| Size | 8.8 kB |
| Tags | Python 2 Python 3 |
|
SHA-256 checksum How to use checksums |
08430e45dc16104d4b51cb89b8d407ea66065fadaa1ff9b4fb726d330e3e2962
|
|
BLAKE2b-256 checksum How to use checksums |
7aa4afff0e568e24e057589589339c7aaaa49e9e36869055aaf2bb24b32b8065
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.15.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/43.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/2.7.16
|