Skip to main content

A distributed declarative task scheduler built on Celery

Project description

thescheduler: A Declarative Task Scheduling Engine built on Celery Documentation Status

This project is in pre-alpha, early development, and subject to fast (read: breaking) changes

A proof-of-concept Distributed Task/Job Scheduler, designed largely for batch-driven environments.

thescheduler exposes built-in functionality of Celery through “tags”, declarative syntax used to create job definitions.

It also implements additional features (e.g. centralized logging, log rotation) to cover extended use-cases.


  • Write job definitions using a simple yaml-based syntax.

  • Definitions are file based, so that they can be incorporated into existing development workflows (CI/CD). Write/commit/deploy your job definitions alongside your code.

  • thescheduler will read and parse valid definitions, add them to the schedule, and send to the pool of workers for execution.

  • Build workflows/pipelines by linking jobs using the dependency tag.

External Dependencies

  • redis


  • Development version:

    $ git clone
    $ cd thescheduler
    $ pip install -e .
  • or direct from the repository:

    pip install git+




  1. Configure settings


  2. Start thescheduler


  3. Start the workers


  4. Create a job definition in your configured job path ex: /jobs/test/job_3
    runcmd:      python3
    startat:     13:00 US/Pacific
    failat:      13:05 US/Pacific
    runday:      mon, wed, fri
    pool:        us-west-1-dev
    env:         PATH="/home/user/.local/bin"
    dependon:    /test/job_1, /test/job_2


This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.


0.1.0 (2020-01-01)

  • First release on PyPI.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

thescheduler-0.1.0.tar.gz (12.8 kB view hashes)

Uploaded Source

Built Distribution

thescheduler-0.1.0-py2.py3-none-any.whl (8.8 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page