A Celery beat scheduler in SQL
Project description
IN DEVELOPMENT
celery-sql-beat-reloader
A celery schedule that will sync with a db for the latest schedule of tasks.
Features
- Pulls schedule from a database instead of code.
- Every schedule is timezone aware.
Getting Started
Prerequisites
- Python 3
- celery >= 5.2
- sqlalchemy
Installing
Install from PyPi:
$ pip install celery-sql-beat-reloader
Install from source by cloning this repository:
$ git clone git@github.com:sir-wiggles/celery-sql-beat-reloader.git
$ cd celery-sql-beat-reloader
$ python setup.py install
Usage
celery_sql_beat_reloader
uses sqlalchemy to manage database connections so you should in theory be able to use this on any flavor of SQL database.
Schema
CREATE TABLE celery_beat_schedule (
id SERIAL PRIMARY KEY,
created_date TIMESTAMPTZ DEFAULT NOW(),
updated_date TIMESTAMPTZ DEFAULT NOW(),
last_scheduled_date TIMESTAMPTZ,
active BOOLEAN DEFAULT TRUE,
name TEXT NOT NULL UNIQUE,
task TEXT NOT NULL,
args JSONB DEFAULT '[]',
kwargs JSONB DEFAULT '{}',
type TEXT NOT NULL DEFAULT 'cron',
seconds INTEGER DEFAULT 60,
minute TEXT DEFAULT '*',
hour TEXT DEFAULT '*',
day_of_week TEXT DEFAULT '*',
day_of_month TEXT DEFAULT '*',
month_of_year TEXT DEFAULT '*',
timezone TEXT DEFAULT 'America/Los_Angeles'
);
NOTE: Table is not auto generated.
Column Descriptions
active
- indicates if the schedule is active or not.active=false
, the schedule will be ignored at startup or removed at the next sync interval.active=true
, the schedule will be include at startup or added at the next sync interval.
name
- is a unique name for the schedule.task
- is the import path of the task e.g.test_app.beat1
,test_app.beat2
.type
- is the type of the schedule and supports two types:cron
andperiodic
.cron
- is a crontab with timezone awareness.periodic
- is a schedule with a frequency of seconds from theseconds
column.
last_scheduled_date
- is when the beat was last sent to be processed.
NOTE:
last_scheduled_date
does not mean the task was successful or even if the task was processed. It just means the scheduler sent a request at that time.
Config
In your celery app.conf
you'll need to set beat_reloader_dburl
to the url of your database.
# test_app.py
import celery
app = celery.Celery(
"test",
broker_url="redis://localhost:6379",
timezone="UTC",
beat_reloader_dburl="postgresql+psycopg2://user:pass@127.0.0.1:5432/db",
)
@app.task
def beat1():
print("beat1")
@app.task
def beat2():
print("beat2")
Running
celery --app test_app beat -S celery_sql_beat_reloader:Reloader --max-interval=300 --loglevel=INFO
Specifying the --max-interval
argument will allow you to control how frequently the schedule syncs with the database
Acknowledgments
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for celery_sql_beat_reloader-0.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | fb07e9ca482cd8b3d162c5739db0cea7675e624b70affbf26cb68ed8446d87bf |
|
MD5 | 4de5a424b0d623877fc9a0e90bc88264 |
|
BLAKE2b-256 | 270b7a4e6e3bde35ca632ebe7abf3adbd79f4efea61585879d412cb56e9a078f |
Hashes for celery_sql_beat_reloader-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7f6c823108e64f02ff6495542c68233ef32459556ac8fdf27dabb33359cdc8bd |
|
MD5 | 01a2de8251511985dc6ad0bd1d563b78 |
|
BLAKE2b-256 | 7e538f5f7552de7ff773424ce09cb6e8af4ad7c171e96bb32e2350e466630ca1 |