RQ Retry and Scheduler
Project description
RQ Retry Scheduler
RQ Retry Scheduler extends the RQ worker class to provide functionality to enqueue jobs based on time.
Additionally, it provides a worker that will automatically retry failed jobs using a backoff scheme.
Usage
Queues
The rq_retry_scheduler.Queue class can be used as a drop in replacement for rq.Queue. It provides two additional enqueuing methods:
enqueue_at - Enqueue the job at the specified time. The time should be a datetime.datetime object with the UTC timezone.
enqueue_in - Enqueue the job after the specified amount of time. The time should be a datetime.timedelta object.
Additionally, two other methods are available enqueue_job_at and enqueue_job_in which work like the above methods but take a rq.Job object as the paramter.
Getting list of jobs
Checking if a job is scheduled
You can check if a job or job id is currently scheduled. Example: job in queue
Unscheduling jobs
You can unschedule jobs with Queue.remove_job. This removes the job from the scheduler queue but does not remove it from RQ.
Repeating jobs
Once a job has been enqueued you can set it to repeat with Queue.repeat which takes the job and a datetime.timedelta object. Additionally you can pass a max_runs value to limit the number of times it will repeat.
If the job is not already in the queue it will be enqueued at the given interval.
In order to ensure that job results cleanup doesn’t remove the job (thus breaking the repetition) the job is copied into a new job. The parent job can be accessed via Job.parent for the repeated jobs.
Due to how the jobs are put into the work queues the maximum frequency is controlled by the scheduler’s interval.
Scheduler
In order to move jobs from the schedule queue into the proper RQ queue a scheduler needs to be ran. This can be accomplished via the rqscheduler script. Additionally, you can extend from rq_retry_scheduler.Scheduler to customize the functonality.
Worker
A worker is provided that will requeue jobs that fail using a defiend backoff strategy. The worker is designed to use the functionality from the Queue class to control the backoff.
The worker can be used by using the -w option to rq worker: rq worker -w 'rq_rety_scheduler.Worker.
The backoff times and total attempts is controlled by Worker.retry_delays.
Installation
Install via pip
pip install rq-retry-scheduler
Acknowledgements
This package was based heavily on the work of RQ Scheduler.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for rq_retry_scheduler-0.2.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 52241bcc52de8d3b00112bcd069174389fc68ed03f3eb357d7cf664a15e4cccc |
|
MD5 | 3aecefacb11e8691e3b4e46ccbd3a8fb |
|
BLAKE2b-256 | 2731e444ff416f862bcb57d7d0619eab03b9beae594dc448b35ceb98494a2aa4 |