Skip to main content

Django huey extension which supports multiple huey queues.

Project description


PyPI version

A django extension to run huey with multiple queues. Multiple queues allow tasks to not block each other and to scale independently. Only the redis storage is supported.

  • If you use huey 1.x then install hueyx 0.1.2. Checkout the git tag huey1.x.
  • If you use huey 2.x then install hueyx >= 1.0.


Install it with

pip install hueyx

Add hueyx in your installed apps.


Compared to djhuey, hueyx allows several queues to be defined in the

    'queue_name1': {
        'connection': {
            'host': 'localhost',
            'port': 6379,
            'db': 0,
        'consumer': {
            'workers': 1,
            'worker_type': 'process',
    'queue_name2': {
        'connection': {
            'connection_pool': ConnectionPool(host='localhost', port=6379, db=1)
        'consumer': {
            'multiple_scheduler_locking': True,
            'prometheus_metrics': True,
            'workers': 2,
            'worker_type': 'thread',

The settings are almost the same as in djhuey. Have a look at the huey documentation to see the exact parameter usage.


  • You can only configure redis as storage engine.
  • The name and backend_class parameters are not supported.
  • The options multiple_scheduler_locking and prometheus_metrics_enabled have been added. See below.
  • The parameters heartbeat_timeout for db_task has been added. See below.
from hueyx.queues import hueyx

Define which queue you want to use.
They are predefined in
HUEY_Q1 = hueyx('queue_name1')
HUEY_Q2 = hueyx('queue_name2')

def my_task1():
    print('my_task1 called')

def my_db_task1():
    print('my_db_task1 called')

def my_task2():
    print('my_task2 called')

@HUEY_Q2.periodic_task(crontab(minute='0', hour='3'))
def my_periodic_task2():
    print('my_periodic_task2 called')
    return 1

def my_heartbeat_task(heartbeat: Heartbeat):
    with heartbeat.long_running_operation():
        print('This operation can take a while -> don\'t check for heartbeats')
    print('Now we check for heartbeats -> call heartbeat() periodically')
Push task to queue
from example.tasks import my_task1, my_db_task1, my_task2

my_task1()  # Task for queue_name1
my_db_task1()  # Task for queue_name1
my_task2()  # Task for queue_name2
Run consumer

Consumers are started with the queue_name.

./ run_hueyx queue_name1
Heartbeat tasks

Heartbeat tasks are tasks with the parameter heartbeat_timeout. It defines the timeout in seconds. They get a Heartbeat object which needs to be called in order to send a heartbeat to redis. If no heartbeat occurs in set timeout the task is presumed to be dead and will automatically get restarted. heartbeat_timeout needs to be at least 120 seconds. It does not work together with the parameter include_task.

Additional settings


multiple_scheduler_locking has been added to support multiple huey schedulers. If you run huey in a cloud environment, you will end up running multiple huey instances which each will schedule the periodic task. multiple_scheduler_locking prevents periodic tasks to be scheduled multiple times. It is false by default.

Huey signals

Optionally hueyx pushes all huey signals to the redis pubsub hueyx.huey2.signaling if enabled.

    'enabled': True,
    'environment': 'your environment'

The format of the message is

    'environment': settings.HUEYX_ENVIRONMENT,
    'queue': queue,
    'pid': pid,
    'signal': signal_name,
    'task': task_name

The environment parameter is a optional variable.


The huey-exporter project takes the signals und reports it to prometheus.


Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for hueyx, version 1.0.0
Filename, size File type Python version Upload date Hashes
Filename, size hueyx-1.0.0-py3-none-any.whl (17.1 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size hueyx-1.0.0.tar.gz (12.5 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page