Django huey extension which supports multiple huey queues.
Project description
hueyx
A django extension to run huey with multiple queues. Multiple queue allow tasks not to block each other and to scale tasks independently. Only the redis storage is supported by hueyx.
Usage
Install it with
pip install hueyx
Add hueyx in your installed apps.
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'hueyx',
]
settings.py
Compared to djhuey, hueyx allows several queue's to be defined in the settings.py.
HUEYX = {
'queue_name1': {
'result_store': True,
'store_none': False,
'connection': {'host': 'localhost', 'port': 6379, db: 5},
'consumer': {
'workers': 4,
'worker_type': 'process',
}
},
'queue_name2': {
'connection': {
'connection_pool': ConnectionPool(host='localhost', port=6379, db=1)
},
'consumer': {
'workers': 1,
'worker_type': 'process',
}
},
}
The settings are the same as in djhuey except that you can only configure redis as storage engine.
tasks.py
from hueyx.consumers import hueyx
"""
Define which queue you want to use.
They are predefined in settings.py.
"""
HUEY_Q1 = hueyx('queue_name1')
HUEY_Q2 = hueyx('queue_name2')
@HUEY_Q1.task()
def my_task1():
print('my_task1 called')
@HUEY_Q2.task()
def my_task2():
print('my_task2 called')
@HUEY_Q2.periodic_task(crontab(minute='0', hour='3'))
def my_periodic_task2():
print('my_periodic_task2 called')
return 1
Push task to queue
from example.tasks import my_task1, my_task2
my_task1() # Task for queue_name1
my_task2() # Task for queue_name2
Run consumer
Consumers are started with the queue_name.
./manage.py run_hueyx queue_name1
Collaborators
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hueyx-0.0.2.tar.gz
(6.2 kB
view hashes)
Built Distribution
hueyx-0.0.2-py3-none-any.whl
(10.2 kB
view hashes)