Skip to main content
Join the official 2020 Python Developers SurveyStart the survey!

disque client

Project description

pydisque

Client for Disque, an in-memory, distributed job queue.

Build Status

Documentation

Read The Docs

Usage

Create a new Disque client by passing a list of nodes:

from pydisque.client import Client
client = Client(["127.0.0.1:7711", "127.0.0.1:7712", "127.0.0.1:7713"])
client.connect()

If it can't connect to first node, it will try to connect to second, etc.., if it can't connect to any node, it will raise a redis.exceptions.ConnectionError as you can imagine.

Now you can add jobs:

client.add_job("test_queue", json.dumps(["print", "hello", "world", time.time()]), timeout=100)

It will push the job "print" to the queue "test_queue" with a timeout of 100 ms, and return the id of the job if it was received and replicated in time. If it can't reach the node - maybe it was shutdown etc. - it will retry to connect to another node in given node list, and then send the job. If there is no avail nodes in your node list, it will obviously raise a ConnectionError

Then, your workers will do something like this:

while True:
    jobs = client.get_job(['test_queue'])
    for queue_name, job_id, job in jobs:
        job = json.loads(job)
        print ">>> received job:", job
        client.ack_job(job_id)

also check examples directory.

While waiting jobs your connected node may go down, pydisque will try to connect to next node, so you can restart your nodes without taking down your clients.

Documentation

For now please check docstrings in disque/client.py, implemented commands are

  • info
  • add_job
  • get_job
  • ack_job
  • nack_job
  • fast_ack
  • working
  • qlen
  • qstat
  • qpeek
  • qscan
  • jscan
  • enqueue
  • dequeue
  • del_job
  • show
  • pause
  • hello

Installation

You can install it using pip.

$ pip install pydisque

License

This project is licensed under the terms of the MIT license

Credits

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pydisque, version 0.1.5
Filename, size File type Python version Upload date Hashes
Filename, size pydisque-0.1.5-py3-none-any.whl (8.3 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size pydisque-0.1.5.tar.gz (7.1 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page