This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Background tasks using django’s cache framework.

Quickstart

Install django-cacheq:

pip install django-cacheq

Requirements

  • django>=1.5.1
  • jsonfield>=1.0.3
  • lockfile>=0.10.2

Add it to your installed apps:

INSTALLED_APPS = (
    ...
    'cacheq',
)

And that’s it with setup. You can add some basic settings too, but they are not really required.:

CACHES = {
    'default: ...,
    'cacheq': ...,
    'other': ...
}

CACHEQ = {
    'CACHE': 'cacheq',                      # which cache to use, defaults to 'default'
                                            # note that dummybackend is *not* supported
    'LOCKFILE': '/var/tmp/mycacheq.lock',   # lock file to use if cache is filebased,
                                            # defaults to '/var/tmp/cacheq.lock'
    # these settings are only for testing
    'MEMCACHED_TESTS_USING': 'memcached',   # which cache to use for running memcached
                                            # backend tests, only for development
    'REDIS_TESTS_USING': 'redis',           # which cache to use for running redis
                                            # backend tests, only for development
}

Then use it in your project:

import operator
from cacheq import CacheQ

cq = CacheQ(name='myqueue', using='cacheq') # as in get_cache('cacheq')

# either enqueue one job
job = cq.enqueue(operator.add, 1, 2)

# or several at a time. note that both the args=[...] and kwargs={...}
# arguments are required in this case, even if empty
tasks = [(operator.add, [1,2], {}), (operator.div, [], {'a': 2, 'b': 2})]
job = cq.enqueue_many(tasks) # job with many tasks

# or you can use the @cq.job decorator
@cq.job
def myfunc(a,b):
    return a+b

job = myfunc.delay(1,b=2)

# then wait for results
job.ready() # False
job.ready() # True
job.result # 3

# calling job.ready() or job.result will not hit the database
# it will look for result and status in cache. once it's ready
# it will update job in database.

Running the worker:

python manage.py cqworker --using=cacheq --queue=myqueue --name=worker123 --pulse=0.1

This will run a cqworker with name “worker123” in foreground listening to queue “myqueue” using the cache backend under get_cache(‘cacheq’). The ‘pulse’ option is not really necessary, but it will accept any value between 0.0 and 1.0, which will be the time that the worker will wait to look for a new job again. I don’t know if this is really helpful, as it would still be only one connection to memcached / redis, and time.sleep is blocking.

These are the default values

  • using: ‘default’
  • queue: ‘default’
  • name: ‘worker’
  • pulse: 1.0

When running tests it’s helpful to run the worker and exit when jobs are done. You can do this by either calling the cqworker command with the –burst option or by using the worker.run method.:

python manage.py cqworker --using=cacheq --queue=myqueue --burst

# or programatically
from cacheq import get_worker

worker = get_worker(queue_name='myqueue', using='cacheq')
worker.run(burst=True)

django-cacheq uses django ORM as a backend for job results. This is only something that fitted specific needs I had at the time I wrote this package, but I guess it would be wise to remove it at some point and replace it by a cache backend too, or maybe adding a setting that allows other database to be used specifically as a results backend.

Anyways, for now you can clear jobs by using the cqclear command:

python manage.py cqclear <done failed pending all> [--no-input]

In the case you want to delete pending jobs, you will have to confirm the action if you do not provide the –no-input option. So have this in mind if you wish to use a cronjob to clear jobs periodically.

History

0.1.0 (2015-07-25)

  • First release on PyPI.
Release History

Release History

0.1.1

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
django-cacheq-0.1.1.tar.gz (14.0 kB) Copy SHA256 Checksum SHA256 Source Jul 28, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting