Skip to main content
Help improve PyPI by participating in a 5-minute user interface survey!

Simple offline task queues.

Project Description

https://travis-ci.org/toastdriven/alligator.png?branch=master

Simple offline task queues. For Python.

“See you later, alligator.”

Requirements

  • Python 2.6+ or Python 3.3+
  • (Optional) redis for the Redis backend
  • FUTURE (Optional) beanstalkc for the Beanstalk backend

WHY?!!1!

  • Because I have NIH-syndrome.
  • Or because I longed for something simple (~375 loc).
  • Or because I wanted something with tests (90%+ coverage) & docs.
  • Or because I wanted pluggable backends.
  • Or because testing some other queuing system was a pain.
  • Or because I’m an idiot.

Basic Usage

This example uses Django, but there’s nothing Django-specific about Alligator. You can use it with any Python code.

from alligator import Gator

from django.contrib.auth.models import User
from django.shortcuts import send_email


# Make a Gator instance.
# Under most circumstances, you would configure this in one place &
# import that instance instead.
gator = Gator('redis://localhost:6379/0')


# The task itself.
# Nothing special, just a plain *undecorated* function.
def follow_email(followee_username, follower_username):
    followee = User.objects.get(username=followee_username)
    follower = User.objects.get(username=follower_username)

    subject = 'You got followed!'
    message = 'Hey {}, you just got followed by {}! Whoohoo!'.format(
        followee.username,
        follower.username
    )
    send_email(subject, message, 'server@example.com', [followee.email])


# An simple, previously expensive view.
@login_required
def follow(request, username):
    # You'd import the task function above.
    if request.method == 'POST':
        # Schedule the task.
        # Use args & kwargs as normal.
        gator.task(follow_email, request.user.username, username)
        return redirect('...')

Running Tasks

Rather than trying to do autodiscovery, fanout, etc., you control how your workers are configured & what they consumer. Create a new executable file (bin script, management command, whatever) & drop in the following code.

from alligator import Gator, Worker

# Bonus points if you import that one pre-configured ``Gator`` instead.
gator = Gator('redis://localhost:6379/0')

# Consume & handle all tasks.
worker = Worker(gator)
worker.run_forever()

License

New BSD

Complex Usage (For Future Docs)

# We're re-using the above imports/setup.

def log_func(job):
    # A simple example of logging a failed task.
    if job.result != SUCCESS:
        logging.error("Job {} failed.".format(job.id))

# A context manager for supplying options
with gator.options(retries=3, async=settings.ASYNC_TASKS, on_error=log_func) as task:
    feeds_job = task(sketchy_fetch_feeds, timeout=30)


# Future wishlist items...

# Dependent tasks, will only run if the listed tasks succeed.
with gator.options(depends_on=[feeds_job]) as task:
    task(rebuild_cache)

# Delayed tasks (run in an hour).
with gator.options(run_after=60 * 60) as task:
    task(this_can_wait)

Running Tests

$ virtualenv env2
$ . env2/bin/activate
$ pip install -r requirements.txt
$ python setup.py develop
$ py.test -s -v --cov=alligator --cov-report=html tests

TODO

  • Scheduled tasks
  • Dependent tasks
  • Cancellable tasks

Release history Release notifications

History Node

0.10.0

History Node

0.9.0

History Node

0.8.0

History Node

0.7.1

History Node

0.7.0

This version
History Node

0.6.0

History Node

0.5.1

History Node

0.5.0

History Node

0.1.0-dev

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
alligator-0.6.0-py2.py3-none-any.whl (15.8 kB) Copy SHA256 hash SHA256 Wheel 2.7 Jan 1, 2015
alligator-0.6.0.tar.gz (218.3 kB) Copy SHA256 hash SHA256 Source None Jan 1, 2015

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page