Skip to main content

Interface for TaskQueue

Project description

Google Taskqueue client

Support for Python client for `Task Queue Rest API`_ using `gcloud`_

.. _Task Queue Rest API:
.. _gcloud:

Required Dependencies

The following third-party Python modules are required:
- gcloud

The easiest way to install the dependencies is to run::

$ pip install -r requirements.txt

Why an interface to the Task Queue API?
The Google Cloud Client Library for Python (`gcloud`_) is great, but it's missing
an interface to the Task Queue Rest API. This is understandable, because the API
is experimental and still in alpha, and not fully accesble from outside the App Egine.
And gcloud does support Pubsub.

However, Task Queues offer a few functions that aren't available in Pubsub. And
although Push Queues are restricted to use within App Engine, Pull Queues can be
used outside the App Engine enviroment.

With Pull Queues a worker can lease tasks for a certain period. During this period,
the tasks aren't available to other workers. This prevents processing a single task
multiple times by different workers. This makes Task Queues very useful for background
processing, for example in combination with object change notifications, to process
objects in a Storage Bucket the moment they are changed or created.




>>> from gcloud_taskqueue import Taskqueue, Client

For object change notifications, you need to create a service account. To pull tasks
created with this account, you need to use it's credentials::

>>> json_credentials_path = "/path/to/my-service-credential.json"
>>> client = Client.from_service_account_json(json_credentials_path, project="my-project")

Get taskqueue::

>>> tq = Taskqueue(client=client, id="my-taskqueue")

Get tasks in taskqueue::

>>> for task in tq.list_tasks(client=client):
>>> print("{}\t{}\t{}".format(, task.leaseTimestamp, task.retry_count))

Lease 10 tasks for 60 seconds::

>>> for task in, num_tasks=10, client=client):
>>> print("{}\t{}\t{}".format(, task.leaseTimestamp, task.retry_count))

Lease 10 tasks with specific tag for 60 seconds::

>>> for task in, num_tasks=10, tag='my-tag', client=client):
>>> print("{}\t{}\t{}".format(, task.leaseTimestamp, task.retry_count))
v0.1.0, -- Initial release.
v0.1.1, 2016.04.07 -- bugfix, Taskqueue().exists() returns false if taskqueue does not exist (
v0.1.2, 2016.04.07 -- fix indentation, raise DoesNotExist exception if Taskqueue does not exist

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for gcloud_taskqueue, version 0.1.2
Filename, size File type Python version Upload date Hashes
Filename, size gcloud_taskqueue-0.1.2.tar.gz (7.8 kB) File type Source Python version None Upload date Hashes View
Filename, size gcloud_taskqueue-0.1.2-py2.py3-none-any.whl (13.3 kB) File type Wheel Python version py2.py3 Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page