oops-amqp 0.0.3

OOPS AMQP transport.

Copyright (c) 2011, Canonical Ltd

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.

The oops_amqp package provides an AMQP OOPS http://pypi.python.org/pypi/oops) publisher, and a small daemon that listens on amqp for OOPS reports and republishes them (into a supplied publisher). The OOPS framework permits falling back to additional publishers if AMQP is down.

Dependencies

• Python 2.6+

• bson

• oops (http://pypi.python.org/pypi/oops)

• amqplib

Testing Dependencies

• oops-datedir-repo (http://pypi.python.org/pypi/oops_datedir_repo)

• rabbitfixture (http://pypi.python.org/pypi/rabbitfixture)

• subunit (http://pypi.python.org/pypi/python-subunit) (optional)

• testresources (http://pypi.python.org/pypi/testresources)

• testtools (http://pypi.python.org/pypi/testtools)

Usage

Publishing to AMQP

Where you are creating OOPS reports, configure oops_amqp.Publisher. This takes a connection factory - a simple callable that creates an amqp connection - and the exchange name and routing key to submit to.

>>> factory = partial(amqp.Connection, host="localhost:5672",
...     userid="guest", password="guest", virtual_host="/", insist=False)
>>> publisher = oops_amqp.Publisher(factory, "oopses", "")

Provide the publisher to your OOPS config:

>>> config = oops.Config()
>>> config.publishers.append(publisher)

Any oops published via that config will now be sent via amqp.

OOPS ids are generating by hashing the oops message (without the id field) - this ensures unique ids.

The reason a factory is used is because amqplib is not threadsafe - the publisher maintains a thread locals object to hold the factories and creates connections when new threads are created(when they first generate an OOPS).

Dealing with downtime

From time to time your AMQP server may be unavailable. If that happens then the Publisher will not assign an oops id - it will return None to signal that the publication failed. To prevent losing the OOPS its a good idea to have a fallback publisher - either another AMQP publisher (to a different server) or one that spools locally (where you can pick up the OOPSes via rsync or some other mechanism. Using the oops standard helper publish_new_only will let you wrap the fallback publisher so that it only gets invoked if the primary method failed:

>>> fallback_factory = partial(amqp.Connection, host="otherserver:5672",
...     userid="guest", password="guest", virtual_host="/", insist=False)
>>> fallback_publisher = oops_amqp.Publisher(fallback_factory, "oopses", "")
>>> config.publishers.append(publish_new_only(fallback_publisher))

Receiving from AMQP

There is a simple method that will run an infinite loop processing reports from AMQP. To use it you need to configure a local config to publish the received reports. A full config is used because that includes support for filtering (which can be useful if you need to throttle volume, for instance). Additionally you need an amqp connection factory (to handle the amqp server being restarted) and a queue name to receive from.

This example uses the DateDirRepo publisher, telling it to accept whatever id was assigned by the process publishing to AMQP:

>>> publisher = oops_datedir_repo.DateDirRepo('.', inherit_id=True)
>>> config = oops.Config()
>>> config.publishers.append(publisher.publish)
>>> receiver = oops_amqp.Receiver(config, factory, "my queue")
>>> receiver.run_forever()

For more information see pydoc oops_amqp.

Installation

Either run setup.py in an environment with all the dependencies available, or add the working directory to your PYTHONPATH.

Development

Upstream development takes place at https://launchpad.net/python-oops-amqp. To setup a working area for development, if the dependencies are not immediately available, you can use ./bootstrap.py to create bin/buildout, then bin/py to get a python interpreter with the dependencies available.

To run the tests use the runner of your choice, the test suite is oops_amqp.tests.test_suite.

For instance:

$ bin/py -m testtools.run oops_amqp.tests.test_suite