gocept.amqparchive 1.2.1

Archiving, indexing and search for AMQP messages.

This package is an add-on application for gocept.amqprun that provides three features:

• Archive queue messages by writing them to the filesystesm

• Index those messages using ElasticSearch

• Search for messages with a HTML/JavaScript GUI

Contents

• Installation

• Development

• Changelog

Installation

gocept.amqparchive requires an ElasticSearch server. To set up the archive and index portion, add something like this to your gocept.amqprun configuration:

<eventlog>...</eventlog>
<amqp-server>...</amqp-server>
<worker>
  amount 2
  component-configuration site.zcml
</worker>
<settings>
  gocept.amqparchive.elastic_hostname localhost:9200
</settings>

and in site.zcml:

<configure
    xmlns="http://namespaces.zope.org/zope"
    xmlns:amqp="http://namespaces.gocept.com/amqp">

  <include package="gocept.amqprun" />
  <include package="gocept.amqparchive" />

  <amqp:writefiles
    routing_key="key.one key.two key.three"
    queue_name="archive"
    directory="/path/to/archive"
    pattern="{routing_key}/{date}/{msgid}-{unique}.xml"
    />
</configure>

The HTML/JavaScript GUI expects /elasticsearch to proxy to the ElasticSearch server, and /messages to point to the archive directory (/path/to/archive in our example). Here’s an nginx config snippet:

http {
  upstream elasticsearch {
    server localhost:9200;
  }

  server {
    listen localhost:8080;

    location /search/ {
      alias /path/to/gocept.amqparchive.egg/gocept/amqparchive/browser/;
      index index.html;
    }

    location /elasticsearch/ {
      proxy_pass http://elasticsearch/;
    }

    location /messages/ {
      alias /path/to/archive/;
      autoindex on;
    }
}

Development

The source code is available in the mercurial repository at https://code.gocept.com/hg/public/gocept.amqparchive

Please report any bugs you find at https://projects.gocept.com/projects/projects/gocept-amqparchive/issues

Changelog

1.2.1 (2012-02-22)

• Switch to plone.testing.

1.2.0 (2011-08-23)

• Transform the XML body into a nested dict so ElasticSearch can index the fields individually.

1.1.0 (2011-08-23)

• Add command-line script reindex_directory.

1.0.0 (2011-08-22)

• first release.