Skip to main content

Backup/Restore RabbitMQ queues

Project description

This is a Python library and utilities for saving and loading RabbitMQ messages to and from disk.


  • A program cannot publish a message to RabbitMQ, but does not want to lose the message. Using this library, the program can save the message to a file. Later, after fixing the problem, a person can use a utility in this package to publish the message.
  • The consumer has a bug and messages are piling up in a queue. You have fixed the bug and your tests seem to indicate the consumer will now work, but you want some insurance. Using a program in this package, you can back up the queue to a file. If the program processes the messages incorrectly, you can use use a program in this package to restore the queue from the file.
  • You want to edit messages in a queue. You can dump the queue to a file, use a text editor or a one-off program to change the file, and then write the messages back into the queue.

Why not?

This library is brand new and has not been battle tested. Proceed with caution.

Backup format

Queues are backed up and restored from text files containing json. Each line in the file is a JSON dictionary with this structure:

    "body": "This is a test",
    "delivery_info": {
        "delivery_tag": 1
    "properties": {
        "app_id": null,
        "cluster_id": null,
        "content_encoding": null,
        "content_type": "text/plain",
        "correlation_id": null,
        "delivery_mode": 2,
        "expiration": null,
        "headers": {
            "client_id": null,
            "host_ip": "",
            "host_name": "treebeard",
            "library_id": "olio_msg (Python) 1.9.0",
            "metadata_version": "1.0.0",
            "program_name": "olio_msg_send_test_messages",
            "version": "1.0.0"
        "message_id": "1df3eb23ffeb476b8355d87b475eb627",
        "priority": null,
        "reply_to": null,
        "timestamp": 1434644774,
        "type": "test",
        "user_id": null

The line was shown pretty-printed, but it’s actually just one line:

{"body": "This is a test", "delivery_info": {...}, "properties: {...}}

Using the command-line utilities

To backup a queue

rabbit_droppings --host localhost --queue jobs \
  --file /path/to/save/file --dump

To backup and purge a queue

rabbit_droppings --host localhost --queue jobs \
  --file /path/to/save/file --purge

To restore a queue

rabbit_droppings --host localhost --queue jobs \
  --file /path/to/save/file --restore

Saving a message with the pika library

If you have a program using the pika library to publish messages to a RabbitMQ server, here’s how to save messages that could not be published:

First, import the library:

import rabbit_droppings

If you have a program that needs to save messages that cannot be published using the pika library, it should create a rabbit_droppings.Writer (probably in the constructor):

file = open('/path/to/my/file', 'a')
self._rd_writer = rabbit_droppings.Writer(file)

When you publish a message and a pika exception occurs, save the message:

body = "Message body"
properties = pika.spec.BasicProperties(
except (pika.exceptions.AMQPError, pika.exceptions.ChannelError) as e:
    pika_message = rabbit_droppings.PikaMessage(body, properties=properties)


This library practices Semantic Versioning.

This library is currently in alpha; it’s versions look like “0.1.0”, “0.2.0”, etc. There are no guarantees with alpha versions: Any version bump could be any combination of bug fix, backwards compatible API change, or breaking API change.

When the library becomes stable, its version number will be bumped to “1.0.0”. Semantic versioning makes these promises for stable versions:

  • A patch-level version bump (e.g. “1.0.0” to “1.0.1”) does not change the public API.
  • A minor-level version bump (e.g. “1.0.0” to “1.1.0”) changes the public API in a backward-compatible manner.
  • A major-level version bump (e.g. “1.0.0” to “2.0.0”) changes the public API in some way that is not backward compatible.

Python version

Known to work with Python versions:

  • 2.6.9
  • 2.7.9


Running the tests requires a RabbitMQ server installed locally. The tests are known to pass with these RabbitMQ versions:

  • 3.4.1

To run the tests:

./ test

Working with the pypi test server

To register (only once):

python register -r

To publish to the pypi test server:

python bdist_wheel upload -r pypitest
python sdist upload -r pypitest

To install from the pypi test server:

pip install -i rabbit_droppings

Working with the “real” pypi server

To register (only once):

python register -r

To publish to the “real” pypi server:

python bdist_wheel upload -r pypi
python sdist upload -r pypi

To install from the “real” pypi server:

pip install -i rabbit_droppings

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for rabbit_droppings, version 0.2.0
Filename, size File type Python version Upload date Hashes
Filename, size rabbit_droppings-0.2.0.tar.gz (8.6 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page