Skip to main content

RabbitMQ Easy

PyPI version Python Support License: MIT

A simple, robust RabbitMQ manager for Python applications with built-in connection management, retry logic, and dead letter queue support.

Features

  • Automatic Connection Management: Built-in retry logic and connection recovery
  • Dead Letter Queue Support: Automatic setup of dead letter exchanges and queues
  • Comprehensive Logging: Console logging by default, optional log file
  • Environment Variable Support: Easy configuration through environment variables
  • Idempotent Operations: Safe to run multiple times without errors
  • Context Manager Support: Proper resource cleanup
  • Production Ready: Used in production environments
  • Resource Management: Built-in cleanup and deletion functions

Installation

pip install rabbitmq-easy

Quick Start

Basic Usage

from rabbitmq_easy import RabbitMQManager

# Simple setup
manager = RabbitMQManager(
    host='localhost',
    port=5672,
    username='guest',
    password='guest',
    queues=['orders', 'payments'],
    routing_keys=['orders.*', 'payments.*'],
    exchange='my_exchange'
)

# Publish a message
manager.publish_message('my_exchange', 'orders.new', '{"order_id": 123}')

# Use as context manager
with RabbitMQManager() as manager:  # Uses environment variables
    manager.publish_message('my_exchange', 'orders.new', '{"order_id": 123}')

Environment Variables Setup

Create a .env file:

RABBITMQ_HOST=localhost
RABBITMQ_PORT=5672
RABBITMQ_USERNAME=guest
RABBITMQ_PASSWORD=guest
RABBITMQ_EXCHANGE=my_exchange
RABBITMQ_QUEUES=orders,payments,notifications
RABBITMQ_ROUTING_KEYS=orders.*,payments.*,notifications.*

Then use:

from rabbitmq_easy import create_rabbitmq_manager

# Auto-loads from environment variables
manager = create_rabbitmq_manager()

Advanced Usage

Custom Configuration

from rabbitmq_easy import RabbitMQManager

# Advanced configuration
manager = RabbitMQManager(
    host='rabbitmq.example.com',
    port=5672,
    username='user',
    password='pass',
    queues=['high_priority', 'low_priority'],
    routing_keys=['urgent.*', 'normal.*'],
    exchange='task_exchange',
    dead_letter_exchange='failed_tasks',
    dead_letter_routing_key='failed',
    dead_letter_queue_name='failed_tasks',
    max_retries=3,
    retry_delay=5,
    enable_console_logging=True,
    log_level='INFO'
)

# Setup additional queues dynamically
manager.setup_queue(
    queue_name='special_queue',
    exchange='task_exchange',
    routing_key='special.*'
)

# Get queue information
info = manager.get_queue_info('high_priority')
print(f"Messages in queue: {info['message_count']}")

# Health check
health = manager.health_check()
print(f"Status: {health['status']}")

Consumer Example

import json
from rabbitmq_easy import RabbitMQManager

def process_message(ch, method, properties, body):
    """Process incoming message"""
    try:
        data = json.loads(body)
        print(f"Processing: {data}")
        
        # Your business logic here
        
        # Acknowledge message
        ch.basic_ack(delivery_tag=method.delivery_tag)
    except Exception as e:
        print(f"Error processing message: {e}")
        # Reject message (will go to dead letter queue if configured)
        ch.basic_nack(delivery_tag=method.delivery_tag, requeue=False)

# Setup consumer
with RabbitMQManager() as manager:
    manager.start_consuming('orders', process_message)

Dead Letter Queue Handling

def handle_failed_messages(ch, method, properties, body):
    """Handle messages that failed processing"""
    try:
        # Get failure information
        headers = properties.headers or {}
        death_info = headers.get('x-death', [])
        
        if death_info:
            failure_reason = death_info[0].get('reason')
            failure_count = death_info[0].get('count')
            print(f"Message failed {failure_count} times. Reason: {failure_reason}")
        
        # Reprocess or log the failure
        data = json.loads(body)
        print(f"Dead letter message: {data}")
        
        # Acknowledge to remove from DLQ
        ch.basic_ack(delivery_tag=method.delivery_tag)
        
    except Exception as e:
        print(f"Error handling dead letter: {e}")
        # Acknowledge to prevent infinite loops
        ch.basic_ack(delivery_tag=method.delivery_tag)

# Monitor failed messages
manager.start_consuming('failed_messages', handle_failed_messages)

Configuration Options

Parameter Environment Variable Default Description
host RABBITMQ_HOST localhost RabbitMQ server host
port RABBITMQ_PORT 5672 RabbitMQ server port
username RABBITMQ_USERNAME guest Username for authentication
password RABBITMQ_PASSWORD guest Password for authentication
exchange RABBITMQ_EXCHANGE '' Exchange name
queues RABBITMQ_QUEUES [] Comma-separated list of queues
routing_keys RABBITMQ_ROUTING_KEYS [] Comma-separated list of routing keys
dead_letter_exchange RABBITMQ_DEAD_LETTER_EXCHANGE {exchange}_dlx Dead letter exchange name
dead_letter_routing_key RABBITMQ_DEAD_LETTER_ROUTING_KEY dead_letter Dead letter routing key
max_retries RABBITMQ_MAX_RETRIES 5 Maximum connection retry attempts
retry_delay RABBITMQ_RETRY_DELAY 5 Delay between retry attempts
heartbeat RABBITMQ_HEARTBEAT 60 Connection heartbeat interval
publisher_confirms RABBITMQ_PUBLISHER_CONFIRMS True Wait for broker acknowledgement on publish
prefetch_count — 1 Unacknowledged messages allowed per consumer
enable_console_logging RABBITMQ_CONSOLE_LOGGING True Log to stdout
log_level RABBITMQ_LOG_LEVEL INFO Logging level
log_file — None Path to write a log file. No file is written unless set
load_env — True Read a .env file when the manager is constructed

Keyword arguments always take precedence over the environment. Passing an empty list (queues=[]) means empty — it does not fall back to RABBITMQ_QUEUES.

Changed in 1.1.0: every variable in the table above is now actually read. Earlier versions documented nine of them but only ever read RABBITMQ_HOST, RABBITMQ_PORT, RABBITMQ_USERNAME, RABBITMQ_PASSWORD and RABBITMQ_EXCHANGE. If you already have any of the others set in a .env file, they will now take effect — check them before upgrading.

Queue and Exchange Operations

# Delete specific resources
manager.delete_queue("old_queue")
manager.delete_exchange("old_exchange")

# Delete only if empty/unused
manager.delete_queue("queue_name", if_empty=True)
manager.delete_exchange("exchange_name", if_unused=True)

# Purge messages without deleting queue
message_count = manager.purge_queue("queue_name")
print(f"Purged {message_count} messages")

# Clean up dead letter infrastructure
manager.cleanup_dead_letter_setup()

# Delete all resources created by this manager
results = manager.delete_all_setup_resources(confirm=True)

# Complete reset
manager.reset_manager(confirm=True)

Error Handling

The package includes custom exceptions for better error handling:

from rabbitmq_easy import RabbitMQConnectionError, RabbitMQConfigurationError

try:
    manager = RabbitMQManager(
        host='invalid-host',
        queues=['queue1', 'queue2'],
        routing_keys=['key1']  # Mismatch with queues count
    )
except RabbitMQConfigurationError as e:
    print(f"Configuration error: {e}")
except RabbitMQConnectionError as e:
    print(f"Connection error: {e}")

Delivery Guarantees

Publishing

Publisher confirms are enabled by default, so publish_message() waits for the broker to acknowledge the message and raises RabbitMQManagerError if it is rejected. Without them a publish returns as soon as the bytes reach the socket, and a broker crash loses messages you believe were sent.

# default: blocks until the broker confirms
manager.publish_message('orders', 'orders.new', '{"id": 1}')

# also fail if no queue is bound for the routing key, instead of
# the broker silently discarding the message
manager.publish_message('orders', 'orders.new', '{"id": 1}', mandatory=True)

Confirms cost a round-trip per publish. If you are publishing in bulk and can tolerate loss, turn them off:

manager = RabbitMQManager(host='localhost', publisher_confirms=False)

Changed in 1.1.0: publishes were previously fire-and-forget. They are now confirmed by default, which is slower and can raise where it previously returned silently.

Consuming

If the connection drops while consuming, the consumer reconnects and resumes automatically, giving up after max_retries consecutive failures. A consumer that stayed connected for a while before dropping gets a fresh retry budget, so a long-running consumer is not penalised for failures spread out over time.

# reconnects on connection loss (default)
manager.start_consuming('orders', process_message)

# fail immediately instead
manager.start_consuming('orders', process_message, reconnect=False)

Changed in 1.1.0: a dropped connection previously raised RabbitMQManagerError and ended the consumer.

Throughput

prefetch_count defaults to 1, meaning the broker sends one message at a time and waits for an acknowledgement before sending the next. That is the safest setting and the slowest: it puts a network round-trip between every message. For higher throughput, raise it:

manager = RabbitMQManager(host='localhost', prefetch_count=100)

Note that pika's blocking connection is not thread-safe. A manager instance, and the channel it holds, should be used from a single thread.

Shutdown and Cleanup

The manager holds an open connection, so it needs to be closed. The context manager is the recommended way, since it closes on both normal exit and exceptions:

with RabbitMQManager(host='localhost') as manager:
    manager.publish_message('my_exchange', 'orders.new', '{"order_id": 123}')
# connection closed here, even if the block raised

Without the context manager, close it yourself:

manager = RabbitMQManager(host='localhost')
try:
    manager.publish_message('my_exchange', 'orders.new', '{"order_id": 123}')
finally:
    manager.close()

Interrupting a consumer

start_consuming() blocks until the consumer stops. Pressing Ctrl-C stops the consumer and then re-raises KeyboardInterrupt, so the interrupt reaches your code and the process terminates as expected:

with RabbitMQManager(host='localhost') as manager:
    try:
        manager.start_consuming('orders', process_message)
    except KeyboardInterrupt:
        print("Shutting down")
    # close() runs on the way out of the with block

Catching KeyboardInterrupt is optional. If you do not catch it, it propagates normally and the with block still closes the connection.

Changed in 1.1.0: earlier versions caught KeyboardInterrupt inside start_consuming() and did not re-raise it, so Ctrl-C never reached the caller and execution continued to the next statement. If you relied on that, catch the exception explicitly as shown above.

Logging

The library attaches no log handlers of its own beyond a NullHandler. Console logging is on by default and can be disabled; file logging is off unless you ask for it:

# quiet: configure the 'rabbitmq_easy' logger from your application instead
manager = RabbitMQManager(host='localhost', enable_console_logging=False)

# write to a file as well
manager = RabbitMQManager(host='localhost', log_file='rabbitmq.log')

Changed in 1.1.0: earlier versions always created rabbitmq_manager.log in the current working directory, and logged the broker password into it. Credentials are no longer logged, and no file is written unless log_file is set. Delete any rabbitmq_manager.log left by an earlier version and rotate credentials that appeared in it.

What Gets Created Automatically

When you initialize with:

manager = RabbitMQManager(
    exchange='orders',
    queues=['new_orders', 'pending_orders'],
    routing_keys=['orders.new', 'orders.pending']
)

RabbitMQ Easy automatically creates:

  1. orders exchange (main exchange)
  2. orders_dlx exchange (dead letter exchange)
  3. new_orders queue → bound to orders exchange with orders.new routing key
  4. pending_orders queue → bound to orders exchange with orders.pending routing key
  5. failed_messages queue → bound to orders_dlx exchange for error handling

All queues are configured with dead letter routing to capture failed messages automatically.

Best Practices

  1. Use Environment Variables: Store sensitive information like passwords in environment variables
  2. Context Managers: Use with statements for automatic cleanup
  3. Dead Letter Queues: Always configure dead letter queues for production
  4. Health Checks: Implement health checks in your applications
  5. Logging: Monitor logs for connection issues and errors
  6. Error Handling: Always handle exceptions in your consumers
  7. Resource Cleanup: Use the provided cleanup functions during development

Queue and Routing Key Validation

The number of queues must match the number of routing keys:

# Correct
manager = RabbitMQManager(
    queues=['queue1', 'queue2'],
    routing_keys=['key1', 'key2']  # Same count
)

# Will raise RabbitMQConfigurationError
manager = RabbitMQManager(
    queues=['queue1', 'queue2'],
    routing_keys=['key1']  # Mismatch!
)

Dead Letter Configuration

Don't configure dead letter queues to point to themselves:

# Don't do this - creates infinite loop
manager.setup_queue(
    'my_dlq',
    dead_letter_exchange='same_exchange'
)

# Do this instead
manager.setup_queue('my_dlq', dead_letter_exchange=None)

Monitoring and Health Checks

# Check connection health
health = manager.health_check()
if health['status'] == 'healthy':
    print("RabbitMQ connection is healthy")
else:
    print(f"RabbitMQ connection issues: {health['error']}")

# Get detailed queue information
info = manager.get_queue_info('orders')
print(f"Queue: {info['queue']}")
print(f"Messages: {info['message_count']}")
print(f"Consumers: {info['consumer_count']}")

Production Deployment

Docker Compose Example

version: '3.8'
services:
  app:
    build: .
    environment:
     - ~/etc/<org_name>/.env
    depends_on:
      - rabbitmq
  
  rabbitmq:
    image: rabbitmq:3-management
    environment:
     - ~/etc/<org_name>/.env

Kubernetes ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: rabbitmq-config
data:
  RABBITMQ_HOST: "rabbitmq-service"
  RABBITMQ_PORT: "5672"
  RABBITMQ_EXCHANGE: "production"
  RABBITMQ_QUEUES: "orders,payments,notifications"
  RABBITMQ_ROUTING_KEYS: "orders.*,payments.*,notifications.*"

Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Make your changes
  4. Add tests for your changes
  5. Run the test suite (pytest)
  6. Commit your changes (git commit -m 'Add amazing feature')
  7. Push to the branch (git push origin feature/amazing-feature)
  8. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

Changelog

See CHANGELOG.md for a detailed list of changes and version history.

Acknowledgments

  • Built on top of the excellent pika library
  • Inspired by the need for simpler RabbitMQ management in Python applications
  • Thanks to the RabbitMQ team for creating an amazing message broker

Made with ❤️ for the Python community

Metadata

Release files for rabbitmq-easy 1.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for rabbitmq-easy 1.1.0
File Size Uploaded Provenance Actions
rabbitmq_easy-1.1.0.tar.gz 34.8 kB Without provenance Details for rabbitmq_easy-1.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for rabbitmq-easy 1.1.0
File Interpreter ABI Platform Actions
rabbitmq_easy-1.1.0-py3-none-any.whl Python 3 none any Details for rabbitmq_easy-1.1.0-py3-none-any.whl

Total release size: 53.9 kB

Release files / rabbitmq_easy-1.1.0.tar.gz

Download URL rabbitmq_easy-1.1.0.tar.gz
Size 34.8 kB
Tags Source
SHA-256 checksum
How to use checksums
6e7cecf8af3c10f392eb4e64c258195684a5790a8abcf51abe5802bc59fc5763
BLAKE2b-256 checksum
How to use checksums
bdc2896e976654655ff1cd45868597005e8867fd27f12d519330bba48e644302
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.11.0

Release files / rabbitmq_easy-1.1.0-py3-none-any.whl

Download URL rabbitmq_easy-1.1.0-py3-none-any.whl
Size 19.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c7c77af9307730ff31cd03570b4e6cb2aa319dbd9c92543c856a471467610ccc
BLAKE2b-256 checksum
How to use checksums
9f31a24e0469894ad1466cbfb8a6439ab5038baf5b4067d1634010136f6e69b1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.11.0

Security for rabbitmq-easy 1.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page