dramatiq_azure 0.3.1

Azure Queue Brokers for Dramatiq

dramatiq-azure

A Dramatiq broker that can be used with Microsoft Azure queue services.

Heavily inspired by Dramatiq SQS, this package currently implements a broker for Azure Storage Queue. An implementation for Azure Service Bus is planned... eventually.

Installation

    pip install dramatiq-azure
    pip install dramatiq-azure[identity]  # for passwordless authentication

Usage

ASQBroker

import dramatiq

from dramatiq.middleware import AgeLimit, TimeLimit, Callbacks, Pipelines, Prometheus, Retries
from dramatiq_azure import ASQBroker


broker = ASQBroker(
    dead_letter=True,
    middleware=[
        Prometheus(),
        AgeLimit(),
        TimeLimit(),
        Callbacks(),
        Pipelines(),
        Retries(min_backoff=1000, max_backoff=900000, max_retries=96),
    ],
)
dramatiq.set_broker(broker)

Authentication

The following authentication methods are supported by the broker:

1. Connection string based: AZURE_STORAGE_CONNECTION_STR environment variable must be set. If this variable is not set, passwordless authentication will be used. Creating a connection string for your Azure account is documented here.
2. Passwordless (token-based) authentication (Recommended): AZURE_STORAGE_ACCOUNT_NAME environment variable must be set.

The list of other mandatory variables depends on where the app is being run. More information can be found here.

Environment variables

The following environment variables can be used to configure the broker:

• AZURE_STORAGE_CONNECTION_STR: Azure Storage connection string;
• AZURE_STORAGE_ACCOUNT_NAME/AZURE_ACCOUNT_NAME: Azure Storage account name;
• AZURE_ENDPOINT_SUFFIX: Azure Storage endpoint suffix;
• AZURE_SSL: Whether to use SSL for the connection;
• AZURE_QUEUE_ACCOUNT_URL: Azure Storage account URL;
• DRAMATIQ_ASQ_MIN_TIMEOUT: The minimum time to wait between polls in second.

Tests

Tests require a running Azurite instance. You can easily launch azurite through Docker.

docker run -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite

Run the test suite

pytest

Contributions

Found an itch you know how to scratch? PR welcome (just remember to read the contribution guide) !