django-outbox-pattern 0.3.0

A django application to make it easier to use the transactional outbox pattern

Django outbox pattern

A django application to make it easier to use the transactional outbox pattern

Installation

Install django-outbox-pattern with pip

pip install django-outbox-pattern

Add to settings

# settings.py

INSTALLED_APPS = [
    "django_outbox_pattern",
]

DJANGO_OUTBOX_PATTERN = {
    "DEFAULT_STOMP_HOST_AND_PORTS": [("127.0.0.1", 61613)],
    "DEFAULT_STOMP_USERNAME": "guest",
    "DEFAULT_STOMP_PASSCODE": "guest",
}

Usage/Examples

The publish decorator adds the outbox table to the model. publish accepts list of Config. The Config accepts four params the destination which is required, fields which the default are all the fields of the model, serializer which by default adds the id in the message to be sent and version which by default is v1.

Note: fields and serializer are mutually exclusive, serializer overwrites the fields.

The Config typing

from typing import List
from typing import NamedTuple
from typing import Optional

class Config(NamedTuple):
    destination: str
    fields: Optional[List[str]] = None
    serializer: Optional[str] = None
    version: Optional[str] = "v1"

Only destination in config

from django.db import models
from django_outbox_pattern.decorators import Config
from django_outbox_pattern.decorators import publish


@publish([Config(destination='/topic/my_route_key')])
class MyModel(models.Model):
    field_one = models.CharField(max_length=100)
    field_two = models.CharField(max_length=100)

This generates the following data to be sent.

producer.send(destination='/topic/my_route_key.v1', body='{"id": 1, "field_one": "Field One", "field_two": "Field Two"}')

With destinations and fields

from django.db import models
from django_outbox_pattern.decorators import Config
from django_outbox_pattern.decorators import publish


@publish([Config(destination='/topic/my_route_key', fields=["field_one"])])
class MyModel(models.Model):
    field_one = models.CharField(max_length=100)
    field_two = models.CharField(max_length=100)

This generates the following data to be sent.

producer.send(destination='/topic/my_route_key.v1', body='{"id": 1, "field_one": "Field One"}')

With destinations and serializer

from django.db import models
from django_outbox_pattern.decorators import Config
from django_outbox_pattern.decorators import publish


@publish([Config(destination='/topic/my_route_key', serializer='my_serializer')])
class MyModel(models.Model):
    field_one = models.CharField(max_length=100)
    field_two = models.CharField(max_length=100)

    def my_serializer(self):
        return {
            "id": self.id,
            "one": self.field_one,
            "two": self.field_two
        }

This generates the following data to be sent.

producer.send(destination='/topic/my_route_key.v1', body='{"id": 1, "one": "Field One", "two": "Field Two"}')

With multi destinations and serializers

from django.db import models
from django_outbox_pattern.decorators import Config
from django_outbox_pattern.decorators import publish


@publish([
    Config(destination='/topic/my_route_key_1', serializer="my_serializer_1"),
    Config(destination='/topic/my_route_key_2', serializer="my_serializer_2"),
])
class MyModel(models.Model):
    field_one = models.CharField(max_length=100)
    field_two = models.CharField(max_length=100)

    def my_serializer_1(self):
        return {
            "id": self.id,
            "one": self.field_one,
        }

    def my_serializer_2(self):
        return {
            "id": self.id,
            "two": self.field_two
        }

This generates the following data to be sent.

producer.send(destination='/topic/my_route_key_1.v1', body='{"id": 1, "one": "Field One"}')
producer.send(destination='/topic/my_route_key_2.v1', body='{"id": 1, "two": "Field Two"}')

Publish/Subscribe commands

Publish command

To send the messages added to the outbox table it is necessary to start the producer.

python manage.py publish

Subscribe command

Django outbox pattern also provides a consumer that can be used to receive outgoing messages.

Create a function that receives an instance of django_outbox_pattern.payloads.Payload

# callbacks.py

def callback(payload):
    try:
        # Do anything
        payload.ack()
    except Exception:
        # Nack is automatically called in case of errors, but you might want to handle the error in another way
        payload.nack()

python manage.py subscribe 'dotted.path.to.callback` 'destination'

Settings

DEFAULT_CONNECTION_CLASS

The stomp.py class responsible for connecting to the broker. Default: stomp.StompConnection12

DEFAULT_CONSUMER_LISTENER_CLASS

The consumer listener class. Default: django_outbox_pattern.listeners.ConsumerListener

DEFAULT_GENERATE_HEADERS

A function to add headers to the message. Default: django_outbox_pattern.headers.generate_headers

DEFAULT_MAXIMUM_BACKOFF:

Maximum wait time for connection attempts in seconds. Default: 3600 (1 hour)

DEFAULT_MAXIMUM_RETRY_ATTEMPTS

Maximum number of message resend attempts. Default: 50

DEFAULT_PAUSE_FOR_RETRY

Pausing for attempts to resend messages in seconds. Defualt: 240 (4 minutes)

DEFAULT_WAIT_RETRY

Time between attempts to send messages after the pause. Default: 60 (1 minute)

DEFAULT_PRODUCER_LISTENER_CLASS:

The producer listener class. Default: django_outbox_pattern.listeners.ProducerListener

DEFAULT_STOMP_HOST_AND_PORTS

List of host and port tuples to try to connect to the broker. Default [("127.0.0.1", 61613)]

DEFAULT_STOMP_QUEUE_HEADERS

Headers for queues. Default: {"durable": "true", "auto-delete": "false", "prefetch-count": "1"}

DEFAULT_STOMP_HEARTBEATS

Time tuples for input and output heartbeats. Default: (10000, 10000)

DEFAULT_STOMP_VHOST

Virtual host. Default: "/"

DEFAULT_STOMP_USERNAME

Username for connection. Default: "guest"

DEFAULT_STOMP_PASSCODE

Password for connection. Default: "guest"