Django-based Message Broker for DASF

A Django-based message broker implementing the messaging protocol of the Data Analytics Software Framework (DASF), inspired by Apache Pulsar.

The broker connects clients (such as a web frontend or a python script) with backend modules that run the requested functions. All of them talk to the broker via websockets, so that backend modules can run anywhere and come and go at any time. It features

• topics with fine-grained permissions, managed in the Django admin and with the dasf_topic command,
• subscriptions in the style of Apache Pulsar (exclusive, failover, shared, spread) with acknowledgements, redelivery of missed messages and a configurable storage of messages,
• short-lived tokens that allow backend modules to act on behalf of the user who sent a request,
• request quotas per user and topic,
• pings and a status endpoint to monitor backend modules,
• an optional web frontend that renders a page with forms, progress reports and results for every topic, and
• a docker-based deployment template.

Deployment

The recommended way to run the broker is the dasf-broker-docker-template. It generates a docker compose setup with daphne, postgres, redis and nginx. See the deployment docs for how to use it.

Installation

To add the broker to your own Django project, install this package in a dedicated python environment via

python -m venv venv
source venv/bin/activate
pip install "dasf-broker-django[drf,jobs]"

and follow the installation instructions in the docs. To use this in a development setup, clone the source code from gitlab, start the development server and make your changes:

git clone https://codebase.helmholtz.cloud/dasf/dasf-broker-django
cd dasf-broker-django
python -m venv venv
source venv/bin/activate
make dev-install
make build-static  # download the files of the frontend

You find the full documentation, including a getting started guide, at https://dasf.readthedocs.io/projects/message-broker/en/latest/.

Technical note

This package has been generated from the template https://codebase.helmholtz.cloud/hcdc/software-templates/django-app-template.git.

See the template repository for instructions on how to update the skeleton for this package.

License information

Copyright © 2022-2025 Helmholtz-Zentrum hereon GmbH

Code files in this repository are licensed under the EUPL-1.2, if not stated otherwise in the file.

Documentation files in this repository are licensed under CC-BY-4.0, if not stated otherwise in the file.

Supplementary and configuration files in this repository are licensed under CC0-1.0, if not stated otherwise in the file.

Please check the header of the individual files for more detailed information.

License management

License management is handled with reuse. If you have any questions on this, please have a look into the contributing guide or contact the maintainers of dasf-broker-django.