Relé makes integration with Google PubSub easier.
Project description
Relé makes integration with Google PubSub easier and is ready to integrate seamlessly into any Django project.
Motivation and Features
The Publish-Subscribe pattern and specifically the Google Cloud PubSub library are very powerful tools but you can easily cut your fingers on it. Relé makes integration seamless by providing Publisher, Subscriber and Worker classes with the following features:
- A
publish
function:- Singleton: Avoids instantiating a
PublisherClient
every time you publish. Otherwise, it will result in a memory leak because the transport is not closed by the Google library.
- Singleton: Avoids instantiating a
- A
sub
decorator to declare subscribers:- In-built acks
- Automatic subscription topic naming
Publisher
andSubscription
classes:- Automatic gc client configuration and building of topic and subscription paths
- A
Worker
class:- In-built DB connection management so open connections don't increase over time
- A
python manage.py runrele
management command- Automatic creation of Subscriptions
- Subscription auto-discovery
What's in the name
"Relé" is Spanish for relay, a technology that has played a key role in history in the evolution of communication and electrical technology, including the telegraph, telephone, electricity transmision, and transistors.
Quickstart
Add it to your INSTALLED_APPS
:
INSTALLED_APPS = [
...,
'rele',
]
You'll also need to set up two variables with the Google Cloud credentials:
RELE_GC_CREDENTIALS
and RELE_GC_PROJECT_ID
.
You can use RELE_SUB_PREFIX
to add a prefix to all your subscription names, for
example, with your project name.
NOTE: Ensure that CONN_MAX_AGE
is set to 0 in your worker. The Django default value is 0.
In other words, make sure CONN_MAX_AGE
is not set explicitly in the environment where you run
python manage.py runrele
.
Usage
Publishing
import rele
def myfunc():
# ...
rele.publish(topic='lets-tell-everyone',
data={'foo': 'bar'},
myevent='arrival')
Declaring Subscriptions
Just decorate your function with the sub
decorator:
# your_app.subs.py
from rele import sub
@sub(topic='lets-tell-everyone')
def sub_function(data, **kwargs):
event = kwargs.get('myevent')
print(f'I am a task doing stuff with an event: {event}')
Subscription suffix
In order for multiple subscriptions to consume from the same topic, you'll want to add a unique suffix to the subscriptions, so they both listen to all the gossip going around.
@sub(topic='lets-tell-everyone', suffix='sub1')
def purpose_1(data, **kwargs):
pass
@sub(topic='lets-tell-everyone', suffix='sub2')
def purpose_2(data, **kwargs):
pass
Running the worker in a process
In your worker, you can run python manage.py runrele
. Once subscribed to
the topic, in another process you can run the publish
function. Your subscription process
should print out the message.
Running Tests
Does the code actually work?
make test
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.