A Kafka broker for Dramatiq.
Project description
Uses kafka-python package to write a Kafka broker for dramatiq.
import dramatiq
from dramatiq.middleware import AgeLimit, TimeLimit, Callbacks, Pipelines, Prometheus, Retries
from dramatiq_kafka import KafkaBroker
broker = KafkaBroker(
bootstrap_servers="localhost:9092",
topic="testdramatiq_topic", # default is `default`
group_id="testdramatiq_consumergroup", # optional, default is `default`
middleware=[
Prometheus(),
AgeLimit(),
TimeLimit(),
Callbacks(),
Pipelines(),
Retries(min_backoff=1000, max_backoff=900000, max_retries=96),
],
)
dramatiq.set_broker(broker)
Use dramatiq to send a message to a Kafka topic (queue_name
) as usual:
message = Message(
queue_name="rita",
actor_name="email_customer",
args=("testing@gmail.com",),
kwargs={},
options={},
)
broker = get_broker()
broker.enqueue(message)
Write tasks as described in django_dramatiq's documentation (i.e. create dramatiq actors in the task.py files) and modify settings as shown in django_example
to get started. To send messages, view the views.py file for an example
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dramatiq_kafka-0.1.9.tar.gz
(2.5 kB
view hashes)
Built Distribution
Close
Hashes for dramatiq_kafka-0.1.9-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | deaf433753566f94e204d498e6c3e8ec8554e8a19cb522eb7270ea488024683a |
|
MD5 | 6f5251eab3da26b32d40618551c4c22c |
|
BLAKE2b-256 | 4ca87661afcdb008aa88a586fd7ebec109eb151b9803a4839c397e25acf74ce7 |