Skip to main content

Background Processing for Python 3.

Project description

remoulade

CircleCI PyPI version Documentation

A fast and reliable distributed task processing library for Python 3. Fork of dramatiq.io


Changelog: https://remoulade.readthedocs.io/en/latest/changelog.html
Documentation: https://remoulade.readthedocs.io


Installation

If you want to use it with RabbitMQ

    $ pipenv install 'remoulade[rabbitmq]'

or if you want to use it with Redis

   $ pipenv install 'remoulade[redis]'

Quickstart

  1. Make sure you've got RabbitMQ running, then create a new file called example.py:
from remoulade.brokers.rabbitmq import RabbitmqBroker
import remoulade
import requests
import sys

broker = RabbitmqBroker()
remoulade.set_broker(broker)


@remoulade.actor
def count_words(url):
    response = requests.get(url)
    count = len(response.text.split(" "))
    print(f"There are {count} words at {url!r}.")


broker.declare_actor(count_words)

if __name__ == "__main__":
    count_words.send(sys.argv[1])
  1. In one terminal, run your workers:
   $ remoulade example
  1. In another, start enqueueing messages:
   $ python3 example.py http://example.com
   $ python3 example.py https://github.com
   $ python3 example.py https://news.ycombinator.com

Visit the user guide to see more features!.

Dashboard

Check out SuperBowl a dashboard for real-time monitoring and administrating all your Remoulade tasks. See the current progress, enqueue, requeue, cancel and more ... Super easy to use !.

Kubernetes

Remoulade is tailored to run transparently in containers on Kubernetes and to make the most of their features. This does not mean it cannot run outside of Kubernetes ;)

Monitoring/Prometheus

Remoulade provides a Prometheus Middleware that exposes various metrics (processed messages, retries, fails etc.), Check it out.

Tracing/OpenTelemtry

Take a look at opentelemetry-instrumentation-remoulade if you want to enable tracing for Remoulade operations (enqueue, message processing, retries etc.). See here for an example. The library follows OpenTelemetry standards.

License

remoulade is licensed under the LGPL. Please see COPYING and COPYING.LESSER for licensing details.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

remoulade-0.55.0.tar.gz (86.2 kB view hashes)

Uploaded source

Built Distribution

remoulade-0.55.0-py3-none-any.whl (135.8 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page