Skip to main content

Arroyo is a Python library for working with streaming data.

Project description

Arroyo

Arroyo is a library to build streaming applications that consume from and produce to Kafka.

Arroyo consists of three components:

  • Consumer and producer backends
    • The Kafka backend is a wrapper around the librdkafka client, and attempts to simplify rebalancing and offset management even further
    • There is also an in memory and a file based consumer and producer implementation that can be used for testing
  • A strategy interface
    • Arroyo includes a number of pre-built strategies such as RunTask, Filter, Reduce, CommitOffsets and more.
    • Users can write their own strategies, though in most cases this should not be needed as the library aims to provide generic, reusable strategies that cover most stream processing use cases
    • Strategies can be chained together to form complex message processing pipelines.
  • A streaming engine which manages the relationship between the consumer and strategies
    • The StreamProcessor controls progress by the consumer and schedules work for execution by the strategies.

All documentation is in the docs directory. It is hosted at https://getsentry.github.io/arroyo/ and can be built locally by running make docs

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sentry_arroyo-2.27.0.tar.gz (86.8 kB view details)

Uploaded Source

Built Distribution

sentry_arroyo-2.27.0-py3-none-any.whl (113.1 kB view details)

Uploaded Python 3

File details

Details for the file sentry_arroyo-2.27.0.tar.gz.

File metadata

  • Download URL: sentry_arroyo-2.27.0.tar.gz
  • Upload date:
  • Size: 86.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.2

File hashes

Hashes for sentry_arroyo-2.27.0.tar.gz
Algorithm Hash digest
SHA256 0f54a3aa8bf26a3ff6229b741f62e8767b55de6fa90f8f1b58e29edb22e67090
MD5 19871837dc1f0380a49783a2fa06aa7a
BLAKE2b-256 9d9fdffe5479007cc352baba1b808f11884d13347e33f0249396efd7eea9c92e

See more details on using hashes here.

File details

Details for the file sentry_arroyo-2.27.0-py3-none-any.whl.

File metadata

  • Download URL: sentry_arroyo-2.27.0-py3-none-any.whl
  • Upload date:
  • Size: 113.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.11.2

File hashes

Hashes for sentry_arroyo-2.27.0-py3-none-any.whl
Algorithm Hash digest
SHA256 eee650daa7e9a9ed4ceaea23ca78e943314d898173f01f2d7e9fd1984cbb6fa4
MD5 6827508738d9407ca4c9558eb6f0291a
BLAKE2b-256 5f6633438aff0ad616c17bec9d18ca9279a1bc6e69f29d47df4d42a7778f4ae5

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page