Skip to main content

Celery Redis Poll Backend

A specialized Redis backend for Celery that replaces the default pub/sub mechanism for task result retrieval with a polling-based approach.

Why Polling Instead of Pub/Sub?

The default Celery Redis backend uses Redis pub/sub for real-time task result notifications. While pub/sub provides immediate updates, it can face challenges in certain scenarios:

  • Deadlocks in highly concurrent/multi-threaded workloads due to single-threaded nature of Redis and Celery clients.
  • Higher overhead with SUBSCRIBE channels.

This backend provides a more robust alternative by using a polling mechanism instead.

Features

  • Polling-Based Results: Replaces pub/sub with a polling mechanism for task result retrieval
  • Compatible with Existing Code: Drop-in replacement for the standard Redis backend
  • Configurable Polling: Adjust polling intervals and timeouts to match your needs
  • Resource Efficient: Reduces Redis memory usage by eliminating pub/sub channels

Installation

pip install celery-redis-poll

After installation new backends are automatically registered with Celery:

  • redis+poll
  • rediss+poll
  • redis+cluster_poll
  • rediss+cluster_poll

Usage

Configure your Celery application to use the polling backend:

from celery import Celery

app = Celery('your_app',
             broker='redis://localhost:6379/0',
             backend='redis+poll://localhost:6379/0')

For clustered Redis, use redis+cluster_poll.

Requirements

  • Python >= 3.7
  • Celery >= 5.0.0
  • Redis >= 4.5.0
  • celery-redis-cluster >= 0.1.6

Development

For development, install extra dependencies:

pip install celery-redis-poll[dev]

License

This project is licensed under the MIT License.

Contributing

Contributions are welcome! Please feel free to submit issues and pull requests.

Metadata

Release files for celery-redis-poll 0.1.12

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for celery-redis-poll 0.1.12
File Size Uploaded Provenance Actions
celery_redis_poll-0.1.12.tar.gz 4.6 kB Without provenance Details for celery_redis_poll-0.1.12.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for celery-redis-poll 0.1.12
File Interpreter ABI Platform Actions
celery_redis_poll-0.1.12-py3-none-any.whl Python 3 none any Details for celery_redis_poll-0.1.12-py3-none-any.whl

Total release size: 9.1 kB

Release files / celery_redis_poll-0.1.12.tar.gz

Download URL celery_redis_poll-0.1.12.tar.gz
Size 4.6 kB
Tags Source
SHA-256 checksum
How to use checksums
3b91d38f23fad179afbad5aeb26957608d00e55944b51fbd522d4a2e4b9db270
BLAKE2b-256 checksum
How to use checksums
3d35db08f41fdd4ee8d5495fe02d89f72b60051859f2cfb09cbb57b5c68145e0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.13.1

Release files / celery_redis_poll-0.1.12-py3-none-any.whl

Download URL celery_redis_poll-0.1.12-py3-none-any.whl
Size 4.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
74854ae050fa0a1d7a374f477f422983f69dd4da841785c0179312da0240bf17
BLAKE2b-256 checksum
How to use checksums
c68565340f2566d37b5100a5d1bed4c66b67d0997c75f33ba12b2ecc99ded040
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.13.1

Security for celery-redis-poll 0.1.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page