Celery Redis Poll Backend
A specialized Redis backend for Celery that replaces the default pub/sub mechanism for task result retrieval with a polling-based approach.
Why Polling Instead of Pub/Sub?
The default Celery Redis backend uses Redis pub/sub for real-time task result notifications. While pub/sub provides immediate updates, it can face challenges in certain scenarios:
- Deadlocks in highly concurrent/multi-threaded workloads due to single-threaded nature of Redis and Celery clients.
- Higher overhead with
SUBSCRIBEchannels.
This backend provides a more robust alternative by using a polling mechanism instead.
Features
- Polling-Based Results: Replaces pub/sub with a polling mechanism for task result retrieval
- Compatible with Existing Code: Drop-in replacement for the standard Redis backend
- Configurable Polling: Adjust polling intervals and timeouts to match your needs
- Resource Efficient: Reduces Redis memory usage by eliminating pub/sub channels
Installation
pip install celery-redis-poll
After installation new backends are automatically registered with Celery:
redis+pollrediss+pollredis+cluster_pollrediss+cluster_poll
Usage
Configure your Celery application to use the polling backend:
from celery import Celery
app = Celery('your_app',
broker='redis://localhost:6379/0',
backend='redis+poll://localhost:6379/0')
For clustered Redis, use redis+cluster_poll.
Requirements
- Python >= 3.7
- Celery >= 5.0.0
- Redis >= 4.5.0
- celery-redis-cluster >= 0.1.6
Development
For development, install extra dependencies:
pip install celery-redis-poll[dev]
License
This project is licensed under the MIT License.
Contributing
Contributions are welcome! Please feel free to submit issues and pull requests.
Metadata
Release files for celery-redis-poll 0.1.12
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| celery_redis_poll-0.1.12.tar.gz | 4.6 kB | Without provenance | Details for celery_redis_poll-0.1.12.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| celery_redis_poll-0.1.12-py3-none-any.whl | Python 3 | none | any | Details for celery_redis_poll-0.1.12-py3-none-any.whl |
Total release size: 9.1 kB
Release files / celery_redis_poll-0.1.12.tar.gz
| Download URL | celery_redis_poll-0.1.12.tar.gz |
|---|---|
| Size | 4.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3b91d38f23fad179afbad5aeb26957608d00e55944b51fbd522d4a2e4b9db270
|
|
BLAKE2b-256 checksum How to use checksums |
3d35db08f41fdd4ee8d5495fe02d89f72b60051859f2cfb09cbb57b5c68145e0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.1
|
Release files / celery_redis_poll-0.1.12-py3-none-any.whl
| Download URL | celery_redis_poll-0.1.12-py3-none-any.whl |
|---|---|
| Size | 4.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
74854ae050fa0a1d7a374f477f422983f69dd4da841785c0179312da0240bf17
|
|
BLAKE2b-256 checksum How to use checksums |
c68565340f2566d37b5100a5d1bed4c66b67d0997c75f33ba12b2ecc99ded040
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.1
|
Security for celery-redis-poll 0.1.12
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.