Polyqueue
Async-first Python job queue that keeps Postgres as the authoritative job state while supporting Redis, SQS, Azure Service Bus, PGMQ, and an in-process broker. Handlers are typed via Pydantic. Return values are persisted to the same Postgres DB — no separate result backend required.
| Dimension | Polyqueue | Celery | Taskiq |
|---|---|---|---|
| Primary model | Small queue runtime with Postgres as source of truth plus pluggable broker adapters | Mature distributed task queue with broad ecosystem/tooling | Async-first distributed task runner with modular brokers/extensions |
| Async support | Yes, async-first. Worker, handlers, DB access, and adapters are designed around asyncio | Not primarily async-first in overall design | Yes, async-first |
| Built-in broker/backend support | Redis, SQS, Azure Service Bus, PGMQ, none (in-process) | Commonly used with Redis/RabbitMQ and other broker/result-backend combinations | Broker-based, typically via pluggable integrations |
| Redis support | Yes | Yes | Yes, depending on broker plugin/config |
| SQS support | Yes | Yes | Possible via broker integration/plugin, but not the default mental model |
| Azure Service Bus support | Yes | Not a standard first-class default choice in the same way as Redis/RabbitMQ | Possible via ecosystem extension/custom broker, but not a core default pairing |
| No-broker / in-process mode | Yes via queue_backend="none" |
Not the usual model | Not the main model |
| Source of truth for job state | Postgres | Broker/backend-centric task system, optionally paired with result backends | Broker/message-centric |
| Pydantic support | Yes — enqueue() accepts BaseModel, dispatch() validates annotated payload models via model_validate() |
Available via explicit task-side support | Works naturally with typed task signatures and model parsing patterns |
| Pydantic behavior | Strict: model serialised to dict on enqueue, model_validate() on dispatch, ValidationError => TerminalError (no retry) |
Usually task-side validation/conversion | More ergonomic/best-effort style than strict contract-first behavior |
| Result persistence | Postgres jobs.result JSONB — same DB already tracking job state, no extra infrastructure |
Requires a separate result backend (Redis, DB, RPC, etc.) configured explicitly; off by default | Requires a separate result backend plugin (e.g. taskiq-redis); pluggable but extra infrastructure |
| Multi-worker distribution | Yes. Workers compete on claim; one worker owns the in-flight delivery | Yes | Yes |
| Worker identity tracking | Yes — polyqueue_workers registry: worker_id, hostname, pid, current job, heartbeat, status (running/stopped/dead); stale workers reaped by maintenance loop |
Stronger built-in worker identity/inspection model | Workers exist, but less built-in cluster observability than Celery |
| In-flight ownership model | Redis lease / SQS receipt handle + visibility timeout / Azure SB lock token + lock renewal | Broker/worker-managed | Broker/worker-managed |
| Recovery when worker crashes | Yes, via lease expiry / visibility timeout / lock expiry and redelivery/reclaim | Yes | Yes |
| Detecting "alive but wedged" jobs | Basic — progress_heartbeat_at + timeout_at in DB; maintenance backstop reconciles timed-out jobs |
Better operational story; mature controls exist | Typically needs app/broker-level design |
| Time limits | Yes — per-job max_run_seconds, cooperative timeout via asyncio.wait_for, DB backstop, three strategies (retry/fail/ignore) |
More mature built-in support | Less central than in Celery's ops model |
| Operational maturity | Focused, understandable, smaller surface area | Most mature | Lighter-weight, async-friendly |
| Best fit | Internal service jobs where you want simplicity, explicit semantics, and control | Large-scale general background job platform | Async Python apps that want typed/ergonomic task execution |
Summary
- Brokers: Redis, SQS, Azure Service Bus, PGMQ, or in-process (
none). Postgres always holds the authoritative job state regardless of broker. - Pydantic-first: handlers declare typed payloads via
@register;ValidationErroron dispatch becomesTerminalError(no retry). - Result persistence in Postgres: handler return values (
BaseModel, dict, or primitive) are saved tojobs.resultJSONB — no separate result backend needed. Fetch viaget_job_result,get_job_result_typed, orget_job_result_typed_via_registry. Not available with thenone/in-process backend. - Time limits: per-job
max_run_secondsenforced viaasyncio.wait_for, with three strategies (retry/fail/ignore) and a maintenance-loop DB backstop. - Worker registry:
polyqueue_workerstable tracks id, hostname, pid, current job, and heartbeat. Stale workers are reaped todeadby the maintenance loop. - Metrics hook: pluggable
MetricsHookfor Prometheus, CloudWatch, or OpenTelemetry. Pass tomain(metrics=...). - Admin CLI:
polyqueue inspect stats|workers|jobsandpolyqueue admin requeue|fail. - Smaller surface area than Celery — no admin UI or auto-scaling built-in.
1 maintainer
Metadata
1 maintainer
Release files for polyqueue 0.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| polyqueue-0.0.1.tar.gz | 143.1 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| polyqueue-0.0.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 213.1 kB
Release files / polyqueue-0.0.1.tar.gz
| Download URL | polyqueue-0.0.1.tar.gz |
|---|---|
| Size | 143.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6e2ff82a4705fbd894e279403e931d1b3d11864668524fe1ddd700c6236cbc23
|
|
BLAKE2b-256 checksum How to use checksums |
54090159ec5d7ccbd6eb535c17a813790d78eaefc6faccf9341bfcc1a31e105d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.26 {"installer":{"name":"uv","version":"0.9.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / polyqueue-0.0.1-py3-none-any.whl
| Download URL | polyqueue-0.0.1-py3-none-any.whl |
|---|---|
| Size | 70.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a9e6a05cef193d0541d70df675442a75577e3d7e0d31bad7680ce686eedddab3
|
|
BLAKE2b-256 checksum How to use checksums |
e29e7ac05612ef150cef1196e8364c3784adeca47bc97fa8c2b624467ad2eb3e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.26 {"installer":{"name":"uv","version":"0.9.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for polyqueue 0.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
