Roost

Postgres-backed background job queue for Python — Oban for Pythonistas.

🚧 Alpha — under active development. APIs may change before 1.0. Pin exactly.

Why Roost?

• One less piece of infra. No Redis, no RabbitMQ. Your existing Postgres is the queue.
• Transactional enqueue. INSERT INTO roost.jobs ... commits in your transaction. Jobs cannot orphan or vanish.
• Battle-tested concurrency. SELECT ... FOR UPDATE SKIP LOCKED for safe parallel workers.
• Real-time wakeups. LISTEN / NOTIFY — no polling overhead, sub-second pickup.
• Sync and async, first-class. Twin facades over a single SQL surface — Django/Flask + FastAPI/Starlette without a glue layer.
• Crash-tolerant. Heartbeats + orphan reaper recover jobs from SIGKILL'd workers automatically.
• Polished dashboard. roost-web mounts in three lines, live updates via SSE, no Node.js toolchain.

Real numbers (from a laptop)

Bulk enqueue:    15,289 jobs/sec   single async connection
Sustained drain:    822 jobs/sec   16 workers, local PG, noop handler
Dispatch overhead:  p50 3 ms       p99 51 ms — pure Roost overhead
                                   per job (excludes handler runtime)

Run bench/throughput.py against your own Postgres to see what you get.

Quickstart

pip install pgroost                # install
export ROOST_DSN=postgresql://user:pass@localhost/app
roost init --apply                 # CLI command stays `roost`

Distribution vs import name: the PyPI distribution is pgroost (the bare roost name is taken on PyPI). You still import roost and call the roost CLI — only the install line differs.

# tasks.py
from roost import job

@job("send_welcome_email", queue="emails", max_attempts=5, timeout_seconds=30)
async def send_welcome_email(user_id: int) -> None:
    ...

# inside a FastAPI / Starlette / Django handler — same conn, same txn
async with pool.acquire() as conn, conn.transaction():
    user_id = await conn.fetchval("INSERT INTO users ... RETURNING id")
    await roost.enqueue(send_welcome_email, args={"user_id": user_id}, conn=conn)
# both rows commit together — or roll back together. That's the whole point.

roost run --module tasks --queues emails,default --concurrency 8
roost doctor                       # health check
roost status                       # counts per state per queue

Read the docs and the recipes for more.

Feature matrix

Compatibility

• Python: 3.10, 3.11, 3.12, 3.13.
• PostgreSQL: 13, 14, 15, 16. Tested every commit on the full matrix.
• Drivers: asyncpg (async), psycopg[binary] (sync).
• Hosts: any ASGI app — FastAPI, Starlette, Litestar, Quart. Django and Flask via the contrib helpers.

Test suite

• 163 tests, 86% coverage, real Postgres via testcontainers (no mocks of the DB layer).
• Run locally: uv run pytest -q. With coverage: uv run --with pytest-cov pytest --cov=src/roost --cov-report=term.
• Override the test image: ROOST_TEST_PG_IMAGE=postgres:15-alpine uv run pytest -q.

Compared to other queues

See the comparison page for a side-by-side feature matrix vs Celery, RQ, dramatiq, arq, procrastinate, and pgqueuer.

Quick read: pick Roost if you already run Postgres and want transactional enqueue + a polished dashboard. Pick Celery / RQ if you don't run Postgres and have Redis already. Pick procrastinate / pgqueuer if you want the same Postgres-only foundation with a smaller feature surface.

Project structure

Examples

The examples/ directory has six runnable patterns:

• docker/ — one-command stack (Postgres + worker + dashboard).
• fastapi_app/ — transactional enqueue inside a FastAPI request, wait_for for sync results.
• scheduled_reports/ — daily cron with timezone, weekly UTC rollup.
• fanout_join/ — N parallel children + a single aggregator gated on depends_on.
• etl_pipeline/ — chained extract → transform → load with rate limits and concurrency caps.
• plain_python.py — smallest possible enqueue + worker.

Each has its own README; start there.

Contributing

Read CONTRIBUTING.md. Bugs + feature requests via GitHub issues. Security disclosures: see SECURITY.md. All participants are expected to follow the Code of Conduct.

License

MIT — see LICENSE.