dbworker

Your database rows are already the queue.

A portable, single-script background worker for SQLAlchemy. No broker, no enqueue, no duplicate job model.

Celery / Huey / Graphile Worker:
    event → job → worker

dbworker:
    row → worker

Requires Python 3.12+. Install into your application:

poetry add dbworker

Usage

Register a handler for an existing SQLAlchemy model. DBWorker discovers eligible rows, runs the handler in a child process, and commits its database changes with the execution status.

from sqlalchemy import select
from sqlalchemy.orm import Session
from dbworker import Coordinator, Finished

coordinator = Coordinator(session_factory, database_url=database_url)

@coordinator.transactional_worker(
    name="process",
    source=YourModel,
    eligible=lambda: (
        select(YourModel)
        .where(YourModel.your_bool_property.is_(True))
        .order_by(YourModel.id)
    ),
    concurrency=4,
)
def process(source: YourModel, session: Session) -> Finished:
    # Apply your application logic using the supplied session.
    return Finished()

if __name__ == "__main__":
    coordinator.create_worker_tables()
    coordinator.start()

YourModel, session_factory, and database_url come from your application. eligible selects which rows can be claimed and in what order. It is optional: omitting it selects from the whole model. DBWorker automatically excludes completed, failed, and actively claimed work. Return Finished() when processing is complete; no enqueue call is needed.

See the Usage Instructions for eligibility queries, incremental processing, service startup and shutdown, status tracking, dependencies, failure handling, and configuration.

Examples

• Image deduplication: independent FastAPI and worker services, artifact building, paged comparisons, and worker dependencies.
• Redis + Celery equivalent.
• Benchmarks with structured JSON results.

License

MIT © 2026 Ziyang Song.

Benchmark Results

Median wall time in seconds; bold marks the faster backend. Each experiment runs in a fresh Docker container on its own GitHub-hosted Ubuntu VM. Fixed profile: 8 producers, a 60-second scheduled submission window and 8 total execution slots; image hash building uses one native bulk import. Three repetitions per backend. Actual schedule delays and backlog are recorded in JSON; this is one load point, not a maximum-capacity search.

Run: github-38165895322-1. Details and scope · JSON results. Scoped application workloads; Sentry uses historical 24.1.0. Wall time includes the fixed submission window, so a ratio near 1 does not prove equal processing capacity. Task P95 starts at publication; peak outstanding counts already published tasks. Ratios above 1 favor DBWorker; no cross-project average is computed.