Skip to main content
Pre-release

This release is a pre-release and may not be stable for production use.

gylo

A distributed task queue for Python with a Rust core. Postgres-first, async-native, one runtime dependency.

pip install --pre gylo
import gylo

app = gylo.Gylo()


@app.task
async def send_receipt(order_id: int) -> None:
    ...


async with pool.acquire() as conn, conn.transaction():
    order = await create_order(conn, ...)
    await send_receipt.enqueue(conn, order_id=order.id)
gylo migrate
gylo worker --app myapp:app --queue default

Why

The job commits with your data or not at all. enqueue takes the connection you are already holding, so the insert joins your transaction. If the order rolls back, so does the receipt. Every queue that talks to a separate broker has a window where one of the two survives and the other does not.

Nothing degrades quietly. A backend declares what it supports, and asking for something it cannot do stops the worker at startup rather than silently doing less. "Redis supports priorities" is technically true and practically false; gylo will not let that become your problem at 3am.

One process per core, by default. Task code holds the interpreter lock, so a single child uses a single core no matter how high concurrency goes. gylo runs one child per core and they coordinate through nothing but the queue.

What it does

Retries with exponential backoff and jitter · scheduled and delayed jobs · cron without leader election · priorities · unique jobs · keyed concurrency for multi-tenant fairness · results · cancellation · DAG workflows · durable steps, so a retry replays completed steps instead of repeating their side effects · per-task timeouts · graceful shutdown on SIGTERM · Prometheus metrics and a health endpoint.

Numbers

Measured on one 8-core machine with Postgres and Redis in a VM, against each library's own worker at its own defaults, 20,000 jobs. Reproduce with benches/end_to_end.py.

Drain Enqueue
dramatiq 99,272/s 104–119µs
gylo 74,250/s 27–37µs
taskiq 6,747/s 191–202µs
celery 2,021/s 243–260µs
arq 200/s (defaults) 351–383µs

Two honest caveats. dramatiq wins the drain because a list pop from a Redis that acknowledges before it persists is a cheaper operation than a durable state transition — the gap is in the storage layer, not the worker. And arq's default is a rate cap (100 jobs per 0.5s poll); opened up it reaches 1,784/s.

Under kill -9, gylo loses nothing: leases are reclaimed and the work runs again. That result, not the throughput, is the one worth caring about.

Status

Alpha, and released as a pre-release on purpose: pip install gylo will not pick this up without --pre, so nothing reaches a production deployment by accident. The engine is tested — 116 Rust and 61 Python tests, plus a chaos harness that kills workers and restarts Postgres — and the semantics are settled, but it has not run in production anywhere yet.

The Redis backend exists and is not yet reachable from the worker; use Postgres.

Requirements

Python 3.11+ and PostgreSQL 14+, on Linux or macOS.

Windows is not supported. The supervisor talks to its Python children over a Unix domain socket, and giving it a named-pipe transport is a piece of work rather than a build flag.

Licence

Apache-2.0.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gylo-0.1.0a1.tar.gz (82.5 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

gylo-0.1.0a1-py3-none-musllinux_1_2_x86_64.whl (4.0 MB view details)

Uploaded Python 3musllinux: musl 1.2+ x86-64

gylo-0.1.0a1-py3-none-musllinux_1_2_aarch64.whl (3.7 MB view details)

Uploaded Python 3musllinux: musl 1.2+ ARM64

gylo-0.1.0a1-py3-none-manylinux_2_28_aarch64.whl (3.7 MB view details)

Uploaded Python 3manylinux: glibc 2.28+ ARM64

gylo-0.1.0a1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.9 MB view details)

Uploaded Python 3manylinux: glibc 2.17+ x86-64

gylo-0.1.0a1-py3-none-macosx_11_0_arm64.whl (3.6 MB view details)

Uploaded Python 3macOS 11.0+ ARM64

gylo-0.1.0a1-py3-none-macosx_10_12_x86_64.whl (3.7 MB view details)

Uploaded Python 3macOS 10.12+ x86-64

File details

Details for the file gylo-0.1.0a1.tar.gz.

File metadata

  • Download URL: gylo-0.1.0a1.tar.gz
  • Upload date:
  • Size: 82.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for gylo-0.1.0a1.tar.gz
Algorithm Hash digest
SHA256 0971b7e8fda652bebe37e6d4ed78b660d549e7cbe3f82c1dd85953e7b508634b
MD5 74cb30589f022bfc481480c8feadc060
BLAKE2b-256 d48afa34007a7c6bd74d163ace335447a50b70ce1d4b168e5850c1934fbe17ec

See more details on using hashes here.

Provenance

The following attestation bundles were made for gylo-0.1.0a1.tar.gz:

Publisher: release.yml on globallstudent/gylo

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file gylo-0.1.0a1-py3-none-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for gylo-0.1.0a1-py3-none-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 df81962559ca7986566d3713f021176b22be083943220e6f58fa7a9a5e57681e
MD5 f53e45ae52996ee915a37d89c6fe92e9
BLAKE2b-256 78ebec73e558f8e50a444b0592bfcd7338a45959eadf4bde33c92a67de9bb971

See more details on using hashes here.

Provenance

The following attestation bundles were made for gylo-0.1.0a1-py3-none-musllinux_1_2_x86_64.whl:

Publisher: release.yml on globallstudent/gylo

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file gylo-0.1.0a1-py3-none-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for gylo-0.1.0a1-py3-none-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 fd4a7f24c2b336923c542f9b7bebf7f6415be14a8e961a8ce6c2939a6e7cca1a
MD5 741ae3aacf2fba637737d87d557f805c
BLAKE2b-256 67984703bab9bbe76b0572c66c86b4d2c6a9a20f78cf5517d132fb5f889d7991

See more details on using hashes here.

Provenance

The following attestation bundles were made for gylo-0.1.0a1-py3-none-musllinux_1_2_aarch64.whl:

Publisher: release.yml on globallstudent/gylo

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file gylo-0.1.0a1-py3-none-manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for gylo-0.1.0a1-py3-none-manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 3e98e74627295cf6b5a95346f63909ac59b7b40a6a5bbb25407c52e2e845983d
MD5 ce6f00eb9f63f85f8512750998b85fb4
BLAKE2b-256 af7dee336191538f5b2defce5be31cb2d76987ea75f1009c8bac27b691e1eb56

See more details on using hashes here.

Provenance

The following attestation bundles were made for gylo-0.1.0a1-py3-none-manylinux_2_28_aarch64.whl:

Publisher: release.yml on globallstudent/gylo

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file gylo-0.1.0a1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for gylo-0.1.0a1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 3708b1a2bc343a82a96e154222add221aa055fd709e2eb5113ca3dc9770e668b
MD5 51924f737c8d062079a258bd91202f27
BLAKE2b-256 e72dec5e1e40d23e837422b85799fc95d102457baf7911437e1211988fe99c68

See more details on using hashes here.

Provenance

The following attestation bundles were made for gylo-0.1.0a1-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on globallstudent/gylo

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file gylo-0.1.0a1-py3-none-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for gylo-0.1.0a1-py3-none-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 9fdbe783c44021ab54f1b6dffb90b126405f6f8ab362b770bd131bd3b16962d7
MD5 41b5b4541d9ee1bce29fb5d0349471a3
BLAKE2b-256 1bcc7098565f7e5a8c88488f38445093633eedb8e05a2bc3308d298d459b8a39

See more details on using hashes here.

Provenance

The following attestation bundles were made for gylo-0.1.0a1-py3-none-macosx_11_0_arm64.whl:

Publisher: release.yml on globallstudent/gylo

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file gylo-0.1.0a1-py3-none-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for gylo-0.1.0a1-py3-none-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 54c6b6d1f2680d559a917b253b741715a0a17d502fc68e7602d701c6f1343e86
MD5 579c812a7e6fa5640584e6a3ab220034
BLAKE2b-256 0fd05a99b8f5ef96f17a2bb72245e271a53a92e54ed519ca86a7bafd76866663

See more details on using hashes here.

Provenance

The following attestation bundles were made for gylo-0.1.0a1-py3-none-macosx_10_12_x86_64.whl:

Publisher: release.yml on globallstudent/gylo

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.1.0a1 This release

7 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page