Skip to main content

Distributed Task Queue with native asyncio support

Project description

celery-asyncio

Distributed task queue with native asyncio support. An asyncio-native rewrite of Celery, one of the most widely used distributed task systems in the Python ecosystem.

This project is exploratory. It is not affiliated with or endorsed by the Celery project. If you're looking for the official production-ready task queue, use the original Celery.

Features

  • Native asyncio worker with hybrid thread pool for mixed async/sync workloads
  • async def tasks run directly on the event loop, no thread overhead
  • Sync tasks run in a thread pool alongside async tasks in the same worker
  • Valkey/Redis and AMQP transports via kombu-asyncio
  • Celery Flower works out of the box for monitoring
  • Django 6.0 Tasks support via django-tasks-celery
  • Targeting Python 3.14t free-threading for true parallelism

Quick Start

from celery import Celery

app = Celery("tasks", broker="redis://localhost:6379/0")

@app.task
async def add(x, y):
    return x + y

@app.task
def multiply(x, y):
    return x * y
celery -A tasks worker --loglevel=info -E

Both add (async) and multiply (sync) run in the same worker.

Documentation

Full documentation at oliverhaas.github.io/celery-asyncio

Requirements

  • Python 3.14+
  • kombu-asyncio 6.0+
  • Valkey 8+ or Redis 7+ or RabbitMQ 4+

Acknowledgments

This project builds on the ideas and design of Celery and Kombu by Ask Solem and the Celery contributors.

License

BSD-3-Clause

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

celery_asyncio-6.0.0a3.tar.gz (295.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

celery_asyncio-6.0.0a3-py3-none-any.whl (342.1 kB view details)

Uploaded Python 3

File details

Details for the file celery_asyncio-6.0.0a3.tar.gz.

File metadata

  • Download URL: celery_asyncio-6.0.0a3.tar.gz
  • Upload date:
  • Size: 295.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for celery_asyncio-6.0.0a3.tar.gz
Algorithm Hash digest
SHA256 5350b4a78e74aefe2634b54cba8fa4de5ef7d14d22481bb3d98c5ec51f4012ae
MD5 0289f1ad8cfe417a8bf9de00ae578ea6
BLAKE2b-256 040d356d311db09dcbbb79b3f55cdec8a18bc22e69d9bb2c53ddec14af0075fb

See more details on using hashes here.

Provenance

The following attestation bundles were made for celery_asyncio-6.0.0a3.tar.gz:

Publisher: publish.yml on oliverhaas/celery-asyncio

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file celery_asyncio-6.0.0a3-py3-none-any.whl.

File metadata

File hashes

Hashes for celery_asyncio-6.0.0a3-py3-none-any.whl
Algorithm Hash digest
SHA256 c5737e1abef05304cff8dc79e18d7aed93dd5f082e9270bbe9b9f3fefefd7cea
MD5 2aaa5166c9053b542a86cb629729d34a
BLAKE2b-256 9bf9293f96627ab3eeea5a470bf323ad3d96b92480de5899b8388d1034a3ec03

See more details on using hashes here.

Provenance

The following attestation bundles were made for celery_asyncio-6.0.0a3-py3-none-any.whl:

Publisher: publish.yml on oliverhaas/celery-asyncio

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page