Skip to main content

ThreadWeave Python

The official Python runtime for ThreadWeave.

Write idiomatic Python. Execute anywhere.


⚠️ Project status

ThreadWeave Python is currently in active design and early development.

The API shown below represents the intended developer experience and may evolve before the first public release.


Why?

Most distributed task queues were designed long before modern Python.

Today we write applications with:

  • FastAPI
  • async/await
  • asyncio
  • SQLAlchemy Async
  • httpx
  • AI workloads
  • GPUs

Yet background execution often forces developers back into synchronous workers, complex concurrency models, or infrastructure-specific code.

ThreadWeave takes a different approach.

The execution engine is written in Rust.

The Python runtime focuses exclusively on providing a natural Python API.

Everything else—scheduling, retries, supervision, observability, distributed execution, resource allocation—is handled by the ThreadWeave engine.


A familiar API

If you know FastAPI, this should feel natural.

from threadweave import ThreadWeave

app = ThreadWeave()


@app.task
async def hello(name: str) -> str:
    return f"Hello {name}"

No worker configuration.

No broker-specific code.

No infrastructure inside your business logic.


Native async

Async is not an afterthought.

It is a first-class citizen.

@app.task
async def generate_embeddings(document_id: UUID):

    document = await database.documents.get(document_id)

    embedding = await embedding_model.embed(document.text)

    await database.embeddings.save(
        document.id,
        embedding,
    )

No event-loop hacks.

No synchronous wrappers.

No hidden thread pools.


Resources instead of workers

Traditional task queues schedule work on workers.

ThreadWeave schedules work on resources.

@app.task(
    resources={
        "cpu": 4,
        "memory": "8Gi",
        "gpu": 1,
    }
)
async def train_model(dataset: Dataset):
    ...

The application describes what it needs.

The scheduler decides where it should run.


Retry policy

@app.task(
    retries=5,
    retry_delay="30s",
)
async def send_email(message):
    ...

Retry strategies belong to the infrastructure.

Not to your business logic.


Timeouts

@app.task(
    timeout="10m",
)
async def process_video(video):
    ...

Scheduling

@app.task(
    schedule="0 * * * *",
)
async def cleanup():
    ...

Or

await cleanup.delay()

Or

await cleanup.delay(
    countdown="5m",
)

Resource-aware AI workloads

AI jobs often require GPUs.

Instead of manually routing queues:

@app.task(
    resources={
        "gpu": 2,
        "gpu_memory": "40Gi",
    }
)
async def run_llm(prompt):
    ...

The scheduler automatically selects a compatible node.


Pipelines

Tasks compose naturally.

document = await download(url)

text = await extract_text(document)

summary = await summarize(text)

await publish(summary)

Distributed execution should not force a different programming model.


Events everywhere

Every execution emits structured events.

TaskSubmitted

↓

TaskScheduled

↓

ResourcesAllocated

↓

TaskStarted

↓

TaskCompleted

Observability is built into the architecture.

Not added later.


Compare with Celery

Celery

@app.task
def generate_pdf(document_id):

    document = load(document_id)

    pdf = build_pdf(document)

    save(pdf)

Scaling often requires thinking about:

  • queues
  • routing
  • workers
  • concurrency
  • pools
  • broker tuning

ThreadWeave

@app.task(
    resources={
        "cpu": 2,
        "memory": "4Gi",
    },
    timeout="5m",
)
async def generate_pdf(document_id):

    document = await repository.get(document_id)

    pdf = await renderer.render(document)

    await storage.save(pdf)

Business code stays focused on the business.

Infrastructure stays inside ThreadWeave.


Long-running tasks

Minutes.

Hours.

Days.

ThreadWeave is designed to supervise long-running executions, survive failures, and recover from crashes without sacrificing observability.


Built for extension

The Python runtime is only one runtime.

The ThreadWeave protocol allows additional runtimes to be implemented without changing the core engine.

Future runtimes may include:

  • JavaScript / TypeScript
  • Java
  • Go
  • .NET
  • WebAssembly

Philosophy

ThreadWeave is not a Python task queue.

It is a distributed execution platform whose first runtime happens to be Python.

Python developers get an idiomatic API.

The engine takes care of everything else.


Current status

  • RFCs in progress
  • Architecture under design
  • Rust core in development
  • Python runtime specification in progress

Contributions, discussions and feedback are welcome.

Metadata

Release files for threadweave-python 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for threadweave-python 0.1.0
File Size Uploaded Provenance Actions
threadweave_python-0.1.0.tar.gz 22.2 kB With provenance Details for threadweave_python-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for threadweave-python 0.1.0
File Interpreter ABI Platform Actions
threadweave_python-0.1.0-py3-none-any.whl Python 3 none any Details for threadweave_python-0.1.0-py3-none-any.whl

Total release size: 48.3 kB

Release files / threadweave_python-0.1.0.tar.gz

Download URL threadweave_python-0.1.0.tar.gz
Size 22.2 kB
Tags Source
SHA-256 checksum
How to use checksums
87b7243b42d4b759510c7e1724ee7f3ddad44b071195b2aef8e3a42d8c82eb3f
BLAKE2b-256 checksum
How to use checksums
56b82961924bd7bc4af689e64685385b108516114d379c3445294064f7a47ec9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 4, 2026.

Transparency log

Release files / threadweave_python-0.1.0-py3-none-any.whl

Download URL threadweave_python-0.1.0-py3-none-any.whl
Size 26.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
55dc2217289e5fae9951a52599585380d2514a750c29166b850fb896a237fac5
BLAKE2b-256 checksum
How to use checksums
caab1a4a6bdf249e6f3c2c38cb47c397291596eecf6bc82487d15137cff66a6e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 4, 2026.

Transparency log

Security for threadweave-python 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for threadweave_python-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 4, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page