Skip to main content

ThreadWeave Python

The official Python runtime for ThreadWeave.

Write idiomatic Python. Execute anywhere.


⚠️ Project status

ThreadWeave Python is currently in active design and early development.

The API shown below represents the intended developer experience and may evolve before the first public release.


Why?

Most distributed task queues were designed long before modern Python.

Today we write applications with:

  • FastAPI
  • async/await
  • asyncio
  • SQLAlchemy Async
  • httpx
  • AI workloads
  • GPUs

Yet background execution often forces developers back into synchronous workers, complex concurrency models, or infrastructure-specific code.

ThreadWeave takes a different approach.

The execution engine is written in Rust.

The Python runtime focuses exclusively on providing a natural Python API.

Everything else—scheduling, retries, supervision, observability, distributed execution, resource allocation—is handled by the ThreadWeave engine.


A familiar API

If you know FastAPI, this should feel natural.

from threadweave import ThreadWeave

app = ThreadWeave()


@app.task
async def hello(name: str) -> str:
    return f"Hello {name}"

No worker configuration.

No broker-specific code.

No infrastructure inside your business logic.


Native async

Async is not an afterthought.

It is a first-class citizen.

@app.task
async def generate_embeddings(document_id: UUID):

    document = await database.documents.get(document_id)

    embedding = await embedding_model.embed(document.text)

    await database.embeddings.save(
        document.id,
        embedding,
    )

No event-loop hacks.

No synchronous wrappers.

No hidden thread pools.


Resources instead of workers

Traditional task queues schedule work on workers.

ThreadWeave schedules work on resources.

@app.task(
    resources={
        "cpu": 4,
        "memory": "8Gi",
        "gpu": 1,
    }
)
async def train_model(dataset: Dataset):
    ...

The application describes what it needs.

The scheduler decides where it should run.


Retry policy

@app.task(
    retries=5,
    retry_delay="30s",
)
async def send_email(message):
    ...

Retry strategies belong to the infrastructure.

Not to your business logic.


Timeouts

@app.task(
    timeout="10m",
)
async def process_video(video):
    ...

Scheduling

@app.task(
    schedule="0 * * * *",
)
async def cleanup():
    ...

Or

await cleanup.delay()

Or

await cleanup.delay(
    countdown="5m",
)

Resource-aware AI workloads

AI jobs often require GPUs.

Instead of manually routing queues:

@app.task(
    resources={
        "gpu": 2,
        "gpu_memory": "40Gi",
    }
)
async def run_llm(prompt):
    ...

The scheduler automatically selects a compatible node.


Pipelines

Tasks compose naturally.

document = await download(url)

text = await extract_text(document)

summary = await summarize(text)

await publish(summary)

Distributed execution should not force a different programming model.


Events everywhere

Every execution emits structured events.

TaskSubmitted

↓

TaskScheduled

↓

ResourcesAllocated

↓

TaskStarted

↓

TaskCompleted

Observability is built into the architecture.

Not added later.


Compare with Celery

Celery

@app.task
def generate_pdf(document_id):

    document = load(document_id)

    pdf = build_pdf(document)

    save(pdf)

Scaling often requires thinking about:

  • queues
  • routing
  • workers
  • concurrency
  • pools
  • broker tuning

ThreadWeave

@app.task(
    resources={
        "cpu": 2,
        "memory": "4Gi",
    },
    timeout="5m",
)
async def generate_pdf(document_id):

    document = await repository.get(document_id)

    pdf = await renderer.render(document)

    await storage.save(pdf)

Business code stays focused on the business.

Infrastructure stays inside ThreadWeave.


Long-running tasks

Minutes.

Hours.

Days.

ThreadWeave is designed to supervise long-running executions, survive failures, and recover from crashes without sacrificing observability.


Built for extension

The Python runtime is only one runtime.

The ThreadWeave protocol allows additional runtimes to be implemented without changing the core engine.

Future runtimes may include:

  • JavaScript / TypeScript
  • Java
  • Go
  • .NET
  • WebAssembly

Philosophy

ThreadWeave is not a Python task queue.

It is a distributed execution platform whose first runtime happens to be Python.

Python developers get an idiomatic API.

The engine takes care of everything else.


Current status

  • RFCs in progress
  • Architecture under design
  • Rust core in development
  • Python runtime specification in progress

Contributions, discussions and feedback are welcome.

Metadata

Release files for threadweave-python 0.1.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for threadweave-python 0.1.5
File Size Uploaded Provenance Actions
threadweave_python-0.1.5.tar.gz 23.6 kB With provenance Details for threadweave_python-0.1.5.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for threadweave-python 0.1.5
File Interpreter ABI Platform Actions
threadweave_python-0.1.5-py3-none-any.whl Python 3 none any Details for threadweave_python-0.1.5-py3-none-any.whl

Total release size: 55.4 kB

Release files / threadweave_python-0.1.5.tar.gz

Download URL threadweave_python-0.1.5.tar.gz
Size 23.6 kB
Tags Source
SHA-256 checksum
How to use checksums
7f8e9fbe92fda595cc6e85bc678688fd864455e16e658c2a3f4e1f9287cca608
BLAKE2b-256 checksum
How to use checksums
8025fcc467f7b878186b39365855af9599772fe2b7251422b9fad6d655f8abf7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 14, 2026.

Transparency log

Release files / threadweave_python-0.1.5-py3-none-any.whl

Download URL threadweave_python-0.1.5-py3-none-any.whl
Size 31.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
04e9f6713e7091f31dc040dc09cd0acfd7a62324b0f128210a5500099d2a643b
BLAKE2b-256 checksum
How to use checksums
bf1cec708caec0db6fa4c5dfb1e5e166c022e90e78dfb73ba10f227b5acb99ca
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 14, 2026.

Transparency log

Security for threadweave-python 0.1.5

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for threadweave_python-0.1.5.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 14, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page