Skip to main content

ThreadWeave Python

The official Python runtime for ThreadWeave.

Write idiomatic Python. Execute anywhere.


⚠️ Project status

ThreadWeave Python is currently in active design and early development.

The API shown below represents the intended developer experience and may evolve before the first public release.


Why?

Most distributed task queues were designed long before modern Python.

Today we write applications with:

  • FastAPI
  • async/await
  • asyncio
  • SQLAlchemy Async
  • httpx
  • AI workloads
  • GPUs

Yet background execution often forces developers back into synchronous workers, complex concurrency models, or infrastructure-specific code.

ThreadWeave takes a different approach.

The execution engine is written in Rust.

The Python runtime focuses exclusively on providing a natural Python API.

Everything else—scheduling, retries, supervision, observability, distributed execution, resource allocation—is handled by the ThreadWeave engine.


A familiar API

If you know FastAPI, this should feel natural.

from threadweave import ThreadWeave

app = ThreadWeave()


@app.task
async def hello(name: str) -> str:
    return f"Hello {name}"

No worker configuration.

No broker-specific code.

No infrastructure inside your business logic.


Native async

Async is not an afterthought.

It is a first-class citizen.

@app.task
async def generate_embeddings(document_id: UUID):

    document = await database.documents.get(document_id)

    embedding = await embedding_model.embed(document.text)

    await database.embeddings.save(
        document.id,
        embedding,
    )

No event-loop hacks.

No synchronous wrappers.

No hidden thread pools.


Resources instead of workers

Traditional task queues schedule work on workers.

ThreadWeave schedules work on resources.

@app.task(
    resources={
        "cpu": 4,
        "memory": "8Gi",
        "gpu": 1,
    }
)
async def train_model(dataset: Dataset):
    ...

The application describes what it needs.

The scheduler decides where it should run.


Retry policy

@app.task(
    retries=5,
    retry_delay="30s",
)
async def send_email(message):
    ...

Retry strategies belong to the infrastructure.

Not to your business logic.


Timeouts

@app.task(
    timeout="10m",
)
async def process_video(video):
    ...

Scheduling

@app.task(
    schedule="0 * * * *",
)
async def cleanup():
    ...

Or

await cleanup.delay()

Or

await cleanup.delay(
    countdown="5m",
)

Resource-aware AI workloads

AI jobs often require GPUs.

Instead of manually routing queues:

@app.task(
    resources={
        "gpu": 2,
        "gpu_memory": "40Gi",
    }
)
async def run_llm(prompt):
    ...

The scheduler automatically selects a compatible node.


Pipelines

Tasks compose naturally.

document = await download(url)

text = await extract_text(document)

summary = await summarize(text)

await publish(summary)

Distributed execution should not force a different programming model.


Events everywhere

Every execution emits structured events.

TaskSubmitted

↓

TaskScheduled

↓

ResourcesAllocated

↓

TaskStarted

↓

TaskCompleted

Observability is built into the architecture.

Not added later.


Compare with Celery

Celery

@app.task
def generate_pdf(document_id):

    document = load(document_id)

    pdf = build_pdf(document)

    save(pdf)

Scaling often requires thinking about:

  • queues
  • routing
  • workers
  • concurrency
  • pools
  • broker tuning

ThreadWeave

@app.task(
    resources={
        "cpu": 2,
        "memory": "4Gi",
    },
    timeout="5m",
)
async def generate_pdf(document_id):

    document = await repository.get(document_id)

    pdf = await renderer.render(document)

    await storage.save(pdf)

Business code stays focused on the business.

Infrastructure stays inside ThreadWeave.


Long-running tasks

Minutes.

Hours.

Days.

ThreadWeave is designed to supervise long-running executions, survive failures, and recover from crashes without sacrificing observability.


Built for extension

The Python runtime is only one runtime.

The ThreadWeave protocol allows additional runtimes to be implemented without changing the core engine.

Future runtimes may include:

  • JavaScript / TypeScript
  • Java
  • Go
  • .NET
  • WebAssembly

Philosophy

ThreadWeave is not a Python task queue.

It is a distributed execution platform whose first runtime happens to be Python.

Python developers get an idiomatic API.

The engine takes care of everything else.


Current status

  • RFCs in progress
  • Architecture under design
  • Rust core in development
  • Python runtime specification in progress

Contributions, discussions and feedback are welcome.

Metadata

Release files for threadweave-python 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for threadweave-python 0.1.1
File Size Uploaded Provenance Actions
threadweave_python-0.1.1.tar.gz 21.6 kB With provenance Details for threadweave_python-0.1.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for threadweave-python 0.1.1
File Interpreter ABI Platform Actions
threadweave_python-0.1.1-py3-none-any.whl Python 3 none any Details for threadweave_python-0.1.1-py3-none-any.whl

Total release size: 50.3 kB

Release files / threadweave_python-0.1.1.tar.gz

Download URL threadweave_python-0.1.1.tar.gz
Size 21.6 kB
Tags Source
SHA-256 checksum
How to use checksums
68023eac2c78ca95cb5a1ac5ec5a5c9a2a65d435c6828ea7814a20ea78fcb90b
BLAKE2b-256 checksum
How to use checksums
09179c4228544cd188782b72b0da9563df7b60763e8610a561b1ff3262ffcdc3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 13, 2026.

Transparency log

Release files / threadweave_python-0.1.1-py3-none-any.whl

Download URL threadweave_python-0.1.1-py3-none-any.whl
Size 28.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
cb284ae1db7b9d3d09cf454c01a7a3ef19428f7d3bf2f27f164ec91b3a9f16be
BLAKE2b-256 checksum
How to use checksums
62f392aa7b1cc65a3f4b71c0152acb3ca64b60ffd84485bebca8bc1f728717c4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 13, 2026.

Transparency log

Security for threadweave-python 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for threadweave_python-0.1.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Aug 13, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page