offwork 0.2.0

Distributed Python task execution via automatic function serialization

offwork

Run any Python function on a remote worker with just two lines of code.

Put .connect() somewhere at the start of your script, add @offwork.task to your function, that's it. You can now run it remotely — no shared codebase, no deployment pipeline.

offwork captures its entire dependency graph (helpers, imports, closures, constants) and ships it to the worker as a self-contained payload. The worker doesn't need to have any prior knowledge of your code.

Quick start

pip install offwork
offwork worker --backend local://localhost:9748 --tmp  # start a worker in a temp venv

import asyncio, math, offwork

offwork.connect("local://localhost:9748")

def add(a: float, b: float) -> float:
    return a + b

@offwork.task  # only the entry point needs this - add() is captured automatically
def hypotenuse(a: float, b: float) -> float:
    return math.sqrt(add(a**2, b**2))

async def main():
    print(await hypotenuse.run(3.0, 4.0))  # 5.0

asyncio.run(main())

.run() serializes the function graph, submits it to the worker, and returns the result. The worker reconstructs source, installs any missing packages, and executes.

Multi-machine

Swap local:// for Redis or RabbitMQ to run on a remote worker:

pip install offwork[redis]
offwork worker --backend redis://other-machine:6379

Features

Async-native	.run(), .submit(), .map(), asyncio.gather — all coroutines
Scheduling	submit(run_in=delay), submit(run_at=dt), submit(run_every=interval) with cancellation
Retry & timeout	@offwork.task(timeout=30, retries=3) with exponential backoff
Throttling	@offwork.task(throttle=timedelta(hours=24)/50) — rate-limit executions
Progress & cancellation	offwork.progress(3, 10) inside tasks; fut.cancel() / await fut.cancel() on client
Heartbeat & stall detection	Workers heartbeat every second; clients raise TaskStalled on silence
Package auto-install	Workers pip install missing packages before execution
Docker sandbox	Optional container isolation, fully transparent to clients
Signed execution	Pre-shared token or PIN pairing + HMAC-SHA256 task authentication
...	See Features for more information.

Security

Signing

To make sure your worker only executes trusted code, use --require-signing.
Configure a shared token or pair interactively with a PIN. See Signing & Pairing for details.

# Token (CI/CD)
offwork token generate
export OFFWORK_SIGNING_TOKEN=<token>                         # client & worker
offwork worker --backend redis://localhost:6379 --require-signing

# PIN pairing (interactive)
offwork worker --backend redis://localhost:6379 --pair       # shows 6-digit PIN
offwork pair --backend redis://localhost:6379                # client: enter PIN

After pairing, tasks are signed automatically with no client code changes. See Signing & Pairing.

Sandbox

To avoid side-effects on the worker machine, run tasks inside Docker containers:

offwork sandbox setup                                      # build image (once)
offwork worker --backend redis://localhost:6379 --sandbox  # run with isolation

See Sandbox for configuration.

Documentation

Features	Full feature guide and API walkthrough
Technical Overview	Architecture, serialization format, internals
Signing & Pairing	Cryptographic task signing protocol
Sandbox	Docker container isolation

Examples

offwork worker --backend local://localhost:9748 --tmp  # start worker
offwork run examples/remote_execution.py              # run any example

See examples/README.md for a guide to all examples.

License

AGPL-3.0