offwork 0.1.2

Distributed Python task execution via automatic function serialization

offwork

Run any Python function on a remote worker — zero setup, zero deployment.

Add @offwork.task to a function. offwork captures its source, all dependencies, and all imports automatically. Workers reconstruct and execute everything from scratch — no shared filesystem, no deployment pipeline. Missing packages are installed on the fly.

Quick start

pip install offwork

import asyncio, math, offwork
import offwork

offwork.connect("local://localhost:9748")

def add(a, b):
    return a + b

@offwork.task
def hypotenuse(a: float, b: float) -> float:
    return math.sqrt(add(a**2, b**2))

async def main():
    print(await hypotenuse.run(3.0, 4.0))  # 5.0

asyncio.run(main())

Only the entry point needs @offwork.task — everything it calls is captured automatically.

offwork worker --backend local://localhost:9748 --tmp   # start a worker
python my_script.py                                    # → 5.0

For multi-machine, swap local:// for redis:// or an https:// managed broker URL.

Features

Auto dependency capture	Functions, classes, constants, closures — recursive AST analysis
Package auto-install	Workers pip install missing packages before execution
Async-native	.run(), .start(), .map(), asyncio.gather
Retry & timeout	@offwork.task(timeout=30, retries=3) with exponential backoff
Scheduling	.run_in(delay), .run_at(datetime), .run_every(freq) with cancellation
Throttling	@offwork.task(throttle=timedelta(hours=24)/50) — rate-limit executions
Progress & cancellation	offwork.progress(3, 10) inside tasks; await future.cancel() on client
Heartbeat & stall detection	Workers heartbeat; clients raise TaskStalled on silence
Content-hash caching	Same code = cache hit, regardless of client
Pluggable backends	local:// (TCP), redis://, amqp:// (RabbitMQ), https:// (hosted)
Docker sandbox	Container isolation, transparent to clients
Signed execution	Pre-shared token or PIN pairing + HMAC-SHA256 task authentication
Graceful shutdown	Ctrl+C drains in-flight tasks; second Ctrl+C force-quits

Sandbox

Run tasks inside Docker containers for isolation — transparent to clients:

offwork sandbox setup                                      # build image (once)
offwork worker --backend redis://localhost:6379 --sandbox  # run with isolation

See Sandbox for configuration.

Signing

Pre-shared token or PIN-based pairing + HMAC-SHA256 — workers reject untrusted or tampered tasks:

# Token-based (recommended for CI/CD)
offwork token generate
export OFFWORK_SIGNING_TOKEN=<token>                         # set on client & worker
offwork worker --backend redis://localhost:6379 --require-signing

# PIN-based pairing (interactive)
offwork worker --backend redis://localhost:6379 --pair       # displays a 6-digit PIN
offwork pair --backend redis://localhost:6379                # on client: enter the PIN

After setup, tasks are signed automatically. No client-side code changes. See Signing & Pairing.

Documentation

Quick Start	Tutorial and API walkthrough
Technical Overview	Architecture, serialization format, internals
Signing & Pairing	Cryptographic task signing protocol
Sandbox	Docker container isolation

Examples

offwork worker --backend local://localhost:9748 --tmp
offwork run examples/remote_execution.py

remote_execution.py · async_execution.py · package_installation.py · progress_reporting.py · cancellation.py · scheduling.py · throttling_and_retry.py · large_module.py

License

AGPL-3.0