larztask

A durable background job/task queue in pure Python. Zero dependencies, no broker.

Register functions as tasks, enqueue jobs (now or scheduled), and run workers that execute them with retries, exponential backoff, priorities, and a dead-letter queue — without Redis, without RabbitMQ, without a single third-party package.

from larztask import TaskQueue

app = TaskQueue("jobs/")              # durable, multi-worker-safe

@app.task(max_retries=3)
def resize_image(path):
    ...                              # raising schedules a retry

resize_image.delay("/tmp/a.jpg")     # enqueue a job
app.work(burst=True)                 # run everything ready, then return

Why

• Zero dependencies, no broker. No Redis, no message queue, no server to run. A directory is your queue.
• Durable. Each job is a JSON file, written with fsync. Kill the process and restart — pending jobs are still there.
• Safely multi-worker. Claiming a job is a single atomic os.rename from queued/ to running/. If two workers race, exactly one wins — no lock files, no double-runs, no coordination service.
• Real retry semantics. Per-task max_retries with exponential backoff, then a dead-letter queue so failures are visible instead of lost.
• Scheduling & priorities. Delay a job, run it at a specific time, or bump its priority.
• Swappable store. MemoryStore for tests and ephemeral work, FileStore for durability — same API.

Install

pip install larztask

Usage

Define tasks

@app.task
def send_welcome(user_id):
    ...

@app.task(max_retries=5, backoff=2.0)     # 2s, 4s, 8s, 16s, 32s
def charge_card(order_id):
    ...

Enqueue

send_welcome.delay(42)                             # as soon as a worker is free
app.enqueue("send_welcome", args=(42,), delay=60)  # in 60 seconds
app.enqueue("charge_card", args=(7,), priority=10) # ahead of lower-priority jobs

Run workers

app.work(burst=True)     # drain all ready jobs, then return  (great for cron)
app.work()               # loop forever, polling; call app.stop() to exit

Run the same script in several processes pointed at the same directory and they share the queue safely. Watch what happens with an event hook:

app.work(on_event=lambda event, job: print(event, job["name"]))
# -> "done send_welcome", "retry charge_card", "dead charge_card", ...

Inspect

app.pending()      # jobs waiting to run
app.counts()       # {"queued": 3, "running": 1, "done": 40, "failed": 0, "dead": 2}
app.store.get(job_id)         # full job record incl. result / error / traceback
app.store.list("dead")        # everything in the dead-letter queue

How a job flows

enqueue ─▶ queued ─▶ (worker claims via atomic rename) ─▶ running
                                                            │
                       success ──────────────────────────▶ done
                       failure, attempts <= max_retries ─▶ queued (after backoff)
                       failure, retries exhausted ───────▶ dead

Scope

larztask is an embedded queue: workers are threads/processes you run, jobs run in-process, and the store is a local directory (or memory). That's the right tool for a huge range of apps — email sending, image processing, webhooks, scheduled cleanups, background computation — without operating a broker. It is not a distributed cross-machine queue; for that you'd point many machines at shared storage or reach for a networked broker.

Tests

python -m unittest discover -s tests -v   # 22 tests (both stores + atomic claim), zero deps

The Larz stack

Pure-Python, zero-dependency building blocks:

• larz — money-native web framework
• larzchain — from-scratch PoW blockchain
• larzmoney — exact, penny-perfect money
• larzcrypt — pure-Python cryptography toolkit
• larzdb — crash-safe embedded database
• larzagent — zero-dep AI agent framework
• larzchart — data to inline SVG charts
• larzmark — Markdown + SEO static sites
• larztask — this library

License

MIT © larz-scripter