mayu
mayu is a fast, Redis-backed Python task queue with a Rust core, built for free-threaded (no-GIL) Python. Its workers run tasks on real parallel OS threads under CPython 3.14t, so a few processes can do the work that Celery needs dozens of forked processes for.
Status: alpha. Delivery is at-least-once: tasks must be idempotent.
- Fast: about 31× Celery's no-op throughput and 5× on a realistic task, with an eighth of the memory (benchmarks).
- Free-threaded first: thread mode on Python 3.14t, with automatic fallback to process mode on GIL builds (3.13, 3.14).
- Reliable: acks after the task finishes, crash recovery, poison-task detection, dead letters and replay, and it survives Redis restarts.
- Familiar API:
@app.task,.delay(),.send(), retries, countdown/ETA, rate limits, time limits, queue priority,link=callbacks, sync andasync deftasks. - Operable: Prometheus metrics, JSON logs, OpenTelemetry tracing, and an
inspectCLI.
Install
pip install mayu # or: uv add mayu
pip install "mayu[otel]" # with OpenTelemetry tracing
pip install "mayu[gpu]" # with NVIDIA GPU monitoring (NVML)
Requires Python 3.13+ and Redis. Use a free-threaded build (python3.14t, e.g.
uv venv --python 3.14t) to get thread mode.
Quickstart
# tasks.py
from mayu import App, Retry
app = App(broker="redis://localhost:6379/0", result_ttl=3600)
@app.task(max_retries=5, retry_on=(TimeoutError,))
def add(a: int, b: int) -> int:
return a + b
@app.task
async def fetch(url: str) -> str:
...
mayu worker tasks:app --threads 32
from tasks import add
add.delay(2, 3)
h = add.send(2, 3, countdown=10)
h.result(timeout=30) # 5
Configuration comes from constructor arguments, overridden by MAYU_BROKER,
MAYU_RESULT_TTL (0 disables results), MAYU_DEFAULT_QUEUE, MAYU_SERIALIZER.
countdown, eta, queue and link are reserved option names and cannot be task parameters.
Reliability
mayu delivers at least once: a task is acknowledged only after it finishes, so a crash means the task runs again, never that it is lost. Write idempotent tasks.
- Each worker heartbeats to
mayu:workers:{worker_id}(TTL = 3 ×--heartbeat-interval). - A worker keeps ownership of tasks it is running by re-claiming them every half
--visibility-timeout(default 30 s). - When a worker dies (
kill -9, OOM, segfault), the other workers reclaim its tasks once they have been silent for--visibility-timeout. - A task whose worker dies
--max-deliveriestimes in a row (default 5) is treated as poison and dead-lettered instead of being retried forever. - Redis restarts are survived: connections reconnect, and outcomes produced while Redis is down are recorded when it returns. Use Redis persistence (AOF) if queued tasks must survive a Redis restart.
- Python threads cannot be killed: set timeouts on network clients inside tasks.
Chaos test (needs Docker): .venv/bin/pytest tests/chaos -v
(MAYU_CHAOS_N / MAYU_CHAOS_SECONDS scale it).
Time limits
@app.task(soft_time_limit=30, hard_time_limit=60)
def crunch(): ...
- Soft limit:
mayu.SoftTimeLimitExceededis raised inside the task at its next Python bytecode, so it can be caught to clean up. It cannot interrupt blocking C calls or socket reads: set timeouts on network clients. - On Python 3.13 a soft limit that lands in a
whileloop with an empty body can escape the task'stry/except(a CPython 3.13 bug, fixed in 3.14). The task then fails withSoftTimeLimitExceededwithout running its cleanup. - Hard limit: Python threads cannot be killed. The task is recorded as failed with
mayu.TimeLimitExceeded(retried if it is inretry_on), its thread is abandoned as a zombie, and a replacement thread takes over. Whatever the zombie produces later is discarded. - Async tasks are cancelled at their limit instead.
@app.task(threadsafe=False)runs a task on one dedicated thread, one call at a time, for legacy code that is not thread-safe.
The worker recycles itself (graceful stop, then a fresh interpreter in the same process) when
zombies exceed --max-zombie-threads (default 4), after --max-tasks-per-process tasks, or
above --max-memory-per-process MB of RSS.
Rate limits
@app.task(rate_limit="100/s") # also "10/m", "5/h"; a bare number means per second
def call_api(): ...
Starts are spaced evenly (no bursts). The limit is per worker process: with
--processes 4, a 10/s task can start 40 times a second across them. Tasks that would wait
more than a second for their turn go back to Redis until then, so a rate-limited backlog never
fills a worker's prefetch buffer or delays other tasks. A deferred task returns to Redis as a
fresh entry, so its crash-redelivery count (--max-deliveries) starts over. Deferred tasks are
spaced for the worker that deferred them, so adding workers doesn't speed up a backlog that is
already deferred.
Resources
@app.task(resources={"gpu": 0.3, "vram_gb": 1.2})
def embed(texts): ...
mayu worker tasks:app --resources gpu=1,vram_gb=7.5 # or MAYU_RESOURCES
mayu worker tasks:app --gpus 0,1 --resources gpu=1 # one worker process per GPU
A worker starts a task only when everything it requests fits in what is left of what the worker
declared, so three of the tasks above run side by side on one GPU and a fourth waits for a
finisher. Amounts are in whatever unit you pick (GPUs, GB, cores), down to 0.001. A worker
refuses to start if a task on a queue it serves requests a resource it didn't declare, or more
than it has; tasks on other queues are ignored. Tasks without resources= are never held back.
- No starvation: if the oldest, highest-priority waiting task has been due for 5 seconds, later tasks may start only if they request none of the resources it is waiting for.
- Per process: each worker process counts only its own tasks. Several processes sharing a
GPU each think they own it, so run one process per GPU and size
--resourcesfor that process.--gpus 0,1does this: it starts one process per id withCUDA_VISIBLE_DEVICESset to it, before your app is imported (ids are opaque, soGPU-abc...UUIDs work).--processes, if given, must equal the number of ids. - Zombies: resources are released when a task's outcome is reported. A task past its hard time limit is reported as failed while its thread keeps running, so until the worker recycles, a zombie can use a GPU that a new task has already been given.
Model pools
@app.pool("gemma", vram_gb=5.8) # resources are per instance
def gemma(ctx): # ctx.name, ctx.index in range(ctx.size), ctx.size
return load_gemma(device="cuda")
@gemma.close
def _(model):
model.unload()
@app.task(queue="gpu")
def encode(text, model=app.use("gemma")): ... # model is one instance, yours alone until the task ends
mayu worker tasks:app --queues gpu --pools gemma=2,tts=3 --resources gpu=1,vram_gb=40 # or MAYU_POOLS
Code says how to build an instance; the operator says how many. A worker builds every instance
before it fetches anything, one at a time and logging each load time, and keeps them for its life.
At most N tasks that use a pool of N run at once, so a task always finds a free instance and never
waits inside the task body: it waits in the worker's queue, like any task held back by
resources=. A task gets its instances before its time limits start and gives them back before
its outcome is reported.
- Reservations: a pool's resources are reserved for the worker's life, N × the amount, and
come off what
--resourcesleaves for tasks.gemma=2above holds 11.6 GB ofvram_gb; a task asking for more than the remaining 28.4 makes the worker refuse to start. Resource names starting withpool:are reserved. - Startup checks: a worker refuses to start if a task it serves uses a pool that the process
doesn't register with
@app.poolor that--poolsdoesn't size, if--poolsnames an unregistered pool, or if the reservations exceed what it declares. A pool sized but unused by the queues it serves is not loaded (a warning is logged). If a factory raises while loading, the instances built so far are closed and the worker exits. - Broken instances: raise
DiscardInstance(DiscardInstance(pool="gemma")to discard only that pool's instance, otherwise all the task holds), or list exception classes in@app.pool(..., discard_on=CudaError). The task still fails or retries by its usual rules (retry_on=(DiscardInstance,)retries it on another instance). After the outcome is reported, the instance is closed and a replacement built with the samectx.index; the pool runs one instance short meanwhile. If the rebuild fails, the pool stays smaller andmayu_pool_sizeshows it. A pool with no instances left makes the worker recycle. - Close:
@pool.closeruns on discarded instances and, at shutdown, on every instance that is back in the pool. Instances still held by threads that outlive the shutdown grace period are not closed. Errors incloseare logged and ignored. - Zombies: a task past its hard time limit is reported as failed while its thread keeps its instance, so the pool shrinks by one for good: the instance is never reused or closed, because the thread may still be using it. Recycling brings the pool back to full size.
- Per process: each worker process loads its own instances, in process mode and with
--gpustoo, so size pools for one process or one GPU. A recycle restarts the process and reloads every instance, so recycling a worker with big models costs their load time. - Sync tasks only: an
async deftask can't use a pool (TypeErrorat registration). A task can use each pool in only one parameter, and the parameter must be usable by keyword. - Producers need no pool code:
app.use("gemma")is only a name, resolved when a worker starts, so a producer can declare the task (see below) without torch or@app.pool. Passing the pool's parameter tosend(),wf.add()or alink=raisesTypeError. A payload that carries it anyway, only possible withapp.sendby name, has its keyword value overwritten by the instance, and a positional one fails the task with a duplicate-argumentTypeError.
Lazy pools (models that don't fit together)
Eager pools (the default, described above) load everything at startup and keep it. If the models don't fit in memory together, run the worker in lazy mode: it loads a model when a task needs it and closes idle ones to make room.
mayu worker tasks:app --queues gpu --pools gemma=1,tts=1 --pool-mode lazy --schedule throughput \
--preload gemma --resources gpu=1,vram_gb=7.5 --prefetch-multiplier 1
# or MAYU_POOL_MODE=lazy, MAYU_SCHEDULE, MAYU_PRELOAD
Here gemma and tts don't fit in 7.5 GB together, so only one is in memory at a time. The pool
code and the tasks are the same as for eager pools; --pools gemma=1 is now the most instances
loaded at once, and nothing is reserved for it at startup. --pool-mode applies to every pool of
the worker.
- Loading and eviction: a pool's resources are held only while an instance is loaded. A task that finds no idle instance of its pool has the worker build one, after closing idle instances (least recently used first) until it fits. Only idle instances are closed, never one a task is using. A task without a pool may close an idle model too, if it has no room beside it. Loading and closing happen before the task's time limits start.
--preload gemma,tts: load one instance of each at startup, in that order. They must fit together, or the worker refuses to start. Without it the first task for a model pays the load.--schedule: a worker holds back tasks that don't fit, and lazy mode lets it prefer ones whose model is already loaded. Queue priority still comes first; this only orders tasks within a priority.throughput(fewest swaps) lets a task with a loaded model run ahead of other work that arrived up to 60 seconds before it.oldest(the default in lazy mode) orders by the start of the workflow, and by arrival for tasks outside one, so a workflow already under way finishes first; a loaded model is worth 10 seconds, about one model load. A task that needs a swap is passed only by loaded-model tasks that arrived (or started) less than that bound after it, so none waits forever. The bounds are not configurable.- Only prefetched tasks are reordered: the worker chooses among tasks it has already claimed
from Redis. Run GPU queues with a small
--prefetch-multiplier(the default is 2): a larger buffer gives it more to choose from but claims more tasks before they can run. - Startup checks:
--scheduleand--preloadneed--pool-mode lazy. A worker refuses to start if a task's own resources plus one instance of each pool it uses exceed what--resourcesdeclares, or if the preloaded pools don't fit together. - Broken instances:
DiscardInstanceanddiscard_onclose the instance after the outcome is reported, as with eager pools, but nothing is rebuilt: the next task that needs the model loads it. - Failed loads: if a factory raises while loading for a task, that task fails with
RuntimeError("pool 'gemma' failed to load: ..."), soretry_onapplies, and the worker keeps running. Instances closed to make room for it stay closed. A failing--preloadstops the worker at startup, as in eager mode. - Zombies: as with eager pools, a task past its hard time limit keeps its instance, so the pool shrinks by one for good. Its memory also stays counted as in use until the worker recycles.
- Close:
@pool.closeruns on evicted and discarded instances and at shutdown, as above. linger=:@app.pool("gemma", vram_gb=5.8, linger=30)keeps an idle instance from being closed for room for up to 30 seconds while a workflow still has a task to come that uses the pool, so a workflow that returns to a model after a step on another doesn't pay its load twice. The time counts from when the instance went idle, not from its last use. Only workflows this worker has fetched a task of count, and once the time is up, or no such task is left, the instance is closed like any idle one. Tasks that need the room wait meanwhile.mayu_pool_demandshows the count.lingermust be above 0 and is ignored in eager mode.--lookahead: while a workflow's current task runs, load the model its next task needs (mayu worker ... --pool-mode lazy --lookahead, orMAYU_LOOKAHEAD=1), so the next task finds it loaded. It is a low-priority guess: it loads only when the model fits without closing anything, the pool has no idle instance and is below its--poolssize, and the worker has nothing due to run. Likelinger, it knows only the workflows this worker has fetched a task of. A load that fails is logged and the task loads the model itself later; that pool isn't tried ahead again for 60 seconds.mayu_pool_lookahead_loads_totalcounts the loads.--lookaheadneeds--pool-mode lazy.
GPU monitoring and self-tuning
pip install "mayu[gpu]"
mayu worker tasks:app --resources gpu=auto,vram_gb=auto --autotune gpu
With mayu[gpu] installed, the worker reads the GPUs it can see through NVIDIA's NVML every 2
seconds. That feeds the mayu_gpu_* metrics, a guard that keeps vram_gb honest when other programs
use the card, and optionally the tuner below. Without the extra, or on a machine without a GPU,
nothing changes.
--resources gpu=auto,vram_gb=auto: read the amounts from the visible GPUs instead of typing them:gpuis their number andvram_gbtheir memory in GiB minus 0.5 GiB per GPU, for the CUDA context and the desktop (an 8 GiB card gives 7.5). Either can be mixed with plain numbers, likegpu=1,vram_gb=auto. The worker refuses to start if no GPU is visible.--gpu-monitor auto|on|off(MAYU_GPU_MONITOR):auto(the default) monitors when NVML works and a GPU is visible,onrefuses to start if it can't,offnever loads NVML.offcan't be combined withautoresources or--autotune, which need the readings.--gpu-interval SECONDS(MAYU_GPU_INTERVAL, default 2) sets how often the GPUs are read.- Which GPUs: only the ones
CUDA_VISIBLE_DEVICESshows, so--gpus 0,1gives each process its own GPU to monitor. It takes indexes andGPU-...UUIDs. NVML numbers GPUs asnvidia-smidoes, which is CUDA's order only withCUDA_DEVICE_ORDER=PCI_BUS_ID: on a machine with several different GPUs, set it or use UUIDs, or an index can name a different card in CUDA than in the monitor. Other forms (MIG-...) are an error unless monitoring is off. - VRAM guard: with
vram_gbdeclared, the worker compares what the GPUs have in use with what its own processes (and their children) use, and treats the difference as other programs' memory. The GPUs' memory beyondvram_gbis slack that absorbs a small foreign user at no cost; past that, the guard takes the excess out ofvram_gb, so tasks that no longer fit wait instead of running out of memory. It shrinks at once and gives memory back only after 3 samples in a row ask for less, so a desktop app that comes and goes can't make admission flap. Running tasks and loaded pools are never evicted for it: it only holds back new ones. Each change is logged. If NVML can't say which memory is the worker's, the guard turns itself off with a warning. - Under-declared warning: if the worker's processes use more VRAM than its running tasks and loaded
pools declare (by more than 0.5 GiB or 10%) for 3 samples in a row, a warning says to raise
vram_gb=on them, at most once every 5 minutes. mayu can't stop a task from using more than it declared. --autotune gpu(MAYU_AUTOTUNE=gpu): run only as many GPU tasks as help. The configuredgpuis the ceiling, the smallestgpurequest among the tasks the worker serves is the step, and the largest is the floor, so the biggest task always fits. Every 5 samples (10 seconds by default) the tuner takes the mean GPU utilization. At 90% or more it removes a step ofgpu, since more concurrency on a saturated GPU only adds latency, unless it saw the GPU below 90% one step lower (with tasks waiting, in the last 5 minutes): then this level gets more done, and it stays. Below 70%, with tasks waiting forgpuand neither the guard nor the cards' memory (95% in use) under pressure, it adds a step back. After a shrink it waits 6 windows before it grows again. Each change is logged. So it settles on the fewest GPU tasks that keep the GPU busy. It needs agpucapacity, a served task that requestsgpu, and monitoring.
The GPU metrics are labelled gpu (NVML index) and uuid. With several GPUs they are per GPU, and the
tuner and the guard use their mean and sum.
Queue priority
mayu worker tasks:app --queues high,default,low
The order of --queues is a strict priority: whenever a worker picks the next task to start,
it prefers high tasks over default ones. Tasks already claimed into its prefetch buffer, or
already running, are not preempted, and a constantly busy high queue still starves the ones
after it. Without --queues, a worker consumes every queue your app uses, at equal priority.
Callbacks
add.send(2, 3, link=notify.s("sum")) # after add succeeds: notify(5, "sum")
add.send(2, 3, link=[audit, notify.s("sum")])
A link= callback runs only after its parent succeeds, with the parent's result as its first
argument (even when results are disabled). The callback is enqueued in the same atomic step
that acks the parent, so it is enqueued exactly once per successful parent. Like every task, it
can still run more than once if a worker crashes. Callbacks run on the callback task's queue
and do not chain further; for a DAG of tasks see Workflows.
Workflows
A DAG of tasks, submitted as one unit. A node runs once all its parents have succeeded.
wf = app.workflow(name="video:42")
plan = wf.add(plan_run.s(prompt, run))
voice = wf.add(voice_lines.s(run), after=[plan])
scenes = [wf.add(render_scene.s(run, s), after=[voice]) for s in ("S1", "S2", "S3")]
wf.add(finalize.s(run), after=scenes)
handle = wf.send() # WorkflowHandle
handle.status() # {"state": "running", "waiting": 4, "queued": 1, "done": 1, "dead": 0, ...}
handle.nodes[-1].result(60) # a TaskHandle per node, in add order
handle.wait(timeout=600) # "done", "failed", "cancelled" or "expired"
handle.cancel()
wf.add takes a task or task.s(...) with exactly the arguments the task needs; after= takes a node
or a list of nodes. send() writes the whole DAG and enqueues the nodes with no parents in one atomic
step. When a node succeeds, the step that acks it also enqueues each child whose parents are now all
done, so each child is released exactly once. Nodes still run at least once, like any task.
- Nodes do not receive their parents' results; pass data through your own storage. Nodes can't use
link=,countdownoreta. - A node that fails for good (dead-lettered) blocks its descendants and the workflow reports
failed.mayu dead replaythe node and, once it succeeds, the rest of the DAG continues. handle.cancel()stops waiting nodes from being released and skips nodes that are queued but not yet started (they are acked with a "workflow cancelled" failure). A node already running finishes.app.workflow(name, ttl=7 * 86400)sets how long Redis keeps the workflow after its last progress. If it expires, running nodes still finish but release nothing.- Release computes stream keys at run time, so workflows, like acks with results, are not Redis-Cluster-safe.
- Upgrade workers before submitting workflows: an older worker acks a node but never releases its children.
Declaring tasks without their code
A producer often can't import a task's dependencies (a GPU model, a heavy library). Declare the task there with only its name, options and signature, and attach the code in the worker:
# shared module, imported by producers and workers
@app.declare(name="ml.embed", queue="gpu")
def embed(text: str, *, normalize: bool = True): ...
# worker module, imported only by workers that serve the "gpu" queue
@embed.implement
def _(text: str, *, normalize: bool = True):
return model.encode(text, normalize=normalize)
embed.send("hello") # producer: arguments are checked against the declaration
app.send("ml.embed", "hello") # by name: checked if registered here, sent as-is if not
implement must match the declaration's parameter names, kinds and defaults (annotations are ignored)
and returns the task. Calling a declared task that isn't implemented raises TypeError.
A worker without --queues serves only queues where it implements every task (the default queue
included), so it never takes work meant for another worker. With --queues, naming a queue that has
an unimplemented task is an error (mayu: queue 'gpu' has tasks declared but not implemented in this process: ..., exit 2). If a delivery for an unimplemented task still arrives, it
fails with "not implemented" instead of running the stub.
Modes
mayu worker tasks:app --threads 32 --processes 2 # thread mode: 2 processes × 32 threads
mayu worker tasks:app --mode process --processes 8 # process mode: 8 processes × 1 task each
--mode auto (default) uses thread mode when the GIL is off after importing your app, and
process mode otherwise, naming the module that re-enabled the GIL. With more than one process,
a supervising parent keeps the workers running, restarts any that crash (with backoff), and
shuts them all down gracefully on SIGTERM. In process mode, a task that exceeds its hard time
limit takes its process down with it, and the process is replaced.
Prefer a few processes × many threads: a segfault in a C extension takes down every task in its process.
Memory and huge pages
On Linux with transparent huge pages set to always (the default on Arch, Fedora and RHEL), a
free-threaded worker can use about 4x the memory it needs: every thread's slice of CPython's
object heap is backed by a 2 MB huge page. Measured with 2 processes × 32 threads, that was
525 MB PSS against 135 MB with huge pages off, at the same throughput. Process mode on a GIL
build is not affected.
mayu worker tasks:app --thp python # default; or MAYU_THP=python
python(default): on a free-threaded build, opt only Python's own object heap out of huge pages, before your app is imported. Memory your tasks allocate themselves (numpy, torch and othermalloc/mmapbuffers) keeps the system setting.keep: change nothing.off: disable huge pages for the whole process, which child processes inherit, including tasks that start subprocesses and, under the supervisor, the worker processes.
WorkerRunner(..., thp="keep") takes the same modes. A failed system call is logged as a
warning and the worker starts anyway. Linux only; elsewhere every mode does nothing. Limitation:
CPython reserves further heap arenas only after about 1 GB of live Python objects, and those
keep the system setting.
Metrics
mayu worker tasks:app --metrics-port 9100 # GET http://host:9100/metrics (Prometheus)
Each worker process serves its own metrics: with --processes 4 --metrics-port 9100, the
processes listen on 9100–9103, so scrape all four. MAYU_METRICS_PORT sets the default;
--metrics-host picks the interface (default 0.0.0.0).
| Metric | Type | Labels |
|---|---|---|
mayu_tasks_received_total |
counter | queue, task |
mayu_tasks_settled_total |
counter | queue, task, outcome (success, retry, dead, cancelled) |
mayu_task_duration_seconds |
histogram | queue, task |
mayu_tasks_deferred_total |
counter | queue, task (rate-limited tasks sent back to Redis) |
mayu_tasks_reclaimed_total |
counter | queue (taken over from a dead worker) |
mayu_broker_errors_total |
counter | op |
mayu_worker_concurrency, mayu_tasks_inflight, mayu_tasks_running |
gauge | |
mayu_queue_ready, mayu_queue_pending, mayu_queue_delayed, mayu_queue_dead |
gauge | queue |
mayu_resource_capacity, mayu_resource_in_use |
gauge | resource (only with --resources) |
mayu_pool_size, mayu_pool_busy |
gauge | pool (only with --pools; instances in the pool, and held by running tasks) |
mayu_pool_rebuilds_total |
counter | pool (instances replaced after DiscardInstance or discard_on) |
mayu_pool_loaded |
gauge | pool (only with --pool-mode lazy; instances in memory, idle or busy) |
mayu_pool_loads_total, mayu_pool_evictions_total |
counter | pool (lazy instances built, preloads included, and closed to make room) |
mayu_pool_demand |
gauge | pool (only in lazy mode with --lookahead or linger=; tasks still to come in workflows this worker has seen) |
mayu_pool_lookahead_loads_total |
counter | pool (lazy instances built by --lookahead before their task) |
mayu_gpu_utilization_ratio |
gauge | gpu, uuid (0 to 1) |
mayu_gpu_memory_used_bytes, mayu_gpu_memory_total_bytes |
gauge | gpu, uuid (used by all processes, and installed) |
mayu_gpu_process_memory_bytes |
gauge | gpu, uuid (used by this worker and its children; omitted if NVML can't say) |
mayu_gpu_power_watts, mayu_gpu_temperature_celsius |
gauge | gpu, uuid (omitted if no GPU reports them) |
Pools are counted in the resource ledger as pool:<name>, but they appear only as mayu_pool_*, never under
mayu_resource_*. The mayu_gpu_* metrics appear only while GPU monitoring runs (see above).
Queue gauges are read from Redis on each scrape, so every worker reports the same values: use
max by (queue) (mayu_queue_ready). If Redis cannot be read, the scrape still answers without them.
Logging
mayu worker tasks:app --log-format json --log-level INFO
text (default) writes readable lines; json writes one object per line on stderr, from both the
Rust core and Python, with timestamp, level, message, target and event fields such as
task_id, task, queue and error. MAYU_LOG_FORMAT and MAYU_LOG_LEVEL set the defaults. Failed
tasks are logged at WARN, retries at INFO, successes at DEBUG. When you run WorkerRunner
from your own code, call mayu.configure_logging("json") first.
Inspecting and dead letters
mayu inspect tasks:app # queue depths (ready, claimed, delayed, dead), live workers, unfinished workflows
mayu inspect tasks:app --json
mayu dead list tasks:app --queue emails # oldest first: when, task id, task, attempt, error
mayu dead replay tasks:app --queue emails # run them again
mayu dead replay tasks:app --queue emails --id 01J... --limit 10
The workflows table lists those not yet done (id, name, state, node counts, age); finished ones are left
out and expire on their ttl.
A replayed task keeps its id, starts again at attempt 0, and its stored failure is cleared, so
handle.result() waits for the new outcome. Each dead letter is moved back atomically, so
concurrent replays never enqueue a task twice. A replay only takes the dead letters that exist
when it starts: tasks that fail again during the replay stay dead. Entries whose envelope could
not be decoded are listed but never replayed.
Tracing
pip install "mayu[otel]"
With OpenTelemetry installed and a tracer provider configured, send() creates a send <task>
producer span and stores its W3C traceparent in the task envelope. The worker runs the task inside a
process <task> consumer span that is its child, so spans your task creates (HTTP calls, database
queries) join the caller's trace. Retries, rate-limit deferrals and link= callbacks keep the original
trace context. A failed attempt marks its span as an error. Without OpenTelemetry installed, nothing
changes.
Benchmarks
mayu vs Celery, RQ, Dramatiq and streaQ at 64 concurrent tasks: no-op throughput, a realistic task
(JSON decode + 5 ms I/O + 1 ms CPU), memory at equal concurrency, and workers on Python 3.14t, 3.14 and
3.13 serving the same queues at once. Results and method:
bench/RESULTS.md. Reproduce with bench/setup.sh && bench/all.sh (needs Docker and uv).
Development
See CONTRIBUTING.md for setup, the test matrix and the pull request checklist.
Coding agents should also read AGENTS.md. The design is in
docs/specs/2026-10-11-mayu-architecture-design.md.
License
Licensed under either of Apache License, Version 2.0 or MIT license at your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in mayu by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
Metadata
Release files for mayu 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| mayu-0.2.0.tar.gz | 172.7 kB | With provenance | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| mayu-0.2.0-cp314-cp314t-manylinux_2_28_x86_64.whl | CPython 3.14 | CPython 3.14 free-threading | Linux glibc 2.28+ x86-64 | Details |
| mayu-0.2.0-cp314-cp314t-manylinux_2_28_aarch64.whl | CPython 3.14 | CPython 3.14 free-threading | Linux glibc 2.28+ ARM64 | Details |
| mayu-0.2.0-cp314-cp314t-macosx_11_0_arm64.whl | CPython 3.14 | CPython 3.14 free-threading | macOS 11.0+ ARM64 | Details |
| mayu-0.2.0-cp314-cp314t-macosx_10_12_x86_64.whl | CPython 3.14 | CPython 3.14 free-threading | macOS 10.12+ x86-64 | Details |
| mayu-0.2.0-cp314-cp314-manylinux_2_28_x86_64.whl | CPython 3.14 | CPython 3.14 | Linux glibc 2.28+ x86-64 | Details |
| mayu-0.2.0-cp314-cp314-manylinux_2_28_aarch64.whl | CPython 3.14 | CPython 3.14 | Linux glibc 2.28+ ARM64 | Details |
| mayu-0.2.0-cp314-cp314-macosx_11_0_arm64.whl | CPython 3.14 | CPython 3.14 | macOS 11.0+ ARM64 | Details |
| mayu-0.2.0-cp314-cp314-macosx_10_12_x86_64.whl | CPython 3.14 | CPython 3.14 | macOS 10.12+ x86-64 | Details |
| mayu-0.2.0-cp313-cp313-manylinux_2_28_x86_64.whl | CPython 3.13 | CPython 3.13 | Linux glibc 2.28+ x86-64 | Details |
| mayu-0.2.0-cp313-cp313-manylinux_2_28_aarch64.whl | CPython 3.13 | CPython 3.13 | Linux glibc 2.28+ ARM64 | Details |
| mayu-0.2.0-cp313-cp313-macosx_11_0_arm64.whl | CPython 3.13 | CPython 3.13 | macOS 11.0+ ARM64 | Details |
| mayu-0.2.0-cp313-cp313-macosx_10_12_x86_64.whl | CPython 3.13 | CPython 3.13 | macOS 10.12+ x86-64 | Details |
Total release size: 21.2 MB
Release files / mayu-0.2.0.tar.gz
| Download URL | mayu-0.2.0.tar.gz |
|---|---|
| Size | 172.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d8e3885337239663e294d503746876379f7449b7bb142b9fbacf042f84581a72
|
|
BLAKE2b-256 checksum How to use checksums |
870b75d9198cfd2263b59bbad92488140b6aa5c5e30c7d5ec647b5da1bf9f940
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp314-cp314t-manylinux_2_28_x86_64.whl
| Download URL | mayu-0.2.0-cp314-cp314t-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
ceb12970c14718f0438915c001eb4b84facdd260a67fea71dcb2288e205260ad
|
|
BLAKE2b-256 checksum How to use checksums |
8f89f3e04897c4a566bd7acdf2fe30c599b208121126494461dcd87c0ac5ec77
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp314-cp314t-manylinux_2_28_aarch64.whl
| Download URL | mayu-0.2.0-cp314-cp314t-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading Linux glibc 2.28+ ARM64 |
|
SHA-256 checksum How to use checksums |
ef2647b98d058f7655fbe6342e9509e98d5cba7f53073f5b1e4ecc71f71055cd
|
|
BLAKE2b-256 checksum How to use checksums |
ce4b42c05cf538276055f63c3c3dc8293cd508607b540858cbe41603656921cc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp314-cp314t-macosx_11_0_arm64.whl
| Download URL | mayu-0.2.0-cp314-cp314t-macosx_11_0_arm64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
c7e0f22e7c5ecedb394670e4132b5ceb642fc71f6c5abcd98cd4d4d44e4538a8
|
|
BLAKE2b-256 checksum How to use checksums |
dc8536efd2f42e3c8a65ccf40bc40ff4cc099e467bb2fa7a4255fa7b9647f625
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp314-cp314t-macosx_10_12_x86_64.whl
| Download URL | mayu-0.2.0-cp314-cp314t-macosx_10_12_x86_64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.14 CPython 3.14 free-threading macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
107180100d477fd7c7525d9705fc99a0e083d059aab55df72112c3689ebf25e3
|
|
BLAKE2b-256 checksum How to use checksums |
e1232f00ef9abe58e3cf8e318357dc735fdc52985f2c15081fa21bd8cca487ee
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp314-cp314-manylinux_2_28_x86_64.whl
| Download URL | mayu-0.2.0-cp314-cp314-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.14 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
c9782fac61f53ada172664d28e29d4a2d09c7819c165bc6fa8b0f7109b2b51c3
|
|
BLAKE2b-256 checksum How to use checksums |
ed49e13a384cef7004c9c51941d67c1ddbe68243c51d771c0d5c05fa25a79b83
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp314-cp314-manylinux_2_28_aarch64.whl
| Download URL | mayu-0.2.0-cp314-cp314-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.14 Linux glibc 2.28+ ARM64 |
|
SHA-256 checksum How to use checksums |
5fd8576ddbad98117526b56d57c4cc6e958f190968fa30ab94599c36122f55b6
|
|
BLAKE2b-256 checksum How to use checksums |
6afe3a1205e302d0c5b77c027608fd6c61fbfc95b87a80926f249c722eba991e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp314-cp314-macosx_11_0_arm64.whl
| Download URL | mayu-0.2.0-cp314-cp314-macosx_11_0_arm64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.14 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
2d47aa885a3889a0ded73865143e14d50e25661cb59bb41442091f9afc2b7528
|
|
BLAKE2b-256 checksum How to use checksums |
601edd1b2745a0f6ce72e55d906e36fafa503033981c3b30a84a4b3c612064a3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp314-cp314-macosx_10_12_x86_64.whl
| Download URL | mayu-0.2.0-cp314-cp314-macosx_10_12_x86_64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.14 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
bdf08d1a575525a0dcb3555518a79bdd9ac9a8998fee52675106f39aae8b1def
|
|
BLAKE2b-256 checksum How to use checksums |
c75c4531ebbe2cd5b39a06422a7aee55e304a2500f269425f85ac407cf7284ce
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp313-cp313-manylinux_2_28_x86_64.whl
| Download URL | mayu-0.2.0-cp313-cp313-manylinux_2_28_x86_64.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.13 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
97f5acb6c0445a386a23f241f7371218dc7b05fb694b3ef13b381ae2622acecb
|
|
BLAKE2b-256 checksum How to use checksums |
73ae2d21114ce4dc5ef71c00688dd04343c9ee339a16682ccb435b32941de042
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp313-cp313-manylinux_2_28_aarch64.whl
| Download URL | mayu-0.2.0-cp313-cp313-manylinux_2_28_aarch64.whl |
|---|---|
| Size | 1.8 MB |
| Tags | CPython 3.13 Linux glibc 2.28+ ARM64 |
|
SHA-256 checksum How to use checksums |
c09ef69a9acaee0fc3579194f8a32c556d80474b9399ece7e8df1360b39b8b9a
|
|
BLAKE2b-256 checksum How to use checksums |
e4ab9a335b6e08edde7da578eaaf4d9ae89b8da47a8d4d943c264a2a932172dd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp313-cp313-macosx_11_0_arm64.whl
| Download URL | mayu-0.2.0-cp313-cp313-macosx_11_0_arm64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.13 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
2754d2f40c776a392daa4f2de7b85bf58f7bea5e1e3d8de3478ab9dc98472a15
|
|
BLAKE2b-256 checksum How to use checksums |
e807df3ab0d65c71ca280898ced8501d9297b94087e71c8fc9084a5199a4aa0c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logRelease files / mayu-0.2.0-cp313-cp313-macosx_10_12_x86_64.whl
| Download URL | mayu-0.2.0-cp313-cp313-macosx_10_12_x86_64.whl |
|---|---|
| Size | 1.7 MB |
| Tags | CPython 3.13 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
f950542e8287bb1f1ff97a8d76b7721160fce096eab5d94130e8c69359857703
|
|
BLAKE2b-256 checksum How to use checksums |
54a652339a0edc563028896018b294b9e3a898cee38cca651fc81b04144e8d38
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logSecurity for mayu 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 13 files in this release have been uploaded with provenance metadata.
Attestation for mayu-0.2.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 22, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.