Skip to main content

Milvaion Python Worker SDK

Run Milvaion jobs in Python. A Python worker shows up in the dashboard like a .NET worker. The scheduler dispatches jobs to it, and the dashboard shows its live logs, status, retries, cancellation and timeouts.

pip install milvaion-worker

It requires Python 3.10 or later, and connects to the same RabbitMQ and Redis instances as your Milvaion API.

Quick start

from milvaion import JobContext, PermanentJobError, Worker

worker = Worker(
    worker_id="report-worker",
    rabbitmq_url="amqp://guest:guest@localhost:5672/",
    redis_url="redis://localhost:6379",
)


@worker.job("BuildReportJob", max_retries=3, execution_timeout=600)
async def build_report(ctx: JobContext) -> dict:
    data = ctx.data or {}

    if "month" not in data:
        raise PermanentJobError('Job data needs "month".')  # no retries, straight to the DLQ

    ctx.log(f"Building the report for {data['month']}")
    rows = await fetch_rows(data["month"])  # cancelled if the job is cancelled or times out

    return {"rows": len(rows)}  # shown as the occurrence result


@worker.job("CleanupJob")
def cleanup(ctx: JobContext) -> None:  # plain functions run in a thread
    for path in old_files():
        ctx.raise_if_aborted()  # threads cannot be stopped from outside; check in long loops
        path.unlink()


if __name__ == "__main__":
    worker.run()  # blocks; SIGINT/SIGTERM shut down gracefully

Then create a job in the dashboard: pick the worker report-worker and the job BuildReportJob, and set a schedule.

To run the worker inside your own event loop, call await worker.start() and await worker.stop() instead of run().

Jobs

@worker.job(name, **options) or worker.add_job(name, handler, JobOptions(...)) registers a handler under the name the dashboard dispatches it by. Names are unique within a worker and matched case-insensitively.

Handlers can be async def or plain functions. A plain function runs in a thread, so blocking code does not stall other jobs.

The handler receives a JobContext:

Member Description
ctx.data Parsed job data, or None. Data that is not valid JSON fails the job permanently.
ctx.raw_data Job data as the raw JSON string.
ctx.occurrence_id Id of this execution. It stays the same across retries and redeliveries, so you can use it as an idempotency key.
ctx.retry_count 0 on the first attempt, then 1, 2 and so on.
ctx.is_aborted, ctx.abort_reason, ctx.raise_if_aborted() Whether the job was cancelled, timed out or interrupted by shutdown.
ctx.log / debug / warn / error(message, error=None, data=None) Write to the occurrence log in the dashboard.
ctx.job The scheduled job as the API sent it.

How the handler ends decides the outcome:

The handler... Outcome
returns a value Completed. A string is stored as-is; anything else is stored as JSON.
raises Retried with exponential backoff. After max_retries, the occurrence is Failed and the job goes to the dead letter queue.
raises PermanentJobError Failed without retrying, and the job goes to the dead letter queue.
runs past its timeout TimedOut
is cancelled from the dashboard Cancelled
is still running when the worker stops The job goes back to the queue and runs again.

When a job is aborted, an async handler is cancelled, so it gets asyncio.CancelledError at its next await. A plain function cannot be stopped from outside. The occurrence still settles right away, but the thread keeps running until the function returns, so check ctx.raise_if_aborted() in long loops.

Job options

Option Default Description
max_parallel_jobs min(10, worker max_parallel_jobs) Jobs of this type that may run at once on this instance
execution_timeout worker default (3600 s) Seconds. 0 disables the timeout. A timeout set on the job in the dashboard takes precedence.
max_retries 3 Retries after the first failure
base_retry_delay 5 Retry n waits base × 2^(n-1) seconds
data_schema none JSON Schema of the job data, as a dict or string. The dashboard builds the job data form from it; with Pydantic, use Model.model_json_schema().
result_schema none JSON Schema of the result
routing_pattern {worker_id}.{name without "Job"}.* RabbitMQ routing pattern
consumer_id {name without "Job"}-consumer Shown in the dashboard

Worker options

Pass these to Worker(...) or WorkerOptions(...):

Option Default Description
worker_id required Identifies the worker application. Use the same value on every replica.
rabbitmq_url amqp://guest:guest@localhost:5672/
queue_type classic Must match the API's MilvaionConfig:RabbitMQ:QueueType
redis_url redis://localhost:6379 Redis only carries cancellation requests. None disables them.
redis_key_prefix Milvaion:JobScheduler: Must match the API's MilvaionConfig:Redis:KeyPrefix
max_parallel_jobs CPU count × 2 Capacity reported for this instance
execution_timeout 3600 Default job timeout in seconds
heartbeat_interval 30
job_heartbeat_interval 60 Keeps long jobs from being flagged as zombies
shutdown_timeout 30 How long stop() lets running jobs finish
startup_timeout 30 How long start() waits for RabbitMQ and Redis
version 1.0.0 Shown in the dashboard
logger logging.getLogger("milvaion")

For containers, Worker(WorkerOptions.from_env()) reads the following variables:

  • MILVAION_WORKER_ID
  • MILVAION_RABBITMQ_URL
  • MILVAION_RABBITMQ_QUEUE_TYPE
  • MILVAION_REDIS_URL (false disables Redis)
  • MILVAION_REDIS_KEY_PREFIX
  • MILVAION_MAX_PARALLEL_JOBS
  • MILVAION_EXECUTION_TIMEOUT_SECONDS
  • MILVAION_HEARTBEAT_INTERVAL_SECONDS

Reliability

  • Reconnects. If RabbitMQ goes away, the connection is restored with its queues and consumers, and the worker registers again. The Redis listener reconnects and resubscribes.
  • At-least-once delivery. A message is acknowledged only after its outcome has been reported, so a crash makes the job run again. Make handlers idempotent; ctx.occurrence_id helps with that.
  • Graceful shutdown. stop() does the following in order:
    1. Stops taking new jobs.
    2. Waits up to shutdown_timeout for running jobs to finish.
    3. Returns the jobs that are still running to the queue.
    4. Flushes logs and tells the API the instance has left.

How it works

The SDK implements the language-neutral worker protocol: the RabbitMQ queues and message formats the .NET SDK uses.

Development

python -m venv .venv && . .venv/bin/activate
pip install -e ".[test]"
pytest
python examples/basic_worker.py   # configure with MILVAION_* variables

Metadata

Release files for milvaion-worker 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for milvaion-worker 1.0.0
File Size Uploaded Provenance Actions
milvaion_worker-1.0.0.tar.gz 26.1 kB With provenance Details for milvaion_worker-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for milvaion-worker 1.0.0
File Interpreter ABI Platform Actions
milvaion_worker-1.0.0-py3-none-any.whl Python 3 none any Details for milvaion_worker-1.0.0-py3-none-any.whl

Total release size: 52.7 kB

Release files / milvaion_worker-1.0.0.tar.gz

Download URL milvaion_worker-1.0.0.tar.gz
Size 26.1 kB
Tags Source
SHA-256 checksum
How to use checksums
cd2ecb18119f8513a53f17ff8dd22eb11e30b2eb9b87338409ce94dbf190790e
BLAKE2b-256 checksum
How to use checksums
be4e7f445b61e7b2dcfbb5caa7f45a8e11c703d0c5785f2bc1c8063fde8681e8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.

Transparency log

Release files / milvaion_worker-1.0.0-py3-none-any.whl

Download URL milvaion_worker-1.0.0-py3-none-any.whl
Size 26.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
46461201f6834901090446687d7f3f75e702d51f34f08dc0863e4da848a59b66
BLAKE2b-256 checksum
How to use checksums
7733cbf5d5daa9c3220f8300ee5309d42107fa8b3e14df00ece3843a51abeb7d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.

Transparency log

Security for milvaion-worker 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for milvaion_worker-1.0.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page