Skip to main content

Distributed execution backend for Pyoco using NATS

Project description

pyoco-server (NATS backend for Pyoco)

This repository is an early-stage library that enables distributed execution of Pyoco workflows using NATS (JetStream) as the transport and durable queue.

Version: 0.4.0

Goals

  • Run Pyoco flows with a lightweight client/worker model.
  • Use NATS JetStream for a durable work queue (pull-based workers, tag routing).
  • Provide run status visibility (latest snapshot via JetStream Key-Value).
  • Provide an optional HTTP gateway so Pyoco users do not need to handle NATS.
  • Keep local dev/test setup simple: start NATS with nats-server-bin and manage it with nats-bootstrap.

Current state

This repo is under active construction. See docs/concept.md for the initial architecture and message design.

Quickstart

See docs/quickstart.md for a 5-minute end-to-end demo (HTTP submit -> NATS queue -> worker execution -> status query).

Tutorial (multi-worker)

See docs/tutorial_multi_worker.md for a more guided walkthrough with one server and multiple workers (CPU/GPU tags), plus ops endpoints.

Docs

  • Concept: docs/concept.md
  • Spec (contract): docs/spec.md
  • Architecture: docs/architecture.md
  • Library API (Python): docs/library_api.md
  • Config (.env): docs/config.md
  • Roadmap: docs/plan.md

Development

Prerequisites:

  • Python 3.10+
  • uv

Install dependencies:

uv sync

Run tests (will start an ephemeral NATS server for integration tests):

uv run pytest

HTTP Gateway (MVP)

Run the HTTP API (reads NATS settings from env):

export PYOCO_NATS_URL="nats://127.0.0.1:4222"
uv run uvicorn pyoco_server.http_api:create_app --factory --host 0.0.0.0 --port 8000

Tag routing

Runs are routed by subject:

  • publish to pyoco.work.<tag>
  • workers pull from one or more tags (OR semantics)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pyoco_server-0.4.0.tar.gz (106.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

pyoco_server-0.4.0-py3-none-any.whl (29.9 kB view details)

Uploaded Python 3

File details

Details for the file pyoco_server-0.4.0.tar.gz.

File metadata

  • Download URL: pyoco_server-0.4.0.tar.gz
  • Upload date:
  • Size: 106.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.10

File hashes

Hashes for pyoco_server-0.4.0.tar.gz
Algorithm Hash digest
SHA256 0181e575fa4c159e219adbe66be40d18e229d0741c13a2e5807b44ed6eb3838c
MD5 ca70294425941f3d4604a08001e97346
BLAKE2b-256 c138ca92f65fb0e21f458a910c669577a191315367492432b47794a04d5775da

See more details on using hashes here.

File details

Details for the file pyoco_server-0.4.0-py3-none-any.whl.

File metadata

  • Download URL: pyoco_server-0.4.0-py3-none-any.whl
  • Upload date:
  • Size: 29.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.10

File hashes

Hashes for pyoco_server-0.4.0-py3-none-any.whl
Algorithm Hash digest
SHA256 2b444bca79f24208cea75d11d4b42e0552e3eb445839f273f7bae28a7259ded9
MD5 2c72cf28c47498555ee05fd09d0746ce
BLAKE2b-256 b8324151aa2d6280ad1647cf1c829e32dc8c79c2aa2650164c6f60a302ae69d5

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page