pynenc-rustvello

Rust-powered backend plugin for pynenc distributed task orchestration — via the rustvello engine.

pynenc-rustvello provides high-performance Rust implementations of all pynenc storage interfaces (broker, orchestrator, state backend, trigger store, and client data store) across 6 storage backends, with optional native Rust orchestration for maximum throughput.

pynenc tasks remain Python code executed by pynenc's own runners (ThreadRunner, ProcessRunner, etc.). This plugin replaces only the storage layer — giving you Rust-level performance for state management, message queuing, and orchestration coordination without changing how your tasks run.

For full Rust task execution, use rustvello directly.

Components

Installation

pip install pynenc-rustvello

Requires Python ≥ 3.12, pynenc ≥ 0.1.0, and rustvello ≥ 0.1.0.

Quick Start

pynenc-rustvello registers itself as a pynenc plugin via entry points. Once installed, Rust backends are available through pynenc's builder:

from pynenc import PynencBuilder

# In-memory Rust backend (fastest for testing)
app = PynencBuilder().app_id("my_app").rustvello_mem().build()

# SQLite Rust backend (persistent, zero-config)
app = PynencBuilder().app_id("my_app").rustvello_sqlite().build()

# Redis Rust backend (production)
app = (
    PynencBuilder()
    .app_id("my_app")
    .rustvello_redis(redis_url="redis://localhost:6379")
    .build()
)

@app.task
def add(x: int, y: int) -> int:
    return x + y

result = add(1, 2).result  # 3

Mixing Backends

Combine different storage backends per component:

app = (
    PynencBuilder()
    .app_id("my_app")
    .rustvello_redis_broker(redis_url="redis://localhost:6379")
    .rustvello_postgres_state(postgres_url="postgresql://localhost/mydb")
    .rustvello_postgres_orchestrator(postgres_url="postgresql://localhost/mydb")
    .rustvello_redis_trigger(redis_url="redis://localhost:6379")
    .rustvello_redis_cds(redis_url="redis://localhost:6379")
    .build()
)

Architecture

pynenc-rustvello is a storage plugin — it replaces pynenc's storage layer with Rust-backed implementations while keeping task execution in Python.

┌────────────────────────────────────────────────┐
│                   pynenc app                   │
│                                                │
│  Tasks (Python)  ←→  Runner (Python)           │
│       │                    │                   │
│       ▼                    ▼                   │
│  ┌─────────────────────────────────────────┐   │
│  │     pynenc-rustvello (storage layer)    │   │
│  │  Broker · Orchestrator · State Backend  │   │
│  │  Trigger · Client Data Store            │   │
│  │         (all backed by Rust FFI)        │   │
│  └─────────────────────────────────────────┘   │
└────────────────────────────────────────────────┘

For users who want full Rust task execution (not just storage), use rustvello directly.

Documentation

Full documentation: pynenc-rustvello.readthedocs.io

Ecosystem

Development

# Clone the repository
git clone https://github.com/pynenc/pynenc_rustvello.git
cd pynenc_rustvello

# Install in dev mode
uv sync --all-extras

# Run tests
uv run python -m pytest tests/unit/ -q

# Lint
uv run ruff check pynenc_rustvello/ tests/
uv run ruff format --check pynenc_rustvello/ tests/

License

MIT