pynenc-mongo 0.3.0

MongoDB plugging for pynenc, a distributed task queue for Python.

Pynenc MongoDB Plugin

Full-stack MongoDB backend for Pynenc distributed task orchestration

---

Documentation: https://pynenc-mongodb.readthedocs.io

Pynenc Documentation: https://docs.pynenc.org

Source Code: https://github.com/pynenc/pynenc-mongodb

---

The pynenc-mongo plugin provides all five Pynenc backend components running on MongoDB, with automatic document chunking for large payloads and a pseudo-atomic ownership protocol for distributed invocation management.

Components

Component	Class	Role
Orchestrator	MongoOrchestrator	Invocation lifecycle, ownership consensus & blocking control
Broker	MongoBroker	FIFO message queue using MongoDB collections
State Backend	MongoStateBackend	Persistent state, results & exceptions with auto document chunking
Client Data Store	MongoClientDataStore	Argument caching with compression for large payloads
Trigger	MongoTrigger	Event-driven & cron-based scheduling with distributed claims

Installation

pip install pynenc-mongo

The plugin registers itself automatically via Python entry points when installed.

Quick Start

from pynenc import PynencBuilder

app = (
    PynencBuilder()
    .app_id("my_app")
    .mongo(url="mongodb://localhost:27017/pynenc")  # all components on MongoDB
    .process_runner()
    .build()
)

@app.task
def add(x: int, y: int) -> int:
    return x + y

result = add(1, 2).result  # 3

.mongo() registers every component at once. Start a runner with:

pynenc --app=tasks.app runner start

Configuration

Builder Parameters

# URL-based (recommended)
app = (
    PynencBuilder()
    .app_id("my_app")
    .mongo(url="mongodb://localhost:27017/pynenc")
    .multi_thread_runner(min_threads=2, max_threads=8)
    .build()
)

# Individual parameters
app = (
    PynencBuilder()
    .app_id("my_app")
    .mongo(
        host="localhost",
        port=27017,
        db="pynenc",
        username="admin",
        password="secret",
        auth_source="admin",
    )
    .process_runner()
    .build()
)

Component-Specific Configuration

app = (
    PynencBuilder()
    .app_id("my_app")
    .mongo(url="mongodb://localhost:27017/pynenc")
    .mongo_client_data_store(
        min_size_to_cache=1024,          # cache arguments > 1KB
        local_cache_size=1000,           # local LRU cache entries
        max_size_to_cache=16777216,      # max 16MB per document
    )
    .mongo_trigger(
        scheduler_interval_seconds=60,
        enable_scheduler=True,
    )
    .build()
)

Environment Variables

PYNENC__MONGO__MONGO_URL="mongodb://localhost:27017/pynenc"
# Or individual parameters:
PYNENC__MONGO__MONGO_HOST="localhost"
PYNENC__MONGO__MONGO_PORT=27017
PYNENC__MONGO__MONGO_DB="pynenc"

Connection URLs

.mongo(url="mongodb://localhost:27017/pynenc")                          # Standard
.mongo(url="mongodb://user:pass@localhost:27017/pynenc?authSource=admin")  # With auth
.mongo(url="mongodb+srv://cluster.example.net/pynenc")                  # Atlas SRV

Requirements

• Python >= 3.12
• Pynenc >= 0.3.0
• pymongo >= 3.12.2
• A running MongoDB server

Related Plugins

• pynenc-redis: Full-stack Redis backend
• pynenc-rabbitmq: RabbitMQ broker (pairs with MongoDB for state/orchestrator/triggers)

License

This project is licensed under the MIT License - see the LICENSE file for details.