demystify-events

Durable jobs and events for Python — the asyncio mirror of the TypeScript @demystify/events runtime. Postgres-backed (pgmq) with a transactional outbox, idempotent consumers, exponential backoff + full jitter retries, and a dead-letter queue — with a full-fidelity in-memory driver so the same code runs offline with no database.

Part of the Demystify Core Services suite.

Install

pip install demystify-events
# real Postgres driver (psycopg + connection pool):
pip install "demystify-events[pg]"

Quickstart (offline, no database)

from demystify_events import create_events_client
from demystify_events.adapters.memory import MemoryQueueDriver

client = create_events_client(driver=MemoryQueueDriver())

# enqueue a job
await client.enqueue("email.send", {"to": "user@example.com"})

# process it (idempotent, retried with backoff+jitter, DLQ on poison)
async def handler(ctx, job):
    ...  # do work; effects committed in the same transaction as the ack
await client.work("email.send", handler)

Switch to real Postgres by passing the psycopg driver (from the [pg] extra) with a database_url — the handler contract is identical.

Postgres requirements

PsycopgPgmqDriver runs on the pgmq Postgres extension — install it in the target database first (CREATE EXTENSION pgmq;). Stock postgres images do not ship pgmq; use one that does, e.g. ghcr.io/demystify-systems/postgres-pgvector-pgmq (ours, pgvector + pgmq combined) or quay.io/tembo/pg16-pgmq. If the extension is missing, driver operations raise an actionable RuntimeError ("pgmq extension not available in this database. Run CREATE EXTENSION pgmq; ...") with the original psycopg error chained as __cause__, instead of a cryptic relation "pgmq.q_x" does not exist.

MemoryQueueDriver (alias EphemeralQueueDriver — the same class) is ephemeral: jobs live in process memory and are lost on restart. Outside pytest runs its constructor emits a warnings.warn once per process; set DMSTFY_EVENTS_ALLOW_EPHEMERAL=1 to silence it when running in-memory on purpose.

What's in the box

• create_events_client / EventsClient — enqueue, transactional enqueue_in_tx, workers
• contract_backoff, full_jitter_backoff, BackoffConfig — retry schedules
• ContractRegistry — JSON-schema event-type validation
• make_envelope / validate_envelope — canonical event envelopes
• Adapters: MemoryQueueDriver (offline) and psycopg (Postgres/pgmq)

The published wheel is self-contained — it does not vendor demystify-platform-contracts; contract convergence is asserted by the test suite. See the module README for the delivery guarantees, chaos-test results, and the /ops/v1 API.

License

MIT © Demystify Systems