Skip to main content

navigator-eventbus

Standalone async event bus + generic hooks fabric for aiohttp-based servers (Navigator ecosystem: ai-parrot, Flowtask, QuerySource, navigator-auth, ...).

Extracted from ai-parrot's EventBus v2 (FEAT-310) — same design (per-priority asyncio.Queue workers, backpressure, DLQ, meta-events, glob+severity subscription matching, memory/redis-pubsub/redis-streams transports, WebSocket/gRPC ingress, generic hooks with an open HookTypeRegistry) with the parrot.* coupling removed. See TOPICS.md for the topic-namespace registry shared across consuming apps.

Phase 1 of a 5-phase extraction plan (navigator-eventbus-extraction brainstorm, ai-parrot repo). This phase ships the bus core + generic hooks. Lifecycle events, the brokers port, and the ai-parrot migration itself land in later phases.

Install

uv pip install -e .            # core only
uv pip install -e ".[redis]"   # + redis pub/sub & streams backends
uv pip install -e ".[grpc]"    # + gRPC ingress
uv pip install -e ".[notify]"  # + async-notify default sender
uv pip install -e ".[scheduler]"  # + APScheduler-backed hook
uv pip install -e ".[watchdog]"   # + filesystem watchdog hook
uv pip install -e ".[mqtt]"       # + gmqtt-backed broker hook
uv pip install -e ".[all]"        # everything above
uv pip install -e ".[dev]"        # pytest, ruff, mypy

Usage

from navigator_eventbus import EventBus, Event, EventPriority

bus = EventBus()
await bus.emit("bus.example", {"hello": "world"})

Receiving webhooks

Mount one catch-all route and register endpoints — including at runtime, without touching the aiohttp router. Each delivery is authenticated with the endpoint's own HMAC scheme, optionally transformed, and emitted on hooks.webhook.<event_type>.

from aiohttp import web
from navigator_eventbus import EventBus
from navigator_eventbus.hooks import HookManager, WebhookListenerHook

bus, app = EventBus(), web.Application()
manager = HookManager(route_to_bus=True)
manager.set_event_bus(bus)

listener = WebhookListenerHook()
listener.register_endpoint(
    "/github",
    secret=GITHUB_WEBHOOK_SECRET,      # from the environment, never inline
    signature_scheme="github",         # X-Hub-Signature-256
    event_type_header="X-GitHub-Event",
    event_type_prefix="github",
    dedup_header="X-GitHub-Delivery",  # with dedup_ttl_seconds > 0
    preprocessor="myapp.hooks:summarize_pr",   # or preprocessor_fn=<callable>
)
manager.register(listener)
listener.setup_routes(app)             # POST /api/v1/hooks/webhook/github

The preprocessor may be sync or async, and receives an optional WebhookContext second argument. If it raises, the raw payload is emitted anyway and the sender still gets a 202 — a transform bug never costs you a delivery.

Exclude listener.base_path from any auth or body-rewriting middleware: HMAC verification needs the exact bytes on the wire.

For a single provider with real classification logic, subclass ProviderWebhookHook instead — it serves one fixed route and gives you _classify_event / _normalize_payload override points.

Sending webhooks

from navigator_eventbus.subscribers import WebhookDeliverySubscriber

delivery = WebhookDeliverySubscriber(
    url="https://partner.example/hooks/orders",
    patterns=["order.*"],
    secret=PARTNER_SECRET,
    signature_scheme="github",
)
delivery.attach(bus)
...
await bus.close()
await delivery.aclose()

Deliveries are queued and retried on 5xx/429 with capped, jittered backoff, so a slow endpoint never occupies a bus dispatch worker. Exhausted retries emit bus.webhook_delivery_failed.

Pull queues (SQS-style)

Needs the [redis] extra. A producer POSTs an envelope; consumers pull over HTTP with receipt handles, so they can be written in any language and never need Redis access.

from navigator_eventbus.queues import (
    QueueAPI, QueueConfig, QueueGroupConfig, QueueRegistry, QueueStore, ReceiptCodec,
)

registry = QueueRegistry(queues=[
    QueueConfig(
        name="orders",
        # Each group sees EVERY message; within a group, one consumer per message.
        groups=[QueueGroupConfig(name="billing"), QueueGroupConfig(name="analytics")],
        default_visibility_timeout_ms=30_000,
        max_receives=5,                       # then parked to the DLQ
        producer_tokens=(PRODUCER_TOKEN,),
        consumer_tokens=(CONSUMER_TOKEN,),
    ),
])
store = QueueStore(registry, redis_client, ReceiptCodec(RECEIPT_KEYS))
api = QueueAPI(registry, store)
api.setup_routes(app)                          # /api/v1/queues/...
await api.start()
# produce — the body IS an IngressEnvelope (unknown fields are rejected)
curl -X POST /api/v1/queues/orders/messages -H "X-API-Key: $PRODUCER_TOKEN" \
     -d '{"topic": "order.created", "payload": {"id": 7}}'

# consume — long-poll up to 20s, then delete with the receipt handle
curl -X POST /api/v1/queues/orders/messages/receive -H "X-API-Key: $CONSUMER_TOKEN" \
     -d '{"group": "billing", "max_messages": 10, "wait_time_seconds": 20}'

curl -X POST /api/v1/queues/orders/messages/delete -H "X-API-Key: $CONSUMER_TOKEN" \
     -d '{"group": "billing", "entries": [{"id": "1", "receipt_handle": "q1...."}]}'

# nack — visibility 0 releases it for immediate redelivery
curl -X POST /api/v1/queues/orders/messages/visibility -H "X-API-Key: $CONSUMER_TOKEN" \
     -d '{"group": "billing", "entries": [{"id": "1", "receipt_handle": "q1....",
          "visibility_timeout_seconds": 0}]}'

BUS_QUEUE_RECEIPT_KEYS (comma-separated; the first signs, all verify) is required — the API refuses to start without it. A per-process random key would break every multi-replica deployment and every restart.

Delivery is at-least-once: a consumer that dies after receive but before delete gets the message again once its lease expires. Consumers must be idempotent.

Queues are a plane parallel to the bus, not a layer on it — see the "Ingress vs. hooks vs. queues" table in CONTEXT.md for why. Two opt-in bridges connect them: QueueConfig.mirror_to_bus and QueueFeeder.

Configuration knobs

⚠️ Neutral defaults vs. legacy parrot:* deployments. Every prefix below defaults to a neutral evb:* value (this package has no knowledge of ai-parrot). If you are migrating an EXISTING ai-parrot deployment that already has data on parrot:events:* / parrot:stream:* Redis keys, you must override these to the legacy parrot:* values (constructor kwarg or the matching navconfig key below) — otherwise the migrated process will read/write a different set of keys than the deployed one, silently losing continuity. ai-parrot itself pins the legacy values when it migrates (phase 4 of the extraction plan); until then it keeps using its own in-tree copy of the bus.

All keys are read via navconfig (flattened env/TOML keys); every key below has a documented in-code default and can also be overridden per constructor kwarg where noted.

Keyed queues (ordered, head-blocking)

Needs the [redis] extra. A KeyedQueue (in navigator_eventbus.queues.keyed) gives per-key FIFO with at most one message in flight per key across any number of identical consumers — no leader. Messages of a key are ordered by a caller-supplied order score; the consumer leases the head of a key and returns one of four outcomes:

Outcome Effect
Outcome.ack() head removed, next one signalled
Outcome.retry(delay_ms=None, reason=...) head stays, key delayed (capped exponential backoff by default)
Outcome.wait(until_ms=None, reason=...) head stays, key blocked until the deadline (default wait_safety_ms) or a re-signal after it
Outcome.hold(reason) head stays, key parked until release_hold(key)
from navigator_eventbus.queues.keyed import KeyedConsumer, KeyedQueue, KeyedQueueConfig, Outcome

queue = KeyedQueue(KeyedQueueConfig(name="punches", signal=signal_cfg), redis_client, store=store)
await queue.push("worker-7", {"kind": "clock-in"}, message_id="p-1", order=1700000000)

async def handler(msg) -> Outcome:
    ...                                   # talk to the downstream
    return Outcome.ack()

await KeyedConsumer(queue).run(handler, stop=stop_event)
report = await queue.drain(handler, key_prefix="tenant-a:")   # administrative flush

How it stays live: a wake-up signal (a normal pull queue) is only a hint; the per-key sorted set is the order and the per-key lock the exclusion. The signal lease outlives the lock (signal.default_visibility_timeout_ms > lock_ttl_ms), consumers never receive() while paused, and recover() re-signals keys whose hint was lost. A crashed holder delays its key at most max(lock_ttl_ms, signal visibility) (its abandoned signal is reclaimed only after the visibility window); with no backlog a lost signal is recovered within signal visibility + recover_interval_ms, and while the signal group has undelivered entries (or the queue is paused) recover() waits for the backlog to drain instead of amplifying it.

Caveats:

  • push is idempotent by message_id, but after an ack the message is gone: re-pushing the same id from your own record recreates it, so the handler must check that record and ack without side effects when the work is already done. Redis is not the record of truth.
  • KeyedConsumer turns handler exceptions and timeouts into retry. A handler with a non-idempotent side effect must catch asyncio.CancelledError/TimeoutError itself and return its own outcome. A stale outcome (lock expired) raises LeaseLost and changes nothing.
  • recover() may add at most one duplicate signal per key per signal-visibility window (and one per overlapping pass). Duplicates are acked as busy/empty; liveness is preferred over exact-once signalling.
  • Direct lease() users must call lease.extend() for handlers longer than lock_ttl_ms.

BusCore / EventBus facade (evb.py, core.py)

navconfig key Constructor kwarg Default Notes
BUS_WORKERS workers= 4 Dispatch worker-pool size
BUS_QUEUE_SIZE queue_size= 1024 Max size of EACH per-priority queue
BUS_HANDLER_TIMEOUT handler_timeout= 30.0 (s) Per-handler asyncio.timeout
BUS_RETRY_ATTEMPTS retry_attempts= 3 Total delivery attempts per handler
BUS_RETRY_BASE_DELAY retry_base_delay= 0.1 (s) Backoff base delay
BUS_DEFAULT_BACKPRESSURE default_backpressure= "block" block | drop_oldest | reject
BUS_DRAIN_TIMEOUT drain_timeout= 5.0 (s) Deadline for graceful close() drain
BUS_TRANSPORT transport= (on EventBus) unset memory | pubsub | streams; backend= wins, then transport=, then use_redis
BUS_PUBLISH_ONLY publish_only= false Never start the backend consumer (producer processes)
BUS_OUTBOX_PATH outbox= (on EventBus) unset SQLite outbox file — enables the durable producer outbox
BUS_OUTBOX_MAX_ROWS max_rows= (on SqliteOutbox) 1000000 Row cap
BUS_OUTBOX_OVERFLOW overflow= (on SqliteOutbox) "reject_new" reject_new | drop_oldest
BUS_OUTBOX_MAX_AGE max_age= (on SqliteOutbox) 604800 Seconds before an unsent row expires
BUS_OUTBOX_RELAY_INTERVAL relay_interval= 5.0 (s) Relay retry loop period
BUS_CHANNEL_PREFIX channel_prefix= (on EventBus) "evb:events:" Redis pub/sub channel prefix — see warning above

Durable producer outbox

Use a persistent per-instance volume for the SQLite outbox; ephemeral container disk defeats restart recovery. Each process or worker must use its own outbox file—never share one file between gunicorn workers. The store takes an exclusive lock on <path>.lock, so a second store on the same file raises OutboxLockedError. Its expired/dropped/rejected/corrupt counters are per-process and reset on restart (each occurrence is also logged at ERROR); BusCore.stats() adds outbox_append_failed for envelopes published without durability. There is a residual window between a lifecycle emit and the scheduled bus.emit task, so a process crash can still lose that event. Duplicates are expected after an unconfirmed publish; consumers must deduplicate by event_id. There is no global ordering across workers; order by event timestamp when needed. A full outbox is an operational alarm, exposed through BusCore.stats().

Redis Streams backend (backends/redis_streams.py)

navconfig key Constructor kwarg Default Notes
BUS_STREAM_PREFIX stream_prefix= "evb:stream:" Stream key prefix (per topic-class) — see warning above
BUS_DEDUP_PREFIX dedup_prefix= "evb:events:dedup:" Dedup key prefix — see warning above
BUS_GROUP group= "evb-bus" Consumer-group name — see warning above

DLQ / Audit persistence (dlq.py, subscribers/audit.py)

navconfig key Constructor kwarg Default Notes
EVB_DSN dsn= (none — disables persistence) Postgres DSN for navigator.evb_dlq / navigator.evb_audit. Falls back to deriving a DSN from DBUSER/DBPWD/DBHOST/DBPORT/DBNAME navconfig keys (same derivation ai-parrot's parrot.conf.default_dsn used — but read directly, zero parrot.* coupling)

Notification alerting (subscribers/notification.py)

navconfig key Notes
BUS_ALERTS_DEDUP_WINDOW Default 300.0 s
BUS_ALERTS_CHANNEL_THROTTLE Default 10 (per channel)
BUS_ALERTS_THROTTLE_WINDOW Default 60.0 s
BUS_ALERTS_STORM_THRESHOLD Default 25 (ERROR+ events)
BUS_ALERTS_STORM_WINDOW Default 30.0 s

Ingress auth (ingress/websocket.py, ingress/grpc.py)

navconfig key Constructor kwarg Default Notes
BUS_INGRESS_TOKEN auth_token= (none — refuses all connections) Shared bearer token for WS/gRPC ingress

Development

uv sync --extra all --extra dev
uv run pytest tests/ -v
uv run ruff check src/ tests/
uv run mypy src/

Metadata

Release files for navigator-eventbus 0.4.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for navigator-eventbus 0.4.1
File Size Uploaded Provenance Actions
navigator_eventbus-0.4.1.tar.gz 872.8 kB With provenance Details for navigator_eventbus-0.4.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for navigator-eventbus 0.4.1
File Interpreter ABI Platform Actions
navigator_eventbus-0.4.1-py3-none-any.whl Python 3 none any Details for navigator_eventbus-0.4.1-py3-none-any.whl

Total release size: 1.1 MB

Security for navigator-eventbus 0.4.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for navigator_eventbus-0.4.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page