mgf-logship 0.1.0

Centralized log shipping for mgf-common consumers — a fail-open logging handler that batches canonical LogRecord JSON + statistics and POSTs them to a collector over a versioned, language-neutral wire protocol, plus a store-now stub collector. Sibling of mgf-common under the mgf.* namespace.

mgf-logship

Centralized log shipping for the Magogi federation — the Layer-B half of the unified-logging program (see mgf-common/docs/inprogress/UNIFIED_LOGGING_DESIGN.md, decision D4: ship + protocol now, stub the server).

A consumer attaches LogShipper to its logging tree; it batches the canonical mgf-common LogRecord JSON (the same shape every runtime emits — OB-10), redacts each record, and POSTs them to a collector over a versioned, language-neutral wire protocol. A store-now stub collector (FastAPI + sqlite) is included so the protocol works end-to-end today; analysis and presentation are a later program.

Install

pip install mgf-logship                # shipper only (stdlib transport)
pip install mgf-logship[collector]     # + the FastAPI/sqlite stub collector

Ship logs from an app

import logging
from mgf.logship import LogShipper
from mgf.common.observability import current_log_stats  # optional stats shipping

shipper = LogShipper(
    "https://collector.internal",
    token="…",                       # Authorization: Bearer …
    stats_provider=current_log_stats,  # also ship LogStats every 30s
)
logging.getLogger().addHandler(shipper)
# … on shutdown: shipper.close()  (flushes the queue)

LogShipper is fail-open (a slow/down collector never blocks or crashes the app), bounded (the queue is capped; oldest dropped on overflow, counted in dropped_count), and redacts before ship (off-box data is scrubbed).

Run the stub collector

from mgf.logship.collector import create_collector_app
app = create_collector_app(db_path="logship.db", token="…")
# uvicorn module:app

Endpoints: POST /ingest (NDJSON of LogRecords), POST /stats (a LogStats JSON), GET /health. It persists to sqlite and stops there.

The wire protocol

mgf.logship._protocol is the source of truth: SHIP_PROTOCOL_VERSION, the endpoint paths, the X-Logship-* headers, and the NDJSON framing. Deliberately HTTP + NDJSON so a Zig or TS shipper can talk to the same collector with nothing but a socket and a JSON encoder.

License

MIT — see LICENSE.