sysstra-logging

Shared structured (JSON-line) logging for Sysstra services. Stdlib-only — zero install dependencies — so lean scripts (data-collection streamers, cron jobs) can pick it up without pulling in the full sysstra SDK (pandas/numpy/numba/redis/pymongo).

from sysstra_logging import get_logger, bind, clear_context

logger = get_logger("syss_sha_eios", log_file="logs/syss_sha_eios.log",
                    service="sysstra-trading-strategies")

bind(mode="lt", request_id=request_id, strategy="syss_sha_eios")
logger.info("order placed", extra={"order_id": order_id})

The identity block

Every line carries these 13 fields, in this order, blank when unknown — whether or not the writing service knows anything about them:

ts  level  schema  service  component  env  host  tenant  market  event  status  logger  message

That is what lets one Alloy config and one dashboard panel work across every repo: a consumer can group by repo, host, market or environment without knowing which service wrote the line.

• service names the repo (sysstra-data-collection-in) and comes from each repo's own SERVICE constant in its *_common.py. SERVICE in the environment overrides it; the logger name is the last-resort fallback.
• logger is the per-script or per-request logger name (syss_sha_eod-vt-<request_id>). It used to be passed into service's slot, so every line reported a request id as its service and nothing could group by repo.
• component says which part of a repo a line came from (streamer, common_runner, instance-orchestrator) — bind it where a repo has meaningful sub-parts.
• env defaults to prod; host falls back to socket.gethostname() because HOSTNAME is a shell variable and is often not exported.

Precedence for each field: extra= → bind() → environment → default.

Domain fields

Everything else appears only where it applies — request_id, mode, strategy on the trading repos; segment, session on data collection; job, run_id on batch work. They are deliberately not blank-padded: a request_id: "" on a collection line would be indistinguishable from a trading line that failed to populate one, and collection has no such concept.

request_id is the Mongo {mode}_requests._id and is never generated. Batch jobs get run_id instead, so a query for one can never pick up the other.

Batch jobs

from sysstra_logging import get_logger, job_run

logger = get_logger("s3_sync", log_file=..., service="sysstra-data-services")

with job_run(logger, "s3_sync"):
    sync_bucket()

Emits job_start / job_complete / job_failed with duration_s, and binds job and run_id for the duration so every line the job logs in between is correlated too. Failures re-raise, so a cron job still exits non-zero.

Notes

Set LOG_FORMAT=text for a human-readable formatter during local dev / tail -f; the default is single-line JSON.

clear_context() must be called at the start of every unit of work (Celery task, HTTP request) that reuses a process/thread — otherwise bound fields from a previous task (most dangerously mode, paper vs. real-money) leak into the next one's log lines.

See sysstra_logging/logging_utils.py for the full API (bound(), get_context(), redirect_stdout_to(), ScrubFilter).