Skip to main content

autoshiploads-logging

The Shiploads logging spec for Python. Every app writes the same JSON line to the same weekly files as the Node apps (@autoshiploads/logging), so one log viewer can read them all. It plugs into Python's built-in logging, so Django, waitress and APScheduler lines land in the same file, tagged by where they came from.

app code, Django, waitress, APScheduler
  |
  v
stdlib logging (root logger)
  |
  v
one JSON line (ECS field names)
  |
  +--> logs/<service>_<Monday>.jsonl
  |      weekly, Hobart; 90 days kept
  |
  +--> logs/<service>.<stream>_<Monday>.jsonl
  |      optional longer-kept stream
  |
  +--> readable line on stderr
         (only in a terminal)

Install

uv add autoshiploads-logging==1.0.0

Python 3.11 or later. No dependencies except tzdata on Windows (time zone data, which Django already brings).

Django

settings.py: hand logging to the package, and put the middleware first.

LOGGING_CONFIG = None   # Django doesn't configure logging; the package does

MIDDLEWARE = [
    "autoshiploads_logging.django.RequestLoggingMiddleware",
    # ... the rest
]

AUTOSHIPLOADS_LOGGING = {
    "ignore_paths": ["/health/"],
    "trust_x_forwarded_for": True,   # behind IIS, so client.ip is the caller
}

from autoshiploads_logging import setup_logging  # at the end of settings.py

setup_logging("trackpallet", "3.1.0")

Each request gets one http_request line (path only, never the query string) and a trace id that every line logged while handling it carries, Django's own error lines included. The id comes from the caller's X-Request-Id when it's safe, and is sent back in that header.

The catalogue and logging

One module in the app names every business event:

# trackpallet/logging_events.py
from autoshiploads_logging import define_events

events = define_events(
    pallet_recorded="business",
    weekly_report_sent="business",
)
from autoshiploads_logging import get_logger
from .logging_events import events

log = get_logger(__name__)

log.event(events.pallet_recorded, f"Pallet recorded: {store}", {"store": store, "pallets": n})
log.info("Report email queued")                                 # category system
log.error("CI query failed", error=exc, data={"store": "CAM"})
log.event(events.weekly_report_sent, "...", data, level="error", duration_ms=ms)

Plain logging.getLogger(...) calls keep working and produce valid lines; any extra= values become data. Names are snake_case; run_started, run_finished, http_request and log_files_deleted are written by the package. Categories: request, business, job, system.

Jobs, commands and scripts

A fresh trace id per run, and run_started / run_finished with the result and duration. A failure is logged at error and re-raised; SystemExit(0) counts as success.

from autoshiploads_logging import run_job

with run_job(log, "weekly_report"):
    ...

@run_job(log, "nightly_import")      # APScheduler job, management command
def nightly_import(): ...

A script outside an app calls setup_logging("<script>", "<version>", log_dir=r"C:\ProgramData\Shiploads\Logs\<script>") first (spec section 5).

A longer-kept stream

For events people ask about long after (COB's picks, kept a year): those loggers write to their own file with their own retention, and not to the main file.

from autoshiploads_logging import StreamConfig, setup_logging

setup_logging(
    "cob",
    "2.4.0",
    streams={"picks": StreamConfig(loggers=("stock_movement",), retention_days=364)},
)

Options

Option Default
service, version required
level LOG_LEVEL env, else info
log_dir LOG_DIR env, else <cwd>/logs
retention_days LOG_RETENTION_DAYS, else 90
time_zone LOG_TIME_ZONE, else Hobart
output file; stdout for containers
console LOG_CONSOLE 1/0, else terminal only
redact_keys none (adds to the spec's list)
streams none
library_categories adds to the built-in map

LOG_LEVEL wins over level, so a host can turn on debug without a code change. A bad value from the environment is reported once as a warn line at startup and the default used.

Files, retention, safety

  • logs/<service>_<YYYY-MM-DD>.jsonl, dated by the Monday the week starts in Hobart; each line picks its file from its own time, so rotation needs no timer.
  • One write per line on an append-only handle, so several processes (waitress, a management command, the dev reloader) can share a file without mixing lines. On Windows this opens the file with FILE_APPEND_DATA directly: Python's own append mode goes through the C runtime, which seeks and then writes in two steps, and in testing three processes writing one file that way produced broken lines.
  • Retention runs at setup and every 24 hours, by the date in the file name only: a weekly file goes once its Sunday is past retention; WinSW's daily *.yyyyMMdd.out.log / .err.log once their day is; a stream's files by the stream's own retention. Nothing else is touched.
  • A failed write is never raised into the app; the first one prints a line to stderr.
  • Keys in the spec's never-log list (password, token, authorization, cookie, ...) are dropped from data at any depth. Objects JSON can't hold are written as their type name only, never their repr (Django's request repr includes the query string).

Changing this package

The spec (~/.claude/logging-spec.md) changes first, then this package (spec section 9). Adding a field or option: minor version. Renaming, removing or retyping a field: major version. See the repo README for releasing.

Metadata

Release files for autoshiploads-logging 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for autoshiploads-logging 1.0.0
File Size Uploaded Provenance Actions
autoshiploads_logging-1.0.0.tar.gz 17.8 kB Without provenance Details for autoshiploads_logging-1.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for autoshiploads-logging 1.0.0
File Interpreter ABI Platform Actions
autoshiploads_logging-1.0.0-py3-none-any.whl Python 3 none any Details for autoshiploads_logging-1.0.0-py3-none-any.whl

Total release size: 42.0 kB

Release files / autoshiploads_logging-1.0.0.tar.gz

Download URL autoshiploads_logging-1.0.0.tar.gz
Size 17.8 kB
Tags Source
SHA-256 checksum
How to use checksums
ade6bc91ff26a03b7992be97a37ff3f7d3d4b5df77fdbd0c5741ab992f3565d9
BLAKE2b-256 checksum
How to use checksums
6ee369385c0aa00a9a33d384a55bdc0949d1c4c419649d059273953c939ad91b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.15 {"installer":{"name":"uv","version":"0.11.15","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / autoshiploads_logging-1.0.0-py3-none-any.whl

Download URL autoshiploads_logging-1.0.0-py3-none-any.whl
Size 24.2 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b7440ced081643caa5444af5eb66c816b9cfbce3baf6a08ca40fa116081a8db7
BLAKE2b-256 checksum
How to use checksums
e6352d9eef2215e8ce3deeb8156e534314ca3db1dfbca1a02d714564d775d76a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.15 {"installer":{"name":"uv","version":"0.11.15","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Security for autoshiploads-logging 1.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.0.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page