IncidentLogging

A Python logging handler that stays silent during normal operation and only writes logs when something goes wrong.

The problem

Verbose debug logging helps diagnose issues, but writing every DEBUG and INFO message to a file or console creates noise that obscures what matters. The usual workaround — raising the log level to WARNING — means you lose the context that would have explained why the warning happened.

How it works

IncidentHandler wraps any standard logging.Handler. It buffers DEBUG and INFO records silently. The moment a WARNING, ERROR, or CRITICAL is emitted, it flushes the buffered context followed by the triggering message — then clears the buffer and starts over.

Normal operation:        DEBUG INFO DEBUG INFO DEBUG INFO  →  (nothing written)
Something goes wrong:    DEBUG INFO DEBUG INFO ERROR       →  DEBUG INFO DEBUG INFO ERROR

The buffer holds the most recent N records (default 30). Older records are dropped as new ones arrive, so the buffer always contains the last N lines of context leading up to the problem.

Installation

Via pip (recommended)

pip install incident-logging

Manual

No dependencies outside the standard library. Copy incident_logging.py directly into your project.

Requires Python 3.8+.

Usage

Basic — wrap the default stderr handler

import logging
from incident_logging import IncidentHandler

logger = logging.getLogger("myapp")
logger.setLevel(logging.DEBUG)
logger.addHandler(IncidentHandler())

logger.debug("connecting to database")   # buffered
logger.info("query executed in 4 ms")    # buffered
logger.error("connection pool exhausted") # flushes both lines above, then this

With a custom handler and buffer size

import logging
from incident_logging import IncidentHandler

stream = logging.StreamHandler()
stream.setFormatter(logging.Formatter("%(levelname)s %(name)s: %(message)s"))

logger = logging.getLogger("myapp")
logger.setLevel(logging.DEBUG)
logger.addHandler(IncidentHandler(target_handler=stream, buffer_size=50))

With RotatingFileHandler

The log file stays empty during normal operation and only grows when an incident occurs — keeping file sizes minimal while preserving full diagnostic context when you need it.

import logging
from logging.handlers import RotatingFileHandler
from incident_logging import IncidentHandler

rotating = RotatingFileHandler("app.log", maxBytes=1024 * 1024, backupCount=5)
rotating.setFormatter(logging.Formatter("%(asctime)s %(levelname)-8s %(name)s: %(message)s"))

logger = logging.getLogger("myapp")
logger.setLevel(logging.DEBUG)
logger.addHandler(IncidentHandler(target_handler=rotating, buffer_size=30))

API

IncidentHandler(target_handler=None, buffer_size=30)

The handler passes through WARNING, ERROR, and CRITICAL records immediately (after flushing the buffer). DEBUG and INFO records are only ever written as part of a flush.

Comparison to MemoryHandler

Python's standard library includes logging.handlers.MemoryHandler, which is the closest built-in equivalent. Here's how they differ:

The practical difference: MemoryHandler doesn't miss anything in the log. IncidentHandler behaves like a ring buffer — it silently discards unimportant records that are too old to matter and always preserves the most recent context window.

Recommended pattern: combine both

Use a regular handler for full bookkeeping and an IncidentHandler for focused incident output. The regular handler captures everything for audit trails or offline analysis; the IncidentHandler surfaces only what's relevant when something goes wrong.

import logging
from logging.handlers import RotatingFileHandler
from incident_logging import IncidentHandler

logger = logging.getLogger("myapp")
logger.setLevel(logging.DEBUG)

# Full audit log — every record, always
audit = RotatingFileHandler("audit.log", maxBytes=10 * 1024 * 1024, backupCount=5)
audit.setFormatter(logging.Formatter("%(asctime)s %(levelname)-8s %(message)s"))
logger.addHandler(audit)

# Incident log — only emits when WARNING or above fires, with recent context
incident = RotatingFileHandler("incidents.log", maxBytes=1024 * 1024, backupCount=3)
incident.setFormatter(logging.Formatter("%(asctime)s %(levelname)-8s %(message)s"))
logger.addHandler(IncidentHandler(target_handler=incident, buffer_size=30))

audit.log grows continuously and is the source of truth. incidents.log stays small and contains only the context windows around each problem — easy to tail in production or attach to a bug report.

Running the demos

python3 demo.py

Running the tests

python3 -m unittest test_incident_logging -v