django-logpilot

Structured logging and optional alerting for Django with minimal configuration: readable or JSON logs, rotating files, request correlation IDs, and Slack or email notifications when you need them.

---

Features

• Console and optional rotating file output
• Text or JSON log format
• Request ID middleware (default header X-Request-ID on the way in and out) wired into log formatters
• Sensible defaults for the root logger (and optional per-logger Logpilot handlers when you use merge mode)
• Helper to redact sensitive keys in dict-shaped data you log
• Automatic creation of parent directories for file_path
• Optional alerts (Slack webhook, email) with deduplication

---

Requirements

• Python 3.10 or newer
• Django 4.2 or newer

---

Install

pip install django-logpilot

Install a specific version in production:

pip install "django-logpilot>=0.2.0,<0.3.0"

Editable install from a local clone:

pip install -e /path/to/django-logpilot

---

Quick setup (empty LOGGING)

Use this when you do not need a custom LOGGING dictionary.

INSTALLED_APPS = [
    # ...
    "django_logpilot",
]

MIDDLEWARE = [
    "django_logpilot.middleware.RequestIdMiddleware",
    # ... your other middleware
]

LOGGING = {}

DJANGO_LOGPILOT = {
    "format": "text",
    "level": "INFO",
    "outputs": ["console", "file"],
    "file_path": str(BASE_DIR / "logs" / "app.log"),
    "file_max_bytes": 5 * 1024 * 1024,
    "file_backup_count": 3,
    "alerts": {
        "enabled": False,
    },
}

After startup, loggers that propagate to the root (the usual logging.getLogger(__name__) pattern) send records to the configured outputs. Alerts run only if you enable them in alerts (see below).

---

Custom LOGGING (merge mode)

Use this when you already define LOGGING (custom formatters, root, or per-module loggers).

Enable merge

DJANGO_LOGPILOT_MERGE_LOGGING = True

If LOGGING contains a non-empty handlers dict and this flag is false, Logpilot does not install its handlers; only your configuration applies.

Referencing logpilot_* handlers by name

Django applies LOGGING during startup, before application ready() hooks. Logpilot normally adds handlers such as logpilot_file and logpilot_alerts while merging your dictionary. If a logger lists those handler names on the first dictConfig pass, Python raises:

ValueError: Unable to add handler 'logpilot_file'

Point LOGGING_CONFIG at Logpilot’s hook so the merge runs in the same pass as your LOGGING:

LOGGING_CONFIG = "django_logpilot.logging.logpilot_dict_config"

Keep DJANGO_LOGPILOT_MERGE_LOGGING = True. You may then attach logpilot_console, logpilot_file, or logpilot_alerts to any logger.

Alternative: do not reference logpilot_* in LOGGING. Use propagate: True so records reach the root logger, where merge attaches Logpilot’s handlers after startup.

propagate: False and alerts

AlertHandler is normally attached to the root logger (and to django / django.request). A logger with propagate: False and handlers such as console and logpilot_file only does not send records to the root, so alert delivery never runs for that subtree even when the file shows ERROR lines.

Either add logpilot_alerts to that logger’s handlers (with LOGGING_CONFIG as above), or set propagate: True and rely on the root.

Example: verbose console for a module, plus Logpilot file and alerts, without propagating to the root console:

LOGGING_CONFIG = "django_logpilot.logging.logpilot_dict_config"

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "verbose": {
            "format": "{levelname} {asctime} {module} {message}",
            "style": "{",
        },
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "formatter": "verbose",
        },
    },
    "root": {
        "handlers": ["console"],
        "level": "WARNING",
    },
    "loggers": {
        "myapp.views": {
            "handlers": ["console", "logpilot_file", "logpilot_alerts"],
            "level": "INFO",
            "propagate": False,
        },
    },
}

DJANGO_LOGPILOT_MERGE_LOGGING = True

DJANGO_LOGPILOT = {
    "format": "text",
    "level": "ERROR",
    "file_level": "INFO",
    "outputs": ["console", "file"],
    "file_path": str(BASE_DIR / "logs" / "app.log"),
    "alerts": {
        "enabled": True,
        "min_level": "ERROR",
        "deduplicate_for": 0,
        "channels": ["email"],
        "log_delivery_failures": True,
    },
}

• level controls Logpilot’s root and console-related wiring; a value like ERROR limits noise on the console.
• file_level, when set, applies only to the rotating file handler (for example INFO or DEBUG while level stays ERROR).
• For propagate: False, include logpilot_alerts on that logger if you want email or Slack for those records.

Email (including MailHog)

EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
EMAIL_HOST = "localhost"
EMAIL_PORT = 1025
EMAIL_USE_TLS = False
DEFAULT_FROM_EMAIL = "logpilot@localhost"
ADMINS = [("You", "you@example.com")]

For local experiments without SMTP, django.core.mail.backends.console.EmailBackend prints messages in the runserver terminal.

Alerts configuration

Set alerts["enabled"] to Boolean True. String values such as "true" / "false" from environment-driven settings are normalized.

For channels: ["email"], define ADMINS and/or alerts["email_to"]. With no recipients, the email channel is inactive.

"alerts": {
    "enabled": True,
    "min_level": "ERROR",
    "deduplicate_for": 300,
    "channels": ["slack"],
    "slack_webhook": "https://hooks.slack.com/services/.../.../...",
},

An empty channels list means: use every destination that is actually configured (webhook and/or email).

---

What gets logged

• By default, Logpilot attaches to the root logger and to django / django.request. Most application loggers propagate to the root; severity is gated by DJANGO_LOGPILOT["level"] and by each handler (see file_level for the file handler only).
• Loggers you declare in LOGGING receive only the handlers you attach. With propagate: False, attach logpilot_file and logpilot_alerts yourself when needed, and use LOGGING_CONFIG if those names appear in the dict.
• Alerts observe alerts["min_level"] (default ERROR).

---

Settings (DJANGO_LOGPILOT)

Key	Default	Description
format	"text"	text or json (one JSON object per line).
level	"INFO"	Level for root, Logpilot console handler, and django / django.request loggers.
file_level	None	If set, only the file handler uses this level (e.g. verbose file with quiet level).
outputs	["console"]	console, file, or both.
file_path	None	Required when file is in outputs; parent dirs are created if missing.
file_max_bytes	10485760	Rotate after this many bytes (10 MiB default).
file_backup_count	5	Number of rotated backups.
request_id_header	"X-Request-ID"	Incoming header for correlation id.
request_id_response_header	"X-Request-ID"	Outgoing header.
redact_keys	built-in list	Substrings matched against dict keys in redact_mapping().
alerts	see table below	Slack and email alerting.

Default redact_keys include: password, secret, token, authorization, cookie, set-cookie, api_key, apikey, credit_card.

alerts

Key	Default	Description
enabled	False	Turn alert delivery on or off (Boolean; strings normalized).
min_level	"ERROR"	Lowest level sent to alert channels.
deduplicate_for	300	Seconds to suppress duplicate fingerprints; 0 disables.
channels	[]	slack, email, both, or [] to auto-select configured destinations.
slack_webhook	None	Slack incoming webhook URL.
slack_timeout	10	HTTP timeout in seconds (capped at 60).
email_backend	None	Optional Django email backend import path.
email_to	None	Addresses or (name, email) tuples; defaults to ADMINS.
email_subject_prefix	"[Logpilot] "	Email subject prefix.
log_delivery_failures	False	If true, print delivery exceptions and tracebacks to stderr.

---

Filters

If you add "filters": [...] to a handler, every name must exist under LOGGING["filters"]. Logpilot’s built-in formatters do not require a filter for request IDs; they read context set by middleware.

---

Redacting structured data

from django_logpilot.redaction import redact_mapping

logger.info("signup payload: %s", redact_mapping({"email": "a@b.com", "password": "x"}))

---

Troubleshooting

Symptom	What to check
No Logpilot output	Non-empty LOGGING["handlers"] requires DJANGO_LOGPILOT_MERGE_LOGGING = True, or use LOGGING = {}.
Unable to add handler 'logpilot_file' (or logpilot_alerts)	Set LOGGING_CONFIG = "django_logpilot.logging.logpilot_dict_config" with merge on, or drop logpilot_* from LOGGING and propagate to root.
File shows ERROR but no email	Logger may use propagate: False without logpilot_alerts; add it or enable propagation.
Empty app.log	level applies to Logpilot handlers; with ERROR, INFO/DEBUG may be dropped. Set file_level (e.g. INFO) or lower level.
KeyError: 'logpilot_request_id'	Remove obsolete filters: ['logpilot_request_id'] from copied configs, or define that filter in LOGGING.
Alerts never fire	alerts["enabled"] must be true; set recipients (ADMINS / email_to); log at or above min_level; try deduplicate_for: 0 while testing; run python manage.py check.
SMTP errors	Set log_delivery_failures: True and inspect stderr; verify send_mail in manage.py shell.

python manage.py check

• django_logpilot.W001 — Alerts enabled but AlertHandler is not on the root logger (missing app in INSTALLED_APPS, merge disabled, or LOGGING_CONFIG needed when using logpilot_* on loggers).
• django_logpilot.W002 — Alerts enabled but no active channel (for example channels: ["email"] with empty ADMINS and no email_to).

---

Releases

See CHANGELOG.md for version history. The public API is still pre-1.0; pin a compatible range in dependency files for production.

---

Developing this package

git clone <repository-url>
cd django-logpilot
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[dev]"
pytest

---

License

MIT. Full text in LICENSE.