json-logify 0.2.3

Universal structured logging with exact JSON schema for Python frameworks

json-logify

Universal structured logging with exact JSON schema for Python frameworks.

Public repository: https://gitlab.com/mdigital-public/backend/json-logify

Features

• Exact JSON Schema: Consistent log format across all frameworks
• High Performance: Built with structlog and orjson for maximum speed
• Universal: Works with Django, FastAPI, Flask and standalone Python
• Security First: Automatic masking of sensitive data (passwords, tokens, etc.)
• OpenTelemetry Correlation: Top-level trace_id and span_id from the active OTel span for Grafana Tempo/Loki
• Easy Setup: One-line configuration for most use cases
• Rich Context: Request IDs, user tracking, and custom payload support
• Smart Filtering: Configurable path ignoring and request/response body logging
• Modern Python: Full type hints and async support

Quick Start

Installation

# Core only: structlog + orjson, no Django/FastAPI/OpenTelemetry dependencies
pip install json-logify

# OpenTelemetry bootstrap and Django/requests/httpx/gRPC instrumentations
pip install "json-logify[otel]"

Django integration code is included in the package. Keep Django pinned in the application itself. FastAPI support is planned, but no FastAPI extra is published until that integration is implemented.

Version Pinning

0.2.1 is the observability release. It adds the new canonical schema fields and reusable OpenTelemetry bootstrap, so it is intentionally published as 0.2.x, not 0.1.8.

Use the otel extra for Tempo/Loki setups:

json-logify[otel]>=0.2.1,<0.3

Existing projects that are not ready to migrate should pin the old line:

json-logify<0.2

After migrating to the new schema, pin the 0.2 line:

json-logify>=0.2,<0.3

Projects using Poetry-style ^0.1.7 or PEP 440 ~=0.1.7 will not move to 0.2.x automatically. Projects using loose constraints such as json-logify>=0.1.0 can still upgrade during dependency resolution; add an upper bound if you need controlled rollout.

Basic Usage

import logging

from logify import info, error, debug, warning, get_logger
from logify.core import configure_logging

configure_logging(service_name="service-market-core-backend")

# Basic logging with message
info("User logged in")

# With structured context
info("Payment processed", amount=100.0, currency="USD", user_id="user123")

# Named logger (shows module path in logs)
logger = get_logger(__name__)
logger.info("Processing started", task="data_import")

# Standard library logging uses the same JSON schema
logging.getLogger("apps.order.service").info("Order created", extra={"order_id": "ord-123"})

# Different log levels
debug("Debug information", query_time=0.023)
warning("Slow database query detected", query_time=1.52, query_id="a1b2c3")
error("Payment failed", error_code="CARD_DECLINED", user_id="user123")

# Exception handling
try:
    result = some_function()
except Exception as e:
    error("Operation failed", error=e, operation="some_function")

Django Integration

1. Install package and keep Django as an application dependency:

pip install Django
pip install json-logify

2. Configure in settings.py:

from logify.django import get_logging_config

# Add middleware to MIDDLEWARE list
MIDDLEWARE = [
    # ... other middleware
    'logify.django.LogifyMiddleware',  # ← Add this
]

# Configure logging with json-logify
LOGGING = get_logging_config(
    service_name="my-django-app",
    level="INFO",
    max_string_length=200,              # String truncation limit
    sensitive_fields=[                  # Fields to mask with "***"
        "password", "passwd", "secret", "token", "api_key",
        "access_token", "refresh_token", "session_key",
        "credit_card", "cvv", "ssn", "authorization",
        "cookie", "x-api-key", "custom_sensitive_field"
    ],
    ignore_paths=[                      # Paths to skip logging
        "/health/", "/static/", "/favicon.ico",
        "/admin/jsi18n/", "/metrics/"
    ],
    log_request_body=True,              # Default: True, disable in sensitive production paths
    log_response_body=True,             # Default: True
    body_max_bytes=10240,               # Max bytes inspected before JSON parsing/string logging
    body_string_max_length=1000,        # Max raw string chars written when body is not JSON/form
)

# Built-in Django logs are kept and emitted as JSON too:
# django.server, django.request, django.utils.autoreload, django.security

# Optional: keep Django server/startup/request logs enabled by default.
# Only reduce very noisy loggers if needed.
LOGGING['loggers'].update({
    'django.db.backends': {'level': 'WARNING'},
})

3. Use in your views:

from logify import info, error, get_logger

logger = get_logger(__name__)

def process_payment(request):
    # Log with automatic request context
    logger.info("Payment processing started",
         user_id=request.user.id,
         amount=request.POST.get('amount'))

    try:
        # Sensitive data gets automatically masked
        logger.info("User data received",
             username=request.user.username,    # ← Visible
             password=request.POST.get('password'),  # ← Masked: "***"
             email=request.user.email)               # ← Visible

        payment = process_payment_logic(request.POST)

        logger.info("Payment completed",
             payment_id=payment.id,
             status="success")

        return JsonResponse({"status": "success"})

    except Exception as e:
        error("Payment processing failed", error=e)
        return JsonResponse({"status": "error"}, status=500)

4. What you get automatically:

Request logging with OpenTelemetry trace/span IDs:

{
  "time": "2026-05-20T10:01:51.940Z",
  "level": "INFO",
  "msg": "Request started",
  "service": "my-django-app",
  "logger": "logify.django",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "payload": {
    "request_id": "req-456-def",
    "method": "POST",
    "path": "/api/payment/",
    "user_info": "User ID: 123: john_doe",
    "headers": {
      "Content-Type": "application/json",
      "Authorization": "[FILTERED]"
    },
    "request_body": {
      "username": "john_doe",
      "password": "***",
      "credit_card": "***"
    }
  }
}

Your application logs:

{
  "time": "2026-05-20T10:01:51.941Z",
  "level": "INFO",
  "msg": "Payment completed",
  "service": "my-django-app",
  "logger": "myapp.views.payments",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "payload": {
    "request_id": "req-456-def",
    "payment_id": "pay_123456",
    "status": "success"
  }
}

Log Schema

All logs are single-line JSON. The current schema is:

{
  "time": "2026-05-20T10:01:51.940Z",
  "level": "INFO",
  "msg": "order created",
  "service": "service-market-core-backend",
  "logger": "apps.order.service",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "payload": {
    "request_id": "req-123"
  },
  "error": "ValueError: bad input"
}

Field	Type	Description
time	string	ISO 8601 UTC timestamp
level	string	DEBUG, INFO, WARNING, ERROR, CRITICAL
msg	string	Human-readable log message
service	string	Service name from configure_logging() or get_logging_config()
logger	string	Logger name (module path via get_logger(__name__))
trace_id	string/null	Active OpenTelemetry trace ID, 32 lowercase hex chars
span_id	string/null	Active OpenTelemetry span ID, 16 lowercase hex chars
error	string	Error description (only for errors)
payload	object	Custom fields and context

By default, timestamp and message are also emitted as legacy aliases for time and msg. Disable them with legacy_fields=False:

configure_logging(service_name="myapp", legacy_fields=False)

OpenTelemetry Tracing

trace_id and span_id are read from the active OpenTelemetry span:

from opentelemetry.trace import get_current_span

span = get_current_span()
span_context = span.get_span_context()

If OpenTelemetry is not installed, no span is active, or the current span context is invalid, both fields are null. json-logify does not parse X-Trace-ID or traceparent headers itself; use OpenTelemetry instrumentation for Django, FastAPI, ASGI, WSGI, Celery, or your framework so propagation creates the active request span.

Example packages for a Django app with OpenTelemetry:

pip install "json-logify[otel]"

OpenTelemetry bootstrap

json-logify[otel] adds an explicit bootstrap helper for projects that do not want to copy local otel.py files:

# wsgi.py/asgi.py/manage.py
from logify.otel import configure_otel

configure_otel()

The helper configures a TracerProvider, W3C trace-context and baggage propagators, an OTLP gRPC exporter when an endpoint is configured, and optional Django/requests/httpx/gRPC instrumentation when the corresponding instrumentation package is installed. The otel extra installs Django instrumentation, but not Django itself.

Example environment:

OTEL_SERVICE_NAME=global-admin
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=otel-collector:4317
OTEL_TRACES_SAMPLER=parentbased_traceidratio
OTEL_TRACES_SAMPLER_ARG=0.1

OpenTelemetry bootstrap reads these values through decouplet, so process environment variables, Docker-style secrets from SECRETS_PATH (default /run/secrets), and .env files are supported.

If neither OTEL_EXPORTER_OTLP_TRACES_ENDPOINT nor OTEL_EXPORTER_OTLP_ENDPOINT is set, configure_otel() raises an error during startup. This fail-fast behavior prevents services from running with tracing enabled but no exporter. To skip tracing bootstrap intentionally, set OTEL_SDK_DISABLED=true.

Supported sampler values:

Value	Behavior
always_on	Sample every trace
always_off	Drop every new trace
traceidratio	Sample a ratio of traces using OTEL_TRACES_SAMPLER_ARG
parentbased_traceidratio	Keep upstream sampling decisions; apply ratio to new root traces
parentbased_always_on	Keep upstream sampling decisions; sample new root traces
parentbased_always_off	Keep upstream sampling decisions; drop new root traces

Full bootstrap reference: docs/integrations/opentelemetry.md.

Application logs and stdlib logs inside the same active span will carry the same trace context:

import logging

from logify import info
from logify.core import configure_logging

configure_logging(service_name="service-market-core-backend")

info("order created", order_id="ord-123")
logging.getLogger("apps.order.service").info("order indexed")

Do not configure trace_id or span_id as Loki labels. Keep them in the JSON log body and let Grafana/Loki query or derive links from the fields.

Grafana Tempo + Loki

Tempo Trace To Logs

In Grafana, configure the Tempo data source trace-to-logs feature to query Loki by the JSON trace_id field. With provisioning, keep filterByTraceID enabled and use a custom query that parses JSON:

apiVersion: 1
datasources:
  - name: Tempo
    uid: tempo
    type: tempo
    jsonData:
      tracesToLogsV2:
        datasourceUid: loki
        spanStartTimeShift: "-2s"
        spanEndTimeShift: "2s"
        tags:
          - key: service.name
            value: service_name
        filterByTraceID: true
        filterBySpanID: false
        customQuery: true
        query: '{$${__tags}} | json | trace_id=`$${__trace.traceId}`'

If your Loki stream label is named service instead of service_name, map the tag accordingly:

tags:
  - key: service.name
    value: service
query: '{$${__tags}} | json | trace_id=`$${__trace.traceId}`'

The important part is that ${__trace.traceId} is compared to the parsed JSON field trace_id; trace_id is not a Loki label.

Loki Derived Field

Add a Loki derived field that extracts the trace id from the JSON line and links it to Tempo:

apiVersion: 1
datasources:
  - name: Loki
    type: loki
    jsonData:
      derivedFields:
        - name: TraceID
          matcherRegex: '"trace_id":"([0-9a-f]{32})"'
          datasourceUid: tempo
          url: '$${__value.raw}'
          urlDisplayLabel: 'View trace'

For pretty-printed examples or escaped logs, this more tolerant regex also works:

"trace_id"\s*:\s*"([0-9a-f]{32})"

Migration Guide

From 0.1.x to 0.2.x:

Full guide: docs/migration/0.2.md

Security patch guide: docs/migration/0.2.2.md

• Read time and msg as the canonical timestamp and message fields.
• timestamp and message remain enabled by default. Set legacy_fields=False after consumers have migrated.
• Read service from the top level. Custom service=... values passed to a log call remain in payload; the top-level service comes from configuration.
• Stop passing manual trace_id values for Tempo correlation. Top-level trace_id and span_id now come from the active OpenTelemetry span and are null when no valid span exists.
• Django middleware still logs request_id, request/response metadata, and masked bodies, but it no longer generates trace IDs from headers.
• Django request/response body logging remains enabled by default for backward compatibility. 0.2.2+ masks JSON-like raw fallback strings more safely, but use log_request_body=False and/or log_response_body=False for stricter production privacy.
• stdlib logging is now configured by configure_logging() and Django get_logging_config() to emit the same single-line JSON schema.

Recommended rollout:

1. Pin existing production services to json-logify<0.2.
2. Upgrade one service to json-logify>=0.2,<0.3.
3. Update dashboards/parsers to use time, msg, service, trace_id, and span_id.
4. Disable legacy aliases with legacy_fields=False only after all consumers stop reading timestamp and message.

Security Features

• Automatic masking: Passwords, tokens, API keys, credit cards → "***"
• Header filtering: Authorization, Cookie, X-API-Key → "[FILTERED]"
• Recursive masking: Works in nested objects and arrays
• Raw body scrubbing: Detects key=value patterns like password=secret and JSON-like fallback strings like "password":"secret"
• Request/Response body: Optional, limited size + content-type filtering
• Path ignoring: Skip health checks, static files, etc.

Controlling Django body logging:

LOGGING = get_logging_config(
    service_name="myapp",
    log_request_body=False,
    log_response_body=False,
)

Body masking is best-effort and key-name based. For highly sensitive systems, disable body logging by default and add explicit application logs for safe business events.

Configuring sensitive fields:

from logify.core import configure_logging

# Merge with defaults (recommended)
configure_logging(
    service_name="myapp",
    sensitive_fields=["my_custom_secret"]  # Added to defaults
)

# Or replace defaults entirely
configure_logging(
    service_name="myapp",
    sensitive_fields=["only_this_field"],
    replace_sensitive_defaults=True
)

Advanced Usage

Named Loggers

from logify import get_logger

# Creates logger with name shown in "logger" field
logger = get_logger(__name__)  # e.g., "myapp.services.payment"
logger.info("Processing payment")

Context Management

from logify import bind, info, set_request_context, clear_request_context

# Bind context to a logger
logger = bind(service="auth", module="login")
logger.info("Processing login", user_id="123")

# Set request-level context (useful in middleware). Trace/span IDs still come from OpenTelemetry.
set_request_context(request_id="req-456")
info("User action")  # Includes request_id in payload
clear_request_context()

Performance Tracking

from logify import track_performance

@track_performance
def expensive_operation():
    # Your code here
    return "result"

# Automatically logs function start, completion, and duration

Requirements

• Python 3.11+
• structlog >= 23.0.0
• orjson >= 3.8.0
• decouplet >= 0.2.3
• OpenTelemetry is optional; when opentelemetry-api is importable, json-logify reads the active span context.

Examples

• examples/django-example - basic Django JSON logging setup.
• examples/django-otel-example - Django + OpenTelemetry + Grafana/Loki/Tempo Docker Compose stack.

License

MIT License - see LICENSE file for details.

Contributing

Contributions are welcome in the public GitLab repository: https://gitlab.com/mdigital-public/backend/json-logify

Please open an issue or submit a merge request there.