Skip to main content

elven-logs-interceptor-python

High-performance, production-ready log interceptor for Python with Loki transport, batching, compression, circuit breaker, DLQ, and framework integrations.

Installation

pip install elven-logs-interceptor-python

With all extras:

pip install "elven-logs-interceptor-python[all]"

OpenTelemetry Security

Version 0.1.18 raises the optional otel and all extras to opentelemetry-api>=1.45.1,<2.0.0, excluding the vulnerable parser described in CVE-2026-114603. The base installation still does not require OpenTelemetry. Logging APIs and framework minimum versions are unchanged. Existing application lockfiles must be regenerated to receive the security update.

Resource Guardrails

Version 0.1.19 fixes quadratic matching in built-in message redaction without disabling sanitization. Custom user-supplied regular expressions must still be reviewed for their own runtime cost.

LOGS_BUFFER_MAX_MEMORY_MB (default 50 MiB) now budgets estimated retained entry memory across the built-in buffer, queued batches, and in-flight sends. Structured dictionaries and sequences are included; cyclic references are handled and entries exceeding 10,000 inspected values are rejected. This is not a cap on process RSS: Python allocation, serialization, transport buffers, custom object internals, and an optional DLQ have separate overhead.

Pending batches are limited to LOGS_MAX_CONCURRENT_FLUSHES, in addition to that many active sends. Immediate flush timers are coalesced and automatic flushes do not create threads waiting for network completion. When admission or queue limits are exceeded, logs are dropped and counted in get_metrics()["dropped_by_backpressure"]. This intentionally favors bounded resource use over lossless delivery during a slow or unavailable export path. Explicit flush() still waits for admitted sends and propagates its send error.

Quick Start

from logs_interceptor import init, logger

init(
    {
        "appName": "billing-service",
        "interceptConsole": True,
        "transport": {
            "url": "https://loki.example.com/loki/api/v1/push",
            "tenantId": "tenant-a",
            "authToken": "token",
            "compression": "gzip",
        },
    }
)

logger.info("service started", {"port": 3000})

Logging compatibility

The logger accepts both the Elven structured style and common Python logging call patterns. This makes migration safer when legacy code replaces logging.getLogger(__name__) with the Elven proxy.

logger.info("payload created", {"request_id": "req-1"})
logger.info("payload created: %s", payload)
logger.info("payload created: %(id)s", {"id": payload_id})
logger.info("payload created: {}", payload_id)
logger.info("payload created: %s", payload_id, extra={"request_id": "req-1"})
logger.exception("failed to create payload: %s", payload_id)

When intercepting the standard Python logging module, extra={...} is captured under context.extra:

logging.getLogger(__name__).info(
    "payload created",
    extra={"payload": payload},
)

Supported aliases: warning(...), exception(...) and critical(...).

Sensitive fields such as cpf, cnpj, cpfCnpj, documento, numeroDocumento, taxId, apiKey, clientSecret, senha, password, token, authorization, cartao/cartão, chavePix and related credential fields are redacted by default. In messages, the library redacts only the sensitive fragment/value and preserves the rest of the log. In structured context, sensitive keys redact the full value. The key matcher normalizes common snake_case, camelCase, kebab-case, dotted variants, and accents. Set LOGS_FILTER_SANITIZE=false, LOGS_SANITIZE=false, or UO_LOGS_SANITIZE=false only when raw sensitive data is explicitly required and approved.

Environment Variables

The package supports the Python LOGS_* configuration surface aligned with the JS v3 design where applicable.

Required for direct Loki mode:

  • LOGS_URL
  • LOGS_TENANT
  • LOGS_APP_NAME

For collector-first mode, set LOGS_EXPORTER=otlp (or collector) and use OTEL_EXPORTER_OTLP_LOGS_ENDPOINT. If the explicit logs endpoint is missing, the library derives /v1/logs from OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, OTEL_EXPORTER_OTLP_METRICS_ENDPOINT, or OTEL_EXPORTER_OTLP_ENDPOINT. In this mode LOGS_URL, LOKI_URL, LOGS_TENANT and LOGS_TOKEN are not required by the library. LOGS_URL is intentionally ignored in collector mode to avoid accidentally posting OTLP logs to a direct Loki /loki/api/v1/push endpoint. Put tenant or auth headers in OTEL_EXPORTER_OTLP_HEADERS/LOGS_OTLP_HEADERS if your collector gateway requires them.

Core:

  • LOGS_TOKEN
  • LOGS_APP_VERSION
  • LOGS_ENVIRONMENT

Transport:

  • LOGS_EXPORTER (loki|otlp, aliases: collector, otel)
  • OTEL_EXPORTER_OTLP_LOGS_ENDPOINT (collector logs endpoint, e.g. http://collector:4318/v1/logs)
  • LOGS_OTLP_HEADERS (comma-separated headers, e.g. x-tenant=team-a)
  • LOGS_COMPRESSION (none|gzip|brotli|snappy)
  • LOGS_COMPRESSION_LEVEL
  • LOGS_COMPRESSION_THRESHOLD
  • LOGS_USE_WORKERS
  • LOGS_MAX_WORKERS
  • LOGS_WORKER_TIMEOUT
  • LOGS_CONNECTION_POOLING
  • LOGS_MAX_SOCKETS
  • LOGS_ENABLE_STRUCTURED_METADATA (false by default; enable only when Loki allows limits_config.allow_structured_metadata)
  • LOGS_TIMEOUT
  • LOGS_MAX_RETRIES
  • LOGS_RETRY_DELAY

Buffer:

  • LOGS_BUFFER_MAX_SIZE
  • LOGS_BUFFER_FLUSH_INTERVAL
  • LOGS_BUFFER_MAX_MEMORY_MB
  • LOGS_BUFFER_MAX_AGE
  • LOGS_BUFFER_AUTO_FLUSH

Filter:

  • LOGS_FILTER_LEVELS
  • LOGS_FILTER_SAMPLING_RATE
  • LOGS_FILTER_SANITIZE (aliases: LOGS_SANITIZE, UO_LOGS_SANITIZE)
  • LOGS_FILTER_MAX_MESSAGE_LENGTH
  • LOGS_FILTER_EXCLUDE_LOGGER_PREFIXES (comma-separated)

Circuit Breaker:

  • LOGS_CIRCUIT_BREAKER_ENABLED
  • LOGS_CIRCUIT_BREAKER_FAILURE_THRESHOLD
  • LOGS_CIRCUIT_BREAKER_RESET_TIMEOUT
  • LOGS_CIRCUIT_BREAKER_HALF_OPEN_REQUESTS

DLQ:

  • LOGS_DLQ_ENABLED
  • LOGS_DLQ_TYPE (memory|file)
  • LOGS_DLQ_MAX_SIZE
  • LOGS_DLQ_MAX_RETRIES
  • LOGS_DLQ_BASE_PATH

Runtime:

  • LOGS_MAX_CONCURRENT_FLUSHES
  • LOGS_INTERCEPT_CONSOLE
  • LOGS_PRESERVE_ORIGINAL_CONSOLE
  • LOGS_ENABLE_METRICS
  • LOGS_ENABLE_HEALTH_CHECK
  • LOGS_DEBUG
  • LOGS_SILENT_ERRORS
  • LOGS_ENABLED
  • LOGS_AUTO_INIT
  • LOGS_ENABLE_EXPERIMENTAL_PROTOBUF (optional, enables experimental snappy/protobuf transport path)

Labels:

  • Prefix LOGS_LABEL_* (example: LOGS_LABEL_SERVICE=billing)

Public API

  • init(config)
  • get_logger()
  • is_initialized()
  • destroy()
  • adestroy()
  • logger proxy with debug/info/warn/warning/error/exception/fatal/critical/log/track_event/flush/aflush/get_metrics/get_health/destroy/adestroy/with_context/with_context_async

Python import remains:

import logs_interceptor

Integrations

  • Python logging (LoggingHandler)
  • FastAPI / Starlette (FastAPIMiddleware)
  • Django (DjangoMiddleware)
  • Flask (FlaskExtension)
  • Celery (CelerySignals)
  • structlog (StructlogProcessor)
  • loguru (LoguruSink)

Auto Init

Set LOGS_AUTO_INIT=true and import the package.

Preload mode:

python -m logs_interceptor.preload

Development

pip install -e ".[dev]"
ruff check .
mypy src
pytest

Deploy / Publish

Local publish script:

./scripts/publish.sh --repository testpypi --dry-run
./scripts/publish.sh --repository testpypi
./scripts/publish.sh --repository pypi

Token environment variables used by default:

  • TEST_PYPI_API_TOKEN for --repository testpypi
  • PYPI_API_TOKEN for --repository pypi

Optional script flags:

  • --skip-checks (skip lint/type/test)
  • --skip-build (skip build/twine check)
  • --token-env CUSTOM_VAR (custom token env var name)
  • --no-skip-existing (upload fails if version already exists)

Makefile shortcuts:

make qa
make publish-dry-run
make publish-testpypi
make publish-pypi

GitHub Actions publish workflow:

  • File: .github/workflows/publish.yml
  • Trigger: manual (workflow_dispatch)
  • Inputs: repository = testpypi | pypi
  • Required secrets:
    • TEST_PYPI_API_TOKEN
    • PYPI_API_TOKEN

License

MIT

Metadata

Release files for elven-logs-interceptor-python 0.1.19

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for elven-logs-interceptor-python 0.1.19
File Size Uploaded Provenance Actions
elven_logs_interceptor_python-0.1.19.tar.gz 45.0 kB Without provenance Details for elven_logs_interceptor_python-0.1.19.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for elven-logs-interceptor-python 0.1.19
File Interpreter ABI Platform Actions
elven_logs_interceptor_python-0.1.19-py3-none-any.whl Python 3 none any Details for elven_logs_interceptor_python-0.1.19-py3-none-any.whl

Total release size: 114.6 kB

Release files / elven_logs_interceptor_python-0.1.19.tar.gz

Download URL elven_logs_interceptor_python-0.1.19.tar.gz
Size 45.0 kB
Tags Source
SHA-256 checksum
How to use checksums
6c4b2c0c2c0accecb1403e6df056e3c792a01139acb8393bf4c2f55044cd8c3a
BLAKE2b-256 checksum
How to use checksums
4fe52ed06ee008af0bfaf2c83c42a3bb79947e492ef86f70f2a3ec010c30147d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.12

Release files / elven_logs_interceptor_python-0.1.19-py3-none-any.whl

Download URL elven_logs_interceptor_python-0.1.19-py3-none-any.whl
Size 69.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8f03bb2f1e38a5a567c0db78423d8ada2d2b1431329cf912c607c7e9c7c107ef
BLAKE2b-256 checksum
How to use checksums
e103cfc177a52bdacab2e6e51425d28c78bc04dcc0793cf215ea3b9edece1aac
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.12

Security for elven-logs-interceptor-python 0.1.19

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page