OpenTelemetry Instrumentation Python (v1)

Python-first OpenTelemetry bootstrap for backend services, focused on:

• deterministic startup and shutdown
• explicit feature toggles for tracing, metrics and instrumentations
• strict privacy defaults for database statements and user identifiers
• zero-code startup through the official distro/configurator mechanism
• log correlation only, with no log export pipeline inside this package

Installation

pip install "elven-opentelemetry-instrumentation-py[full]"

Runtime support

v1 supports runtime="python" only.

Passing runtime="serverless" or runtime="edge" raises an explicit error.

Zero-code

export OTEL_SERVICE_NAME="my-service"
export OTEL_SERVICE_VERSION="1.0.0"
elven-otel-instrument python app.py

Disable wrapper initialization explicitly:

export OTEL_ZERO_CODE=false

Kubernetes Operator

Build the Python auto-instrumentation image expected by the OpenTelemetry Operator:

docker build -f Dockerfile.operator -t your-registry/elven-opentelemetry-instrumentation-py:0.1.3 .

Use that image in the Instrumentation resource:

apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: instrumentation
spec:
  exporter:
    endpoint: "http://opentelemetrycollector.monitoring.svc.cluster.local:4318"
  propagators:
    - tracecontext
    - baggage
    - b3
    - b3multi
  sampler:
    type: parentbased_traceidratio
    argument: "1"
  python:
    image: docker.io/elvenobservability/python-k8s-operator:latest
    env:
      - name: OTEL_PYTHON_DISTRO
        value: "elven"
      - name: OTEL_PYTHON_CONFIGURATOR
        value: "elven"
      - name: OTEL_PYTHON_LOG_AUTO_INSTRUMENTATION
        value: "false"
      - name: OTEL_LOGS_EXPORTER
        value: "none"
      - name: OTEL_EXPORTER_OTLP_PROTOCOL
        value: "http/protobuf"
      - name: OTEL_METRICS_EXPORTER
        value: "otlp"
      - name: OTEL_TRACES_EXPORTER
        value: "otlp"
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: "http://opentelemetrycollector.monitoring.svc.cluster.local:4318"

This image ships both /autoinstrumentation and /autoinstrumentation-musl, sets OTEL_PYTHON_DISTRO=elven, sets OTEL_PYTHON_CONFIGURATOR=elven, and keeps logs export disabled by default to match the package contract.

If you prefer split endpoints, keep the same http/protobuf protocol and point both variables to the collector HTTP port:

- name: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
  value: "http://opentelemetrycollector.monitoring.svc.cluster.local:4318/v1/traces"
- name: OTEL_EXPORTER_OTLP_METRICS_ENDPOINT
  value: "http://opentelemetrycollector.monitoring.svc.cluster.local:4318/v1/metrics"

For Alpine-based workloads, add:

metadata:
  annotations:
    instrumentation.opentelemetry.io/otel-python-platform: "musl"

Manual initialization

from opentelemetry.sdk.trace.sampling import Decision

from elven_opentelemetry_instrumentation import (
    MetricsConfig,
    ObservabilityConfig,
    PrivacyConfig,
    SamplingRule,
    ServiceConfig,
    TracingConfig,
    init_observability,
)

handle = init_observability(
    ObservabilityConfig(
        service=ServiceConfig(
            service_name="payment-service",
            service_version="1.0.0",
            deployment_environment="production",
        ),
        tracing=TracingConfig(
            enabled=True,
            rules=[
                SamplingRule(
                    attribute_key="http.route",
                    attribute_value="/healthz",
                    decision=Decision.DROP,
                ),
            ],
        ),
        metrics=MetricsConfig(enabled=True, export_interval_millis=60_000),
        privacy=PrivacyConfig(
            redact_db_statement=True,
            hash_user_id=True,
        ),
    )
)

handle.tracer.with_span("startup", lambda span: span.set_attribute("boot.ready", True))
handle.metrics.increment("jobs.started", 1)
handle.force_flush()
handle.shutdown()

Sampling

Standard samplers from the environment

OTEL_TRACES_SAMPLER and OTEL_TRACES_SAMPLER_ARG are honoured. Supported values:

OTEL_TRACES_SAMPLER	OTEL_TRACES_SAMPLER_ARG
always_on	ignored
always_off	ignored
traceidratio	ratio in [0.0, 1.0]
parentbased_always_on	ignored
parentbased_always_off	ignored
parentbased_traceidratio	ratio in [0.0, 1.0]

An unknown sampler name is ignored with a warning and the default sampler is used. A missing, unparseable or out-of-range ratio falls back to 1.0 with a warning.

Precedence

1. TracingConfig(sampler=...) — an explicit Sampler object
2. TracingConfig(ratio=...) or TracingConfig(rules=[...]) — the built-in SmartSampler
3. OTEL_TRACES_SAMPLER / OTEL_TRACES_SAMPLER_ARG
4. Default: ParentBased(root=SmartSampler(ratio=1.0))

TracingConfig.ratio defaults to None, not 1.0, so "not configured" stays distinguishable from "configured to 1.0". That is what allows the environment variables to take effect. Passing ratio=1.0 explicitly still pins the ratio to 1.0.

Injecting your own sampler

Pass the sampler through the public config before initialization:

from opentelemetry.sdk.trace.sampling import ParentBasedTraceIdRatio

init_observability(
    ObservabilityConfig(
        service=ServiceConfig(service_name="reports", service_version="1.0.0"),
        tracing=TracingConfig(enabled=True, sampler=ParentBasedTraceIdRatio(0.1)),
    )
)

Do not assign to provider.sampler, provider._sampler or provider._delegate after startup. TracerProvider.get_tracer() hands its sampler to each Tracer at construction time, so tracers that already exist keep the old sampler and the swap silently does nothing for them. Initialization here is idempotent: calling init_observability() again returns the existing handle without reconfiguring anything. To change the sampler at runtime, call shutdown() and then init_observability() again, or configure it before the zero-code bootstrap starts.

Bounding the number of spans in one trace

ELVEN_OTEL_MAX_SPANS_PER_TRACE (or TracingConfig(max_spans_per_trace=...)) caps how many recording spans a single trace may start in this process. It is disabled by default — traces stay unbounded unless you opt in.

ELVEN_OTEL_MAX_SPANS_PER_TRACE=500

• The guardrail wraps the effective sampler at the top, so it runs for every span, including children of an already-sampled parent. A limiter installed as ParentBased(root=...) would not work: ParentBased.should_sample() routes children of a sampled parent to its parent-sampled delegate and never calls the root sampler.
• The delegated decision is evaluated first and never upgraded. Only spans the delegate would record consume budget, and a trace that was not sampled stays unsampled.
• The root span counts against the limit: with 500, a root plus 499 children are recorded and every later span in that trace is dropped.
• Dropped spans keep the parent's tracestate, so W3C trace state still propagates.
• A non-positive or unparseable value disables the guardrail.
• The process-local registry is memory-bounded. If it is saturated entirely by active traces, new reservations fail closed instead of evicting an active trace and allowing that trace to grow without protection.

The limit is per process, not distributed. Each Gunicorn/Uvicorn worker, and each service in the request path, keeps its own counter. It is a safety net against a single oversized trace, not a global budget.

It is not an architectural fix. A job that processes many chunks under one HTTP request should start an independent root trace per chunk — a fresh Context() without the current parent plus a Link back to the original HTTP context — or move the work to an async job and return 202. Adding one child span per chunk keeps everything in the same trace. Tail sampling in the collector reduces how many traces are stored, but it does not make one sampled, oversized trace safe.

Public API

init_observability(config) returns an ObservabilityHandle with:

• tracer
• metrics
• shutdown()
• force_flush()
• is_started()

The initializer is idempotent and protects against duplicate startup.

Feature precedence

For booleans, precedence is:

1. explicit config
2. environment variable
3. package default

Examples:

• OTEL_TRACES_EXPORTER=none disables traces by default
• TracingConfig(enabled=True) overrides that default
• OTEL_INSTR_PSYCOPG=false disables psycopg instrumentation unless explicitly enabled in config

The same order applies to sampling (TracingConfig.sampler / ratio / rules before OTEL_TRACES_SAMPLER) and to the span guardrail (TracingConfig.max_spans_per_trace before ELVEN_OTEL_MAX_SPANS_PER_TRACE, disabled when neither is set).

Supported coverage

• Web: ASGI, WSGI, FastAPI, Flask, Django
• GraphQL: request-level enrichment plus operation and field spans through graphql-core
• HTTP clients: requests, httpx, urllib3, aiohttp-client
• Data: SQLAlchemy, psycopg, psycopg2, pymysql, mysqlclient, Redis, MongoDB
• Workers and queues: Celery, threading, asyncio, gRPC, Kafka, RabbitMQ/Pika, SQS via boto3
• Logging: stdlib logging correlation

Instrumentation is activated only when the target library is installed and enabled.

Development

python -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
ruff check .
pyright
pytest