agentops-otel

Shared logging + OpenTelemetry bootstrap for AgentOps.

Logging (configure_logging, get_logger)

One entry point sets up the process's logging; every module then grabs a logger and lets records propagate to the single root handler:

from agentops_otel import configure_logging, get_logger

configure_logging()          # once, at process start (idempotent)
log = get_logger(__name__)   # in every module
log.info("worker started", extra={"worker_id": wid})

configure_logging() installs one stdout handler on the root logger and picks its shape from the environment:

• Local (AGENTOPS_ENV / AGENTOPS_DD_ENV / ENV unset or local) — a readable, coloured console line: time LEVEL name file:line message key=val.
• Anything else (staging, production, …) — one-line JSON per event with explicit level/status and Datadog unified-service dd.* tags, so a Datadog agent tailing container stdout reads the real severity and correlates to traces.

Our staging/prod control plane sets AGENTOPS_DD_ENV, and worker pods set OTEL_EXPORTER_OTLP_ENDPOINT — either signal flips a deployed process to JSON automatically, with no extra config. Overrides: json_logs=True|False forces a format; level comes from the arg, then AGENTOPS_LOG_LEVEL, then AGENTOPS_DEBUG, then INFO.

Trace correlation is generic OpenTelemetry. We read the current span through the OTel API only. When no tracer/exporter is configured (the default — configure_telemetry is opt-in), there is no active span and nothing is added, so logging needs no running collector. When a span is active, each JSON record gains otelTraceID/otelSpanID plus the dd.trace_id/dd.span_id (lower-64-bit) form Datadog uses.

Forwarding worker logs to the control plane (CPForwardingHandler)

agentops_otel.CPForwardingHandler bridges the standard logging module to a sink callback — the SDK worker runtime (komodor_agentops.worker.logging_bridge.install_cp_log_forwarding) uses it to also ship every worker log record to the control plane as a log.created event, since Komodor has no pod/log access to customer-hosted workers. Two env vars control it:

• AGENTOPS_CP_LOG_LEVEL (default INFO) — minimum level forwarded to CP.
• AGENTOPS_CP_LOG_CAPTURE_ALL (default off) — forward only the SDK's own loggers (komodor_agentops, agentops) when unset; set to attach to the root logger instead and capture the whole process.

Telemetry (configure_telemetry)

configure_telemetry(service_name, *, app=None, engine=None) builds OTLP tracer/meter/logger providers (HTTP/protobuf or gRPC), attaches the OTel logging handler, and runs the FastAPI / httpx / logging auto-instrumentors. SQLAlchemy instrumentation is the optional agentops-otel[sqlalchemy] extra.

Off by default: nothing is installed unless OTEL_EXPORTER_OTLP_ENDPOINT or AGENTOPS_OTEL_ENABLED is set. Used by the control plane and by the SDK worker runtime; neither duplicates the setup.

Metric naming (metric_name, METRIC_PREFIX)

Every AgentOps metric lives under a single agentops. namespace root so the whole product's metrics are isolated and trivially sliceable in Datadog. When you create an instrument, name it through metric_name() rather than passing a raw string:

from agentops_otel import metric_name
from opentelemetry import metrics

meter = metrics.get_meter("agentops.controlplane.authz")
hist = meter.create_histogram(metric_name("authz.pdp.duration_ms"), unit="ms")
# → exported as "agentops.authz.pdp.duration_ms"

metric_name() is idempotent — a name that already starts with METRIC_PREFIX ("agentops.") is returned unchanged, so passing either the bare suffix or the full name is safe.

Why a helper and not a global rewrite? OpenTelemetry Python has no View/exporter-level facility for dynamic per-instrument renaming (a wildcard View's name is a single static string), so the exported metric name is fixed at instrument creation. This helper is therefore the one canonical entry point. Note that the meter/instrumentation-scope name is separate from the metric name and does not affect it — the agentops. prefix must be on the instrument name.