foam-otel

Foam's Python base package: a thin, safe wrapper around the official OpenTelemetry libraries that turns on automatic instrumentation, ships traces, metrics, and logs to foam, and hands you a small set of bulletproof helper functions. One init() at process start is the whole integration — everything else is optional. Built on OpenTelemetry (the Python API/SDK, the OTLP-over-HTTP exporters, and the official opentelemetry-instrumentation-* contrib packages); see THIRD-PARTY-NOTICES for full credits.

This README is the FDE manual — everything you need to integrate, verify, and troubleshoot lives here, not in foam's source.

Install

pip install foam-otel

pip install foam-otel installs the universal instrumentation floor by default (since 2.1.0). The default bundle pulls the official contrib packages for the web frameworks (fastapi/ASGI, django, flask), outbound HTTP (requests, urllib3, httpx), the primary databases (psycopg, asyncpg, mysql, pymysql, redis, pymongo), the celery queue (trace continuity across the queue hop), host + runtime system metrics, and the LLM floor — openai-v2 (chat/embeddings), the Anthropic instrumentor, and the google-genai instrumentor (Gemini) — alongside the always-on stdlib root-logger bridge and foam's OpenAI Responses gap-filler (sync + async). init() presence-checks every one of them (the entry-point sweep, below): each activates only when its target library is importable, never blind-registers, and stands down rather than double-instrument a library you already instrument — so raising the floor adds signals, never breaks the ones already flowing. SQL statement text and bind params, redis commands (key AND value), HTTP request/response headers, URL query params and credentials, log bodies, and LLM prompt/response content are captured RAW by default — foam flips the upstream capture knobs on and disables the upstream sanitizers at activation (see the raw-capture rows in the env table below). Redaction is fully opt-in (see below): nothing is masked unless you list its key. No adapter code ships in foam.

Instrumentations beyond the floor arrive as a plain install of the official contrib package, activated the same way at init():

pip install opentelemetry-instrumentation-botocore   # AWS SDK, activated on presence

Any other official instrumentation you install (-cassandra, -elasticsearch, -grpc, …) is activated the same way at init() — presence-checked, never blind-registered. (The pre-2.1 foam-otel[fastapi] / [httpx] / [llm] extras still resolve; they are now subsets of the default bundle.)

The scenario matrix

One copy-paste-ready snippet per supported situation (spec rule 33). If your situation is missing here, that is a docs bug — file it. All calls use the real init(...) keyword arguments. Load foam's init() before your app imports the libraries you want instrumented — instrumentation that registers after a library is imported never applies to already-bound references (GOTCHAS G1).

# 1 — CLEAN SERVICE (door 1, the common case) — run at the top of your entrypoint
import os
from foam_otel import init

env = os.environ.get("APP_ENV", "development")
init(
    name="checkout-api",                 # -> service.name
    environment=env,                     # -> deployment.environment.name (verbatim)
    enabled=env not in ("test", "ci"),   # the one export switch (idiomatic recipe)
    token=os.environ["FOAM_OTEL_TOKEN"],      # Bearer auth; required only when enabled
    version=os.environ.get("GIT_SHA"),   # -> service.version (recommended)
)

# 2 — BESIDE A PROPRIETARY AGENT (situation A): disjoint pipelines.
# Foam runs its own pipeline next to the agent's; you wire the agent's intake
# host into the loop guard so foam never traces the agent's own egress.
init(
    name="checkout-api", environment=env, enabled=True, token=FOAM_OTEL_TOKEN,
    ignored_outbound_hosts=["agent-intake.vendor.example"],
)

# 3 — TENANT RIDES FOAM (situation C): a scoped SDK (LLM eval / AI-obs) attaches
# a processor to foam's pipeline. Pass a CONSTRUCTED instance + the REQUIRED
# ignore entry for its exporter's host (or its export traffic becomes a loop).
from eval_tool_sdk import EvalToolSpanProcessor

init(
    name="checkout-api", environment=env, enabled=True, token=FOAM_OTEL_TOKEN,
    additional_span_processors=[EvalToolSpanProcessor(project="prod")],
    ignored_outbound_hosts=["ingest.eval-tool.example"],
)

# 4 — FOREIGN OTEL SDK OWNS A SIGNAL (situation B): door 2 — the ingest entries.
# ONE LINE per claimed signal, added to THEIR OTel setup. The tap is an ADDITIVE
# READER: their pipeline keeps working exactly as before, and now ALSO sends
# foam a masked, identity-stamped COPY. Full contract + the REQUIRED loop step:
# see "Door 2 — the ingest entries" below.
from foam_otel import (
    create_foam_ingest_span_processor,
    create_foam_ingest_log_record_processor,
    create_foam_ingest_metric_reader,
)

# THEIR setup module — one added line per claimed signal:
their_tracer_provider.add_span_processor(create_foam_ingest_span_processor(
    token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV))
their_logger_provider.add_log_record_processor(create_foam_ingest_log_record_processor(
    token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV))
their_meter_provider = MeterProvider(          # THEIR constructor line — THE recipe
    resource=their_resource,
    metric_readers=[their_reader, create_foam_ingest_metric_reader(
        token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV)],
)
# If any signal is free, ALSO run init() — the doors compose (scenario 5).

# 5 — PER-SIGNAL COMPOSITION (traces claimed shown; symmetric over any subset).
# A foreign SDK owns traces only; metrics + logs are free.
init(name="checkout-api", environment=env, enabled=True, token=FOAM_OTEL_TOKEN)
# -> warns ONCE: "traces is owned by <owner> — foam is dark for traces", and the
#    warning names the one-line fix. metrics + logs register to foam on the FULL
#    contract. Trace ids still correlate through the shared W3C context.
their_tracer_provider.add_span_processor(create_foam_ingest_span_processor(
    token=FOAM_OTEL_TOKEN, environment=env))   # the tap rides THEIR tracer
# -> foam now ALSO receives their spans (tier-marked foam.ingest.tier=external,
#    carrying THEIR service identity); door-1 metrics/logs carry foam's identity.
# (Python has per-signal globals, so the doors compose — unlike Java's single
#  composite global.)

# 6 — TESTS / CI: fully inert. No token needed — token validates only when enabled.
init(name="checkout-api", environment="test", enabled=False)
# No SDK loads, no providers registered, no network, no warning. Helpers no-op.

# 7 — SERVERLESS (door 1 + the flush recipe). The platform freezes the process
# the instant the handler returns; flush() before it does, or buffered telemetry
# is lost. flush() never raises and is safe even if init() never ran.
from foam_otel import init, flush

init(name="orders-fn", environment="production", enabled=True, token=FOAM_OTEL_TOKEN)

def handler(event, context):
    try:
        return run(event)
    finally:
        flush()

Scenario 8 (browser) does NOT apply to Python. It exists only in the JS core's matrix — the browser package is a separate companion, build-time gated with a browser-scoped token, and this Python core has no browser surface.

init options

init() is keyword-only. Required arguments raise at boot (rule 10) so a misconfiguration fails on your machine, never silently in production. The unhappy path, shown (every other foam function no-ops instead of raising — init() is the ONE deliberate exception):

# MISCONFIGURED BOOT: a missing/blank required argument raises at init —
# the failure is a red CI run, never a dark production service.
from foam_otel import init

init(name="", environment="production", enabled=True, token="t0k")
# -> ValueError: [foam] init: name is required and must be a non-empty string
# Same shape for a blank/non-string `environment`, a non-bool `enabled`
# (TypeError), and a missing/blank `token` while enabled=True.

Option	Type	Required	Default	What it changes on the wire / why
name	str	YES	—	service.name on every export. Blank/non-string raises ValueError at init.
environment	str	YES	—	deployment.environment.name, exported VERBATIM. Values outside {production, staging, development, test} warn (typo guard) but export unchanged. Blank/non-string raises.
enabled	bool	YES	—	The ONE export switch. False = fully inert: no SDK import, no providers, no network, no warning, token untouched. True = export in that environment. Non-bool raises TypeError. No default because "on by accident in CI" and "off by accident in prod" are both incidents.
token	str	when enabled=True	None	Authorization: Bearer <token> on every OTLP export. Validated (and stripped) ONLY when enabled=True and the kill switch is unset; blank/missing there raises ValueError. Never read from the environment BY THE PACKAGE — you wire it from the service's real secret source. Fleet convention: read it from the FOAM_OTEL_TOKEN env var.
version	str	no	None	service.version, verbatim (git SHA recommended). Omitting warns — foam never detects a deploy id at runtime.
redact_keys	Sequence[str]	no	None	Redaction is fully opt-in: with redact_keys unset/empty, NOTHING is masked — every value (attributes, headers, bodies, URL query params, db.statement, logs) is captured RAW, including secrets. Listing a key TAIL-MASKS its value (********cafe); a matched key holding a dict/list masks in full. Normalized-token match: ["apiKey"] also catches api_key, x-api-key. There is no always-on secret floor — foam auto-masks nothing.
redact_pii_keys	Sequence[str]	no	None	YOUR OWN PII field names, FULL-ERASED to [REDACTED] (never a tail — last-4 of an email is the domain). Opt-in like redact_keys: unset = nothing erased. Foam ships no PII preset and infers nothing — PII posture is your call.
additional_instrumentations	Sequence[object]	no	None	Constructed instrumentor instances registered on foam's pipeline, fault-isolated: one throwing instance is skipped with a loud [foam] warning naming it, boot proceeds.
additional_span_processors	Sequence[object]	no	None	Tenant seam (traces). Your SpanProcessor joins foam's tracer provider AFTER the redaction stage — you see the masked view, byte-identical to what foam exports. A throwing instance is DISABLED and warned; foam's export is unaffected.
additional_log_record_processors	Sequence[object]	no	None	Same seam, logs (LogRecordProcessor).
additional_metric_readers	Sequence[object]	no	None	Same seam, metrics. NOTE: a MetricReader TRIGGERS collection on its own schedule/thread — that is your code on your thread; foam's never-throw guard covers only foam-invoked calls (flush/shutdown), not your reader's collect loop.
ignored_outbound_hosts	Sequence[str]	no	None	EXTENDS the outbound loop guard (rule 24), never replaces it. Explicit hostnames whose outbound calls produce no client spans. REQUIRED for any tenant/agent exporting over HTTP from inside your process (scenarios 2/3).
diagnostics	bool	no	False	Turns on [foam] self-narration at INFO (the init-ok line with per-signal ownership + instrumentation count, skipped-instrumentation notes). Off = the same messages log at DEBUG. Telemetry data never appears here.

Unknown keyword arguments are ignored with a single [foam] warning naming them (forward/backward-compat safety) — they never raise.

Deliberately absent knobs (documented AS absent)

• endpoint — the fleet endpoint (https://otel.api.foam.ai) is pinned in code. The ONE override is the operator-level OTEL_EXPORTER_OTLP_ENDPOINT env var (warns loudly). Never an init option: a code change must not silently re-route telemetry.
• sampling / any sampler knob — none. Every span ships (ParentBased(ALWAYS_ON)); an unsampled inbound flag cannot drop foam's spans, and OTEL_TRACES_SAMPLER is inert. Cost control is server-side (rule 4).
• cadence / batch-delay knobs — not init surface. Batch cadence is delegated to the standard OTEL_BSP_SCHEDULE_DELAY / OTEL_BLRP_SCHEDULE_DELAY / OTEL_METRIC_EXPORT_INTERVAL env vars (operator tuning), read by the SDK's own processors.
• disabled_environments — gone. enabled is an explicit boolean; compute it from your environment in one line (scenario 1).
• exporter / processor injection — the only pipeline seam is the three additive additional_* options. They can never redirect, drop, or displace foam's export.

Environment variables

Variable	Posture
OTEL_SDK_DISABLED=true	HONORED — the emergency kill switch (config-only, no redeploy). SUPERSEDES enabled=True: when set, foam is fully off and warns loudly. Parsed exactly like the SDK: literal true, case-insensitive, trimmed.
OTEL_EXPORTER_OTLP_ENDPOINT (+ OTEL_EXPORTER_OTLP_{TRACES,METRICS,LOGS}_ENDPOINT)	HONORED — the one operator-level override of the pinned fleet endpoint. Per-signal variant wins over the base (OTLP-spec precedence). Active override = one loud [foam] warning per signal naming the destination.
OTEL_BSP_SCHEDULE_DELAY, OTEL_BLRP_SCHEDULE_DELAY, OTEL_METRIC_EXPORT_INTERVAL	HONORED — batch/export cadence, delegated to the SDK's own batch processors and periodic reader. Positive integers only; anything else is ignored. Never customer (init) surface.
OTEL_PROPAGATORS=none	HONORED — the propagation kill lever: injection stops AND inbound extraction stops (the service becomes its own trace root), telemetry keeps flowing. Foam implements none itself and warns loudly. Any OTHER value warns "set but ignored" — foam's propagator set is fixed (W3C tracecontext + baggage).
OTEL_PYTHON_DISABLED_INSTRUMENTATIONS	INERT — set values warn loudly "set but ignored" and disable NOTHING: every installed instrumentation and both foam gap-fillers activate regardless. Deliberate: the honored table above is CLOSED, and an env lever that silently drops whole instrumentations' telemetry is exactly the class it bans. (Versions ≤ 1.1.x HONORED this var — see the migration notes below.)
OTEL_PYTHON_TRACER_PROVIDER / OTEL_PYTHON_METER_PROVIDER / OTEL_PYTHON_LOGGER_PROVIDER	INERT — foam registers by ATTEMPT (set-once, first wins), never by pre-reading a slot, so the API's entry-point provider loaders never fire on foam's account. Proven inert by a dedicated test per var.
TRACEPARENT / TRACESTATE	HONORED (inbound) — a launcher that sets these parents the whole process to its trace (spawn is an inbound call whose carrier is the environment). No-op when unset.
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_{REQUEST,RESPONSE}, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS	WRITTEN BY FOAM (raw-capture posture, never read as config) — foam sets GenAI content capture ON and header capture to ALL (.*, sanitize-fields empty) at activation, both inside the strip window (for instrument()-time readers) and persisted while foam is active (for the ASGI/WSGI middlewares, which read them at middleware construction, and util-genai's request-time paths). The GenAI var has TWO upstream grammars: legacy readers parse true, util-genai — the reader behind google-genai AND openai-v2 at the pinned window — parses a ContentCapturingMode enum; foam writes each reader class its own spelling at its instrument() moment and persists the enum (SPAN_AND_EVENT). Undone at shutdown() (an operator value that predated init is restored). While foam is active an operator value is overridden — these are foam-owned switches, not config (the strip window would hide an operator's value at the moment it matters anyway); the FDE clamp path is redact_keys/redact_pii_keys, never these vars.
upstream URL sanitizers (redact_url — not an env var, an always-on util-http default)	DISABLED while foam is active — upstream unconditionally rewrites URL credentials and AWSAccessKeyId/Signature/sig/X-Goog-Signature query values to REDACTED on every recorded http.url, with no upstream knob. Foam replaces the sanitizers with identity at activation (restored at shutdown()), so URLs ride RAW; list the keys in redact_keys to mask them. See GOTCHAS P19.
Every other OTEL_* (OTEL_SERVICE_NAME, OTEL_RESOURCE_ATTRIBUTES, OTEL_TRACES_SAMPLER, OTEL_EXPORTER_OTLP_HEADERS/_TIMEOUT/_COMPRESSION, OTEL_SEMCONV_STABILITY_OPT_IN, OTEL_*_EXPORTER, attribute limits, …)	INERT — and inert BY CONSTRUCTION. Foam reads every honored var itself, then builds the whole pipeline inside a synchronous window that strips every OTEL_* var from os.environ (restored before init returns), so no un-consumed var reaches an exporter/provider/instrumentation constructor. Identity and auth come from init options only.

API — every public function

Every function below never throws and no-ops silently before init(), when enabled=False, or when the kill switch is active — the one exception is init() itself, which raises on developer error (missing/blank required options) so a misconfigured boot fails in CI, not dark in production. Import everything from the top-level foam_otel.

init(*, name, environment, enabled, token=None, version=None, redact_keys=None, redact_pii_keys=None, additional_instrumentations=None, additional_span_processors=None, additional_log_record_processors=None, additional_metric_readers=None, ignored_outbound_hosts=None, diagnostics=False) -> bool

Wire up foam telemetry (door 1). Validates arguments, then per signal ATTEMPTS to register foam's provider into OTel's set-once global and reads the outcome (attempt-then-read — foam never pre-reads a slot to predict, and never displaces an owner); a lost attempt warns once naming the owner and foam stays DARK for that signal only. Activates installed instrumentations (traces only), wires the loop guard, and installs the stdlib-logging root export. Idempotent: a second call warns and returns. Returns True when foam is exporting at least one signal, False when disabled / killed / every signal is foreign-owned or failed. See the options table for every parameter.

from foam_otel import init

owned = init(
    name="checkout-api",
    environment="production",
    enabled=True,
    token=os.environ["FOAM_OTEL_TOKEN"],
    version=os.environ.get("GIT_SHA"),
)
# owned is True when >= 1 signal reaches foam

flush() -> None

Force-flush every foam-owned signal without tearing anything down. Never raises; safe before init() (silent no-op). The serverless and pre-crash lifeline — call it before the process can freeze (scenario 7).

from foam_otel import flush

flush()  # batched spans/logs/metrics are on the wire when this returns

shutdown() -> None

Flush and stop export for every foam-owned signal, unregister the globals foam set (providers, the TRACEPARENT context token, the stdlib root handler), and reset to a pre-init state — so a later init() (e.g. a post-fork re-init) can run. Fail-open; safe without init; never raises. Helpers no-op afterward.

import signal, sys
from foam_otel import shutdown

def _graceful(*_):
    shutdown()
    sys.exit(0)

signal.signal(signal.SIGTERM, _graceful)

span(name, attributes=None)

A context manager: run a block inside a new ACTIVE span. On exception it records the exception, sets status ERROR, re-raises the IDENTICAL error, and ALWAYS ends the span. When foam is not active it yields None and runs the block uninstrumented — your code never behaves differently because telemetry is off. attributes, if given, are set on the span (masked).

from foam_otel import span

with span("price-cart", {"cart.items": len(cart)}) as s:
    total = price_cart(cart)   # if this raises, the error is recorded and re-raised

set_attribute(key, value) -> None

Set one attribute on the currently active span; no-op when none is active (never fabricates a span). The value rides the redaction engine, so a key you listed in redact_keys/redact_pii_keys is masked before it lands on the span; anything else is captured RAW (redaction is opt-in).

from foam_otel import set_attribute

set_attribute("order.id", order.id)
# with init(redact_keys=["api_key"]):
set_attribute("api_key", key)   # -> "********" + last 4 (listed key). Un-listed → RAW.

set_attributes(attributes) -> None

Set many attributes on the active span at once; each value listed in your redact lists is masked, the rest ride RAW. No-op without an active span.

from foam_otel import set_attributes

# with init(redact_keys=["password"]):
set_attributes({"customer.tier": "gold", "password": "sup3rs3cr3tvalue"})
# -> password: "********alue" (listed); customer.tier RAW

add_event(name, attributes=None) -> None

Add a timestamped event to the active span; listed-key attributes masked, the rest RAW. No-op without an active span.

from foam_otel import add_event

add_event("cache.miss", {"key": cache_key})

record_exception(error, attributes=None) -> None

Record a standard OTel exception event (type/message/stacktrace) on the active span and set status ERROR. Accepts any BaseException, including your own error classes. NEVER notifies a vendor tracker — the bridge direction is tracker -> foam only. No-op without an active span. Listed-key attributes masked, the rest RAW.

from foam_otel import record_exception

try:
    charge(card)
except PaymentError as err:
    record_exception(err)
    raise

get_trace_context() -> dict

The active span's ids for correlating foam traces with your OWN logs and outbound payloads: {"trace_id": <32 hex>, "span_id": <16 hex>}. Returns an EMPTY dict when no valid span is active — never fabricates ids.

from foam_otel import get_trace_context

ctx = get_trace_context()
if ctx:
    my_logger.info("charging card", extra={"trace_id": ctx["trace_id"]})

increment_counter(name, n=1, attributes=None) -> None

Add to a monotonic counter (create_counter). Instruments are lazily created and cached per name (rebound once if the provider swaps). Names pass through verbatim.

from foam_otel import increment_counter

increment_counter("orders_placed", 1, {"plan": "pro"})

record_histogram(name, value, attributes=None) -> None

Record a distribution observation (durations, sizes) via create_histogram.

import time
from foam_otel import record_histogram

record_histogram("checkout_duration_ms", (time.monotonic() - started) * 1000)

add_up_down_counter(name, n, attributes=None) -> None

Add to a counter that can decrease (in-flight counts, pool sizes), via create_up_down_counter. Pass a negative n to subtract.

from foam_otel import add_up_down_counter

add_up_down_counter("jobs_in_flight", +1)
# ... work ...
add_up_down_counter("jobs_in_flight", -1)

set_metric(name, value, attributes=None) -> None

Set a synchronous gauge — last value wins (queue depth, temperature-style readings), via create_gauge.

from foam_otel import set_metric

set_metric("queue_depth", queue.qsize())

Metrics — attribute discipline. Every distinct attribute combination is a live metric stream. increment_counter("requests", 1, {"user_id": uid}) is a memory leak and a flat dashboard (the SDK caps streams protectively). Keep attribute VALUES low-cardinality: plans, regions, status classes — never ids, raw paths, or emails.

log(severity, body, attributes=None) -> None

Emit a log record through foam's pipeline, trace-correlated to the active span. severity is one of trace|debug|info|warn|warning|error|fatal|critical (case-insensitive; anything unknown lands as INFO — never throws). The body and attributes ride the redaction engine, which is opt-in: with no redact_keys the body and attributes are captured RAW; listed keys are masked (dict/list bodies deep-redact by key, URL-query bodies redact the listed keys per key). Both halves are masked at write time, BEFORE emission — so the mask holds even when a foreign OTel SDK owns the logs global and the record rides the foreign pipeline (inert-mode emission; through 2.1.0 only the attributes were write-time masked and a listed key inside the body left the process raw on that path — corrected in 2.1.1).

from foam_otel import log

log("warn", "payment retry scheduled", {"attempt": 2, "gateway": "stripe"})

Most apps don't call log() directly — they use stdlib logging, which foam bridges automatically (see install_root_export below).

redact(value) -> str

Foam's masking on demand, for your OWN sinks (your logger, an audit trail). Same fail-closed engine foam uses internally, with no key context: a scalar string/number gets the tail mask, everything else masks in full. Empty string on hard failure — never the unredacted payload. Never raises.

from foam_otel import redact

my_logger.info(f"token used: {redact(api_token)}")
# redact("sk-live-0123456789abcdef") -> "********cdef"   (len >= 12, tail kept)
# redact("short")                    -> "********"        (len < 12, full mask)
# redact({"nested": "secret"})       -> "********"        (non-scalar, full mask)

There is no default-redacted-keys constant and no always-on floor: the package redacts nothing until you list keys in redact_keys/redact_pii_keys. Capturing secrets/PII by default is a deliberate posture — you own the privacy/legal decision and opt in per key.

get_tracer(name=None) / get_meter(name=None) / get_logger(name=None)

The raw OTel Tracer / Meter / Logger on whatever pipeline owns the process globals (foam's when foam owns the signal; a foreign SDK's in inert mode). Everything the helper set doesn't wrap — span links, explicit span kinds, observable/async instruments, batch log emission — is reachable here; this is what keeps the helper set small and CLOSED. Default instrumentation scope is foam-otel (the package identity, matching the JS core's @foam-ai/otel; versions ≤ 1.1.x used app.custom — see the migration notes below). Before init they return safe no-op objects. NOTE: a span you start_span() here is yours to end() — a never-ended span leaks; the span() helper cannot leak by construction.

from foam_otel import get_tracer
from opentelemetry.trace import SpanKind

tracer = get_tracer("worker")
with tracer.start_as_current_span("drain-batch", kind=SpanKind.CLIENT, links=links) as s:
    drain()   # explicit kind + links — not wrapped by span(), reached via the passthrough

from foam_otel import get_meter
from opentelemetry.metrics import Observation

meter = get_meter("inventory")
def observe(options):
    return [Observation(current_stock())]
meter.create_observable_gauge("stock_level", callbacks=[observe])  # async instrument

install_root_export() -> bool

Attach a plain OTel LoggingHandler to the ROOT logger so every stdlib logger that propagates (the normal case) exports to foam, trace-correlated. init() calls this automatically when foam owns the logs signal — you rarely call it yourself. Idempotent: it stands down if any OTel export handler is already on root, and is RE-callable after logging.basicConfig(force=True) wipes root handlers. Returns True when a foam (or existing OTel) handler covers root, False when foam isn't active or owning logs. Fail-open.

import logging
from foam_otel import install_root_export

logging.basicConfig(force=True)   # your app resets root handlers, wiping foam's
install_root_export()             # re-attach foam's root export

export_handler(record_filter=None, transform=None)

Build a policy-drivable OTel export handler for loggers you attach it to YOURSELF — the path for libraries that set propagate=False and never reach the root handler. record_filter(record) -> bool gates each record; transform(copy) may scrub a COPY of the record (return None to drop it, so other handlers on that logger still see the original). Returns None when foam isn't active or doesn't own logs.

import logging
from foam_otel import export_handler

def only_warnings(record):
    return record.levelno >= logging.WARNING

handler = export_handler(record_filter=only_warnings)

attach_to_non_propagating(handler, *, skip_prefixes=("foam_otel", "opentelemetry")) -> list

Attach handler to every ALREADY-REGISTERED logger with propagate=False (libraries that detach from root, so the root export never sees them). Idempotent per logger by a marker attribute (not class identity), so a --reload re-import can't double-attach. Propagating loggers are never touched. Returns the list of logger names it attached to. Re-call after importing libraries that configure their loggers late.

from foam_otel import export_handler, attach_to_non_propagating

handler = export_handler()
if handler is not None:
    attached = attach_to_non_propagating(handler)   # e.g. ["browser_use", "some.detached.logger"]

Failure modes — what happens, and where the warning appears

All foam warnings are single-line, prefixed [foam], emitted on the foam_otel stdlib logger (visible on the root handler / your logging config).

Situation	Behavior
Missing/blank token while enabled=True (kill switch unset)	init() raises ValueError at boot — fail in CI, not dark in prod.
Blank/non-string name or environment	init() raises ValueError, naming the option.
Non-bool enabled	init() raises TypeError.
enabled=False	Fully inert and SILENT. No SDK import, no providers, no network, token never touched. Helpers no-op.
OTEL_SDK_DISABLED=true	Same inert posture, but LOUD: one [foam] warning that the kill switch superseded enabled=True.
environment outside {production, staging, development, test}	Exports verbatim, plus one [foam] warning (typo guard).
version omitted	Exports without service.version, plus one [foam] warning.
A foreign OTel SDK owns a signal's global	init() registers nothing there and warns ONCE per claimed signal naming the owner (see the coexistence guide); free signals still register to foam.
Export endpoint overridden via OTEL_EXPORTER_OTLP_ENDPOINT	One [foam] warning per signal naming the destination.
A broken exporter / instrumentation at runtime	That signal is disabled with a [foam] warning; the host app never crashes (fail-open per signal). Runtime detail is visible under diagnostics=True.
init() called twice	Second call warns [foam] init called twice … and returns the current ownership; the first init stands.

Coexistence guide — when foam is not alone

init() attempts to register foam's provider into each of the three global provider slots (traces, metrics, logs) — OTel's set-once, first-wins globals — and reads the outcome back: every FREE slot becomes foam's, and every CLAIMED slot warns once naming the owner (foam never pre-reads a slot to predict, and never displaces an owner). Three situations (rule 18):

• A — a proprietary APM agent beside foam (a vendor agent running its own private pipeline, no OTel globals). Disjoint pipelines; both run. Foam never touches the agent. Your job: add the agent's intake host to ignored_outbound_hosts (scenario 2) so foam never traces the agent's own export traffic. (If a modern agent ALSO registers an OTel global, that signal becomes situation B — the classifier decides per signal.)

• B — a foreign OTel SDK owns a signal's global. Foam's init() is dark for that signal — not degraded: NOTHING for that signal reaches foam through door 1. For each claimed signal, init() emits exactly one warning:

  [foam] traces is owned by <Owner> — foam is dark for traces; helper telemetry for it rides the foreign SDK to ITS backend. To also send that pipeline to foam, add one line in THEIR setup: create_foam_ingest_span_processor. See README "Door 2".

  What it means: door 1 registered nothing for that signal, and the fix is door 2 — the shipped create_foam_ingest_* entries ("Door 2 — the ingest entries" below): one line in THEIR setup taps their pipeline additively, so their export keeps working AND foam receives a masked, tier-marked copy. Your other options stay valid: remove the foreign SDK so foam owns the signal (scenario 1), or feed foam server-side from their collector. Your helpers keep working either way, but helper telemetry for the claimed signal rides the foreign provider and carries ITS resource identity, not foam's. (The owner name is read from the provider's class, MODULE-QUALIFIED — e.g. opentelemetry.sdk.trace.TracerProvider vs a vendor's ddtrace.opentelemetry.TracerProvider — because the bare class name is generic; a registered provider doesn't announce its vendor, so "an unknown OTel SDK" is the honest fallback.)

• C — a scoped tenant rides foam's pipeline (an LLM-eval / AI-observability SDK that attaches a processor/reader). Pass its CONSTRUCTED instance via additional_span_processors / additional_log_record_processors / additional_metric_readers (scenario 3). Guarantees: strictly ADDITIVE (foam's export is byte-identical with or without it), FAULT-ISOLATED (a throwing span/log instance is DISABLED loudly, foam unaffected), and ALREADY-MASKED (the redaction stage runs first, so the tenant only ever sees the masked view). REQUIRED: the tenant exporter's host in ignored_outbound_hosts — otherwise foam traces the tenant's export calls, which the tenant re-exports (a loop in the tenant's pipeline). A metric READER is different in kind: it collects on its own schedule/thread, so its collect loop is your code, not covered by foam's guard.

Claim severity is not uniform. A claimed MeterProvider costs foam little — RED metrics derive server-side from foam-owned spans; only custom metric-helper data diverts. A claimed TracerProvider costs the spans AND everything derived from them. Prioritize freeing traces first.

Late arrival. A foreign SDK can register AFTER foam boots. The same once-per-signal warning fires on the next helper use when the cached provider identity no longer matches the global — a coexistence warning is not only a boot-time event.

Door 2 — the ingest entries

When a foreign OTel SDK owns a signal (situation B), init() cannot serve it — door 2 does: one constructed instance per claimed signal, added by you — ONE LINE — to the CUSTOMER'S OWN OTel setup. Their pipeline keeps working exactly as before, and now ALSO sends foam a copy. The package provides the tap; you install the tap (consent is the line in their config — that is also the off-switch: there is no enabled param; to stop the tap, delete the line).

# signatures — identical shape across all three entries (keyword-only)
from foam_otel import (
    create_foam_ingest_span_processor,        # -> real SpanProcessor
    create_foam_ingest_log_record_processor,  # -> real LogRecordProcessor
    create_foam_ingest_metric_reader,         # -> real PeriodicExportingMetricReader
)

tap = create_foam_ingest_span_processor(
    token=os.environ["FOAM_OTEL_TOKEN"],  # REQUIRED — Bearer auth, same as init
    environment=APP_ENV,                  # REQUIRED — -> deployment.environment.name,
                                          #   stamped AT EXPORT TIME on foam's copy
    redact_keys=["internal_ref"],         # optional — OPT-IN tail-mask (off by default)
    redact_pii_keys=["customer_email"],   # optional — YOUR PII fields, full [REDACTED]
    diagnostics=False,                    # optional — tap-scoped [foam] narration
)

Validation is loud and at construction (the same bar as init()): a blank/missing token or environment raises a [foam]-prefixed ValueError at boot — never a dark tap. The ONE exception is the operator kill switch: under OTEL_SDK_DISABLED=true each entry returns a truly inert instance of the correct type (zero threads, zero exporters, zero network — your setup file keeps one shape) with one loud warning, and the token is NOT validated — the operator's off wins and must never crash your boot. The switch is read ONCE at construction; unset it and restart to re-enable.

# unhappy paths, shown:
create_foam_ingest_span_processor(token="", environment="production")
# -> ValueError: [foam] create_foam_ingest_span_processor: token is required
#    and must be a non-empty string
#
# OTEL_SDK_DISABLED=true ->
# [foam] OTEL_SDK_DISABLED=true — this foam ingest entry is disabled by the
# operator kill switch and will send nothing. Unset the variable and restart
# to re-enable.

Per-signal recipes

Traces and logs — post-construction attach (Python's providers support it):

their_tracer_provider.add_span_processor(create_foam_ingest_span_processor(
    token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV))
their_logger_provider.add_log_record_processor(create_foam_ingest_log_record_processor(
    token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV))

Metrics — constructor placement is THE recipe. Foam's reader collects on its own schedule beside theirs (multiple readers per MeterProvider is spec-legal); their Views shape what foam receives — an external-tier caveat, never fought:

their_meter_provider = MeterProvider(     # THEIR constructor line
    resource=their_resource,
    metric_readers=[their_reader, create_foam_ingest_metric_reader(
        token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV)],
)

MeterProvider.add_metric_reader() exists at the pinned floor and works as a FALLBACK when their MeterProvider(...) line is out of reach — but it is lifecycle-incomplete upstream: the provider's force_flush() and shutdown() walk only constructor-time readers, so a fallback-attached reader never receives the provider's final flush (GOTCHAS P16). Foam mitigates with an idempotent shutdown() plus its own atexit hook, which delivers the final collection at normal interpreter exit. Serverless caveat: constructor placement is REQUIRED on serverless — the platform freezes the process, so atexit never fires and a fallback-attached reader's last window is lost. One more invariant: every create_foam_ingest_metric_reader() call returns a fresh instance and each instance goes to exactly ONE provider — the SDK itself raises in THEIR constructor if one reader is registered twice. Never share one.

Door 2 — the required loop step

The tap's exports to foam are outbound HTTP calls THEIR instrumentation will span, and their pipeline hands those spans back to the tap — a feedback loop in THEIR pipeline that foam's own rule-24 guard cannot reach. Three layers stop it:

1. Foam wraps every tap export in the SDK's suppression context — official contrib instrumentations (requests/urllib/urllib3/httpx/aiohttp) bail out.
2. REQUIRED — your step: instrumentation that ignores the suppression key (vendor SDKs, mesh/eBPF layers) still spans foam's endpoint. Add the foam export host — otel.api.foam.ai, or the active OTEL_EXPORTER_OTLP_ENDPOINT override host — to THEIR client-side exclusions. In contrib-Python that is the PER-CLIENT vars (OTEL_PYTHON_REQUESTS_EXCLUDED_URLS, OTEL_PYTHON_URLLIB_EXCLUDED_URLS, OTEL_PYTHON_URLLIB3_EXCLUDED_URLS, OTEL_PYTHON_HTTPX_EXCLUDED_URLS, OTEL_PYTHON_AIOHTTP_CLIENT_EXCLUDED_URLS) — never the global OTEL_PYTHON_EXCLUDED_URLS, which their SERVER instrumentation also honors and which would drop their inbound spans when the collector is on localhost.
3. The echo filter, foam's structural backstop: if both layers are defeated, a client-kind span describing foam's own export call is withheld from FOAM'S copy only (their pipeline keeps it), with one loud warning naming this missing step. The loop is bounded even with the README step forgotten — but the step stays REQUIRED: the filter withholds foam's echo, it cannot stop their pipeline from paying for those spans.

What the tap does — and never does

• Additive reader only: their data, resource, and export are NEVER mutated. Foam's labels merge at serialization onto foam's copy; your pipeline sees your data byte-identical. The tap masks foam's copy only — your pipeline still carries what it captured.
• Identity: foam's copy carries THEIR resource identity (service.name, service.version, service.instance.id untouched) plus exactly four stamp keys: deployment.environment.name (the argument), foam.ingest.tier: external, telemetry.distro.name: foam, telemetry.distro.version.
• Masking: redaction is opt-in — with no redact_keys/redact_pii_keys the tap masks NOTHING and forwards every value RAW. When you list keys, they are masked on foam's copy before it leaves — attributes, event/link attributes, metric datapoint attributes, and log attributes + body (per key). Fail-closed on foam's side: an item whose masked copy cannot be built is dropped from foam's batch with a loud counter, never exported raw.
• Never-throw: after construction, nothing in the tap can break your pipeline or your app — failures warn [foam] and foam's copy alone is lost.
• Self-ingest stand-down: wiring a tap onto foam's OWN door-1 pipeline is a wiring bug — the tap warns once and stands down (door 1 already exports that data; nothing is lost, nothing is doubled), and tears its own machinery down.
• Same-signal double-claim: if init() already exports a signal AND a tap sits on a separate foreign provider, both keep flowing (different pipelines; the tier marker distinguishes them) with one loud warning — if unintended, remove one.
• Transport: OTLP/HTTP protobuf to foam's fleet /v1/* endpoints with Authorization: Bearer <token> — same pins, same operator-override env var (honored + loudly warned) as door 1.
• Forbidden surface: no endpoint, cadence, sampling, temporality, or Views options — deliberately absent, same as init().
• Fork safety: taps built pre-fork self-heal in workers via the SDK's at-fork hooks (regression-pinned). Door-1's post-fork re-init does not govern the tap — door-2 identity is theirs, there is no foam instance id to re-mint.

Recipes

• Per-environment enable (the one-liner): enabled=os.environ.get("APP_ENV") not in ("test", "ci") — scenario 1. Other valid shapes: enabled=os.environ.get("APP_ENV") in ("production", "staging") (explicit allowlist), enabled=os.environ.get("APP_TELEMETRY") == "on" (your own app-named flag, compared EXPLICITLY — never bool(os.environ.get(...)), which is True for "false" and "0"), enabled=True (export everywhere, honestly labeled). Foam itself defines no on/off env flag — the only foam env key that exists is the token you wire yourself.

• Serverless flush: scenario 7 — try: … finally: flush(). The platform freezes the process before any batch timer fires; flush() is the fix. Import cost is lazy: importing foam_otel costs only the OTel API; the SDK and exporters load inside init() (and never with enabled=False).

• Uvicorn / ASGI graceful shutdown — wire shutdown() at lifespan shutdown. Uvicorn re-raises SIGTERM after its graceful stop, so the process dies BY SIGNAL and the SDK's atexit backstop never runs — telemetry buffered since the last batch timer is silently lost. The signal.signal(SIGTERM, ...) recipe above CANNOT be used under uvicorn (uvicorn owns the SIGTERM handler; replacing it disables graceful HTTP shutdown). The reliable path is the ASGI lifespan hook — proven by the conformance app under 60s batch delays:

# uvicorn/ASGI graceful shutdown: flush buffered telemetry before the process dies
from contextlib import asynccontextmanager
from fastapi import FastAPI
import foam_otel

@asynccontextmanager
async def lifespan(app):
    yield
    foam_otel.shutdown()   # flushes every buffered span/log/metric

app = FastAPI(lifespan=lifespan)

• Gunicorn / uvicorn / any pre-fork server — foam re-inits automatically. Foam registers a post-fork hook (os.register_at_fork), so under gunicorn --preload or a uvicorn worker fork each child re-inits automatically and re-mints a fresh service.instance.id — N workers report as N processes, not one. You do NOT need a manual post_fork hook or a shutdown()+init() dance. (A bare init() inside a hand-written post_fork would be a no-op — one init per process — but the automatic hook makes that unnecessary. On platforms without os.fork, the hook is skipped.) See GOTCHAS "Fork / pre-fork servers".

• gevent / eventlet workers: call monkey.patch_all() as line 1 of your entrypoint, BEFORE import foam_otel — patching after threading/ssl/socket are imported breaks context propagation and the export threads. See GOTCHAS.

• What foam deliberately does NOT automate: reading your git SHA (pass version yourself), choosing your PII fields (pass redact_pii_keys yourself), sampling (none exists), and named logger bridges beyond the generic stdlib bridge (structlog/loguru arrive via the update path — log() and the root-export bridge cover the mechanism today). LLM instrumentation IS automated for the floor providers: the default bundle carries openai-v2 (chat/embeddings), the Anthropic instrumentor, and the google-genai instrumentor (Gemini), and foam's own gap-filler covers the OpenAI Responses API (sync + async) — all with prompt/response content captured raw (see the dedicated section below). LLM libraries beyond the floor are covered by installing their official instrumentor (auto-activated) or by tier-3 spans through the helpers/passthroughs.

LLM instrumentation — content captured, raw

The LLM floor ships in the default bundle (provider parity: OpenAI + Anthropic + Gemini): opentelemetry-instrumentation-openai-v2 (chat/embeddings — foam switches its content capture ON at activation), the Anthropic instrumentor (content on by its default), and the official contrib google-genai instrumentor (Gemini — foam sets its content mode to SPAN_AND_EVENT at activation, so raw prompt/response content lands on the gen_ai span attributes AND the inference-details event; the same OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT var speaks a different grammar per reader, and foam writes each reader its own spelling at its instrument() moment), all activated by the sweep on presence.

Gemini provider value, canonicalized (2.1.1). The pinned google-genai instrumentor still derives gen_ai.provider.name as the DEPRECATED semconv values gemini / vertex_ai (semconv 0.65b0: "Replaced by gcp.gemini/gcp.vertex_ai"), while the JS, Ruby, and Java cores emit the current gcp.* well-known values — so a fleet dashboard filtering gen_ai.provider.name="gcp.gemini" silently dropped every Python service. Foam canonicalizes those two VALUES on every export path (span attributes, the inference-details event, metric datapoints, door-2 copies); the attribute NAME is untouched, no other value is rewritten, and the mapping drops out when the upstream fix lands and is re-pinned (GOTCHAS P23). Dashboards keyed on the deprecated gemini/vertex_ai values must re-key to gcp.gemini / gcp.vertex_ai (see the migration notes).

Foam additionally ships its own gap-filler for the OpenAI Responses API (what reasoning-model services call via client.responses.create(...), sync or awaited), which the official openai-v2 window does not cover. Know exactly what the gap-filler does in your process:

• What it patches: Responses.create AND AsyncResponses.create, via wrapt wrappers. Each target stands DOWN independently (no wrap) when openai is absent, or when upstream (openai-v2 or anyone) already wrapped that method — foam never double-instruments.
• What it emits: ONE CLIENT span per logical call, named chat <model>, carrying the gen_ai activity metadata — gen_ai.operation.name, gen_ai.provider.name: openai, gen_ai.request.model, gen_ai.response.model, gen_ai.response.id, gen_ai.usage.input_tokens / output_tokens — AND the content, raw (2026-07-26 ruling: LLM content is captured like any other signal, never gated or dropped), in the fleet-pinned structured shape (rule 8c GenAI-uniformity: ONE content dialect fleet-wide): gen_ai.input.messages / gen_ai.output.messages carry the semconv chat-message JSON — [{"role": ..., "parts": [{"content": ..., "type": "text"}]}], output messages with finish_reason observed from the response status — exactly what the bundled openai-v2 chat / Anthropic / Gemini paths and the other language cores emit; the content TEXT is verbatim, a bare-string input wraps as one user message, a message-list input maps role/content → role/parts (2.1.0 emitted bare strings on these keys — a second dialect, corrected in 2.1.1; see the migration notes). instructions land verbatim on gen_ai.system_instructions. Content is masked ONLY when you list a matching key in redact_keys/redact_pii_keys — same opt-in rule as everything else. On a failed call the span additionally carries the observed error.type and — when the provider error exposes one — http.response.status_code. A successful call carries NO status attribute (nothing is observed, so nothing is stated; versions ≤ 1.1.x emitted gen_ai.response.status and a fabricated http.status_code: 200 — see the migration notes below). The HTTP transport span underneath is suppressed so one call yields one span.
• Streaming: stream=True calls are captured too (since 2.1.1; 2.1.0 ended the span at connection-open and silently lost output/usage/id/model). The span stays open and foam wraps the returned Stream/AsyncStream with a transparent passthrough that only OBSERVES events as YOUR code pulls them — foam never drains, buffers, or consumes the stream. The terminal response.completed/response.incomplete/response.failed event lands the full response shape on the span, which ends when the stream is exhausted, closed, or exits its with block (abandon a stream without closing it and the span is never exported). The responses.stream() and responses.parse() helper APIs are NOT wrapped — a documented boundary of the gap-filler, not silent partial capture.
• Turning it off: there is no env lever — OTEL_PYTHON_DISABLED_INSTRUMENTATIONS is inert (env table above; versions ≤ 1.1.x honored the name openai_responses there). Restricting content for a compliance-sensitive deployment is the FDE clamp path: list the content keys in redact_keys/redact_pii_keys. Errors inside the wrapper never break your call (fail-open).
• History: 1.1.0 briefly emitted gen_ai.system (renamed to gen_ai.provider.name in 1.1.1 — dashboards keyed on gen_ai.system must re-key). 1.1.1 also removed content capture; that removal was OVERRULED by the 2026-07-26 capture ruling and content returned — additively, on the current-semconv content attributes above — in 2.1.0. The attribute-key set is frozen additively by the golden fixture — keys may be added within a major, never removed or renamed.
• Picking ecosystems: contrib, OpenLLMetry, and openinference all register opentelemetry_instrumentor entry points and sometimes wrap the SAME client under different package names; the double-instrument guard is per-instrumentor, not cross-ecosystem. Install ONE ecosystem per provider (the bundled floor already covers OpenAI + Anthropic + Gemini).

Thrown-then-mapped exceptions (FastAPI/Starlette). Also since this line: an exception your @app.exception_handler maps to a clean 4xx is handled BELOW the OTel middleware, so the official instrumentation alone would export the request span with no exception event. Foam records the exception on the server span at handler-lookup time — a throw is a genuine fault signal even when mapped (contract §6). Control-flow exceptions (HTTPException — every 404 — and request-validation errors) are never recorded, and unmapped exceptions are left to the official instrumentation (no double recording). Like the Responses gap-filler, it has no env disable lever (versions ≤ 1.1.x honored the name mapped_exceptions in the now-inert OTEL_PYTHON_DISABLED_INSTRUMENTATIONS).

Supported versions

Surface	Supported	Reason
Python	>= 3.10	init() uses importlib.metadata.entry_points(group=...), whose keyword form landed in 3.10.
opentelemetry-api / opentelemetry-sdk	>= 1.44, < 2	The logs SDK's processor contract is on_emit(ReadWriteLogRecord) at this line (the older emit(LogData) shape is a different, incompatible signature), and foam's redaction rides that hook. API and SDK move in lockstep. The < 2 ceiling is deliberate (rule 40): a future OTel 2.0 is NOT supported until the matrix re-proves it — an open floor would let a breaking major resolve into your install as a silent no-op.
opentelemetry-exporter-otlp-proto-http	>= 1.44, < 2	The OTLP-over-HTTP transport foam exports on. Same deliberate ceiling as the API/SDK.
opentelemetry-instrumentation	>= 0.65b0, < 0.66b0	The entry-point activation base foam sweeps — pre-1.0, so capped to the PROVEN minor (rule 40: 0.x lines break between minors); widening is a deliberate, re-proven bump.
wrapt	>= 1.0, < 3	Direct dependency of the gap-fillers' wrap_function_wrapper (declared, never assumed transitively).
Floor instrumentations (-fastapi, -django, -flask, -requests, -urllib3, -httpx, -psycopg, -asyncpg, -mysql, -pymysql, -redis, -pymongo, -celery, -system-metrics, -openai-v2, -anthropic, -google-genai)	>= 0.65b0, < 0.66b0 (contrib line; openai-v2 >= 2.4b0, < 2.5b0 — every earlier 2.x is dead against this core: 2.0/2.1 import the removed opentelemetry._events, 2.2/2.3 call wrapt with the renamed module= keyword; anthropic >= 0.62.1, < 0.63, google-genai >= 1.0b1, < 1.1b0)	The universal floor, BUNDLED in the default install (2.1.0): web + outbound HTTP + pg/mysql/redis/mongo + queue + system-metrics + LLM (OpenAI + Anthropic + Gemini; activity AND content, raw). Auto-activated at init(), presence-checked; FastAPI + the LLM content path are proven end-to-end by the conformance app; django + flask carry in-suite test-client wire proofs (SERVER span, http.route, raw headers) and system-metrics is wire-pinned (a system.* + process.* datapoint after init, both ownership postures) since 2.1.1. Same rule-40 tight caps as the instrumentation base.

The EXACT versions everything above was proven against (the rule-44 known-good set) are committed in constraints.txt; install with pip install -c constraints.txt foam-otel[...] to reproduce the proven environment byte-for-byte.

Foam builds on the OpenTelemetry project — the Python API/SDK, the OTLP exporters, and the contrib instrumentations — all Apache-2.0. Full credits in THIRD-PARTY-NOTICES.

MIGRATION — deliberate wire/surface corrections after 1.1.x (breaking; ships in the next MAJOR)

Three 1.1.x behaviors were corrected because they violated the wire contract (rule 26: the package never invents, renames, or fabricates a wire-visible name) or the closed env-var posture table. Each is a breaking change, shipped deliberately and gated by the golden fixture; check the following before upgrading.

1. OpenAI Responses gap-filler wire names (finding python-otelapi-4). The chat <model> gen_ai CLIENT span no longer carries gen_ai.response.status (an invented name — it exists nowhere in the gen-ai semantic conventions) or http.status_code (deprecated semconv, and on success it was a fabricated constant 200, never read from any response). Now, per the current GenAI span conventions (open-telemetry/semantic-conventions-genai, docs/gen-ai/gen-ai-spans.md): a successful call carries NO status attribute; a failed call carries error.type (the exception's qualified name, as the official openai-v2 instrumentation emits) plus the observed http.response.status_code when the provider error exposes one. Check your side: dashboards, alerts, or queries keyed on gen_ai.response.status or http.status_code on gen_ai spans will go empty — re-key error monitoring to error.type / span status ERROR (+ http.response.status_code where present); "success" is a span with status unset/OK, not http.status_code == 200.
2. OTEL_PYTHON_DISABLED_INSTRUMENTATIONS is now INERT (finding python-surface-6). 1.1.x honored it (skipping listed instrumentations, including the gap-filler names openai_responses / mapped_exceptions). The spec's honored env table is closed, and this lever silently dropped whole instrumentations' data — so it now warns loudly "set but ignored" and disables nothing. Check your side: any deployment setting this var will, after upgrade, emit telemetry from the instrumentations it used to suppress (more spans, possibly new span names/attributes on dashboards and in billing) and log one [foam] warning at boot. Remove the var, or uninstall the instrumentation package you need gone.
3. Helper telemetry scope renamed app.custom → foam-otel (finding python-surface-10). The instrumentation scope on every helper-emitted span, metric, and log is now the package identity (matching the JS core's @foam-ai/otel); app.custom was producer-classification stamp vocabulary the package must not ship (rule 4) — classification is foam's collector's job, server-side. Check your side: queries, dashboards, or collector/backend rules that filter on instrumentation scope app.custom (e.g. otel.library.name/otel.scope.name == "app.custom") must be re-pointed at foam-otel; scope-based routing in a customer-side collector should be updated the same way.

2.1.0 — the universal-floor bundle + raw-capture posture (ADDITIVE minor)

Non-breaking per the Area 0 floor-raise rule (new signals only; nothing already flowing is renamed or reshaped). What upgrading adds:

• The floor is bundled: the default install now pulls the instrumentors for django/flask, psycopg/asyncpg/mysql/pymysql, redis/pymongo, celery, system-metrics, openai-v2, anthropic, and google-genai (Gemini) (fastapi/requests/urllib3/httpx were already reachable via extras). Each activates only where its target library is present — expect NEW spans/metrics (and volume) for targets you have.
• Raw capture on by default: HTTP request/response headers (all), SQL bind params (psycopg), pymongo command bodies, raw redis statements, raw URLs (upstream credential/signature sanitizers disabled), and LLM prompt/response content (openai-v2 content switch ON; google-genai content mode SPAN_AND_EVENT; the Responses gap-filler extracts content; sync Responses.create newly covered). Everything is maskable per key via redact_keys/redact_pii_keys.
• Session stitching: spans and logs emitted under an inbound context whose W3C Baggage carries session.id (what @foam-ai/browser sends alongside traceparent) are stamped with session.id/session.previous_id attributes — no-op when absent, maskable per key like everything else. Your API's CORS allowlist must include baggage alongside traceparent for browsers to deliver it cross-origin.
• New foam-written env vars while active (removed at shutdown): the GenAI content switch + the three header-capture vars (table above).

2.1.1 — GenAI semconv alignment + capture/boot corrections (review findings 2026-07-26)

Fixes from the adversarial fleet review. Two are deliberate wire-value corrections ratified by the rule-8c GenAI-uniformity ruling (the fleet pins ONE GenAI vocabulary and ONE content dialect; Python was the sole deviator on both) — check your side:

1. Gemini gen_ai.provider.name → canonical gcp.* values. Spans, events, and metric datapoints that carried the deprecated gemini / vertex_ai now carry gcp.gemini / gcp.vertex_ai — the values the JS/Ruby/Java cores already emit. Re-key any dashboard/alert filtering the deprecated spellings.
2. OpenAI Responses gap-filler content is now the fleet-pinned structured dialect. gen_ai.input.messages / gen_ai.output.messages carry the semconv role/parts chat-message JSON (content text verbatim inside) instead of 2.1.0's bare strings/request-schema JSON — the same shape the openai-v2 chat, Anthropic, and Gemini paths emit. Parsers that assumed the bare string must parse the message list; the golden fixture now freezes the three content keys AND the structured dialect.

And the behavior fixes:

3. Streamed Responses calls captured (stream=True): output content, usage, response id/model, and true latency now land on the span via an observe-only stream passthrough — 2.1.0 ended the span at connection-open with none of them. responses.stream()/parse() remain out of the gap-filler's scope (documented boundary).
4. Bare-string redact_keys/redact_pii_keys raises at boot (rule 10). 2.1.0 swallowed the designed rejection inside the fail-open init path and booted the service fully DARK with only a warning; the typo now fails loudly at init(), exactly like door 2.
5. log() bodies are masked at write time (rule 14): with a foreign SDK owning logs (inert-mode emission), a customer-listed key inside a log() body used to leave the process RAW into the foreign backend while the same key in attributes was masked. Both halves now mask before emission.
6. Scenario 5 keeps the host-metrics floor: when a foreign SDK owns traces and foam owns metrics, the metrics-only floor item (system-metrics) now activates instead of dying with the trace sweep — and the system./process. floor is wire-pinned by tests on both postures.
7. django/flask wire proofs added (test-client SERVER-span tests with http.route + raw headers), closing the untested half of the bundled web-framework floor.

License

Proprietary — see LICENSE. Use is permitted only by Foam and customers with an active Foam agreement. Third-party components are credited in THIRD-PARTY-NOTICES (OpenTelemetry is Apache-2.0).