foam-otel

Foam's Python base package: a thin, safe wrapper around the official OpenTelemetry libraries that turns on automatic instrumentation, ships traces, metrics, and logs to foam, and hands you a small set of bulletproof helper functions. One init() at process start is the whole integration — everything else is optional. Built on OpenTelemetry (the Python API/SDK, the OTLP-over-HTTP exporters, and the official opentelemetry-instrumentation-* contrib packages); see THIRD-PARTY-NOTICES for full credits.

This README is the FDE manual — everything you need to integrate, verify, and troubleshoot lives here, not in foam's source.

Install

pip install foam-otel

pip install foam-otel installs the universal instrumentation floor by default (since 2.1.0). The default bundle pulls the official contrib packages for the web frameworks (fastapi/ASGI, django, flask), outbound HTTP (requests, urllib3, httpx), the primary databases (psycopg, asyncpg, mysql, pymysql, redis, pymongo), the celery queue (trace continuity across the queue hop), host + runtime system metrics, and the LLM floor — openai-v2 (chat/embeddings), the Anthropic instrumentor, and the google-genai instrumentor (Gemini) — alongside the always-on stdlib root-logger bridge and foam's OpenAI Responses gap-filler (sync + async). init() presence-checks every one of them (the entry-point sweep, below): each activates only when its target library is importable, never blind-registers, and stands down rather than double-instrument a library you already instrument — so raising the floor adds signals, never breaks the ones already flowing. SQL statement text and bind params, redis commands (key AND value), HTTP request/response headers (inbound server AND outbound client spans), URL query params and credentials, log bodies, and LLM prompt/response content are captured RAW by default — foam flips the upstream capture knobs on and disables the upstream sanitizers at activation (see the raw-capture rows in the env table below). Redaction is fully opt-in above the always-on credential floor (fleet ruling 2026-07-26): the frozen credential/secret NAME denylist always masks to [REDACTED]; nothing else is masked unless you list its key. No adapter code ships in foam.

Instrumentations beyond the floor arrive as a plain install of the official contrib package, activated the same way at init():

pip install opentelemetry-instrumentation-botocore   # AWS SDK, activated on presence

Any other official instrumentation you install (-cassandra, -elasticsearch, -grpc, …) is activated the same way at init() — presence-checked, never blind-registered. (The pre-2.1 foam-otel[fastapi] / [httpx] / [llm] extras still resolve; they are now subsets of the default bundle.)

The scenario matrix

One copy-paste-ready snippet per supported situation (spec rule 33). If your situation is missing here, that is a docs bug — file it. All calls use the real init(...) keyword arguments. Load foam's init() before your app imports the libraries you want instrumented — instrumentation that registers after a library is imported never applies to already-bound references (GOTCHAS G1).

# 1 — CLEAN SERVICE (door 1, the common case) — run at the top of your entrypoint
import os
from foam_otel import init

env = os.environ.get("APP_ENV", "development")
init(
    name="checkout-api",                 # -> service.name
    environment=env,                     # -> deployment.environment.name (verbatim)
    enabled=env not in ("test", "ci"),   # the one export switch (idiomatic recipe)
    token=os.environ["FOAM_OTEL_TOKEN"],      # Bearer auth; required only when enabled
    version=os.environ.get("GIT_SHA"),   # -> service.version (recommended)
)

# 2 — BESIDE A PROPRIETARY AGENT (situation A): disjoint pipelines.
# Foam runs its own pipeline next to the agent's; you wire the agent's intake
# host into the loop guard so foam never traces the agent's own egress.
init(
    name="checkout-api", environment=env, enabled=True, token=FOAM_OTEL_TOKEN,
    ignored_outbound_hosts=["agent-intake.vendor.example"],
)

# 3 — TENANT RIDES FOAM (situation C): a scoped SDK (LLM eval / AI-obs) attaches
# a processor to foam's pipeline. Pass a CONSTRUCTED instance + the REQUIRED
# ignore entry for its exporter's host (or its export traffic becomes a loop).
from eval_tool_sdk import EvalToolSpanProcessor

init(
    name="checkout-api", environment=env, enabled=True, token=FOAM_OTEL_TOKEN,
    additional_span_processors=[EvalToolSpanProcessor(project="prod")],
    ignored_outbound_hosts=["ingest.eval-tool.example"],
)

# 4 — FOREIGN OTEL SDK OWNS A SIGNAL (situation B): door 2 — the ingest entries.
# ONE LINE per claimed signal, added to THEIR OTel setup. The tap is an ADDITIVE
# READER: their pipeline keeps working exactly as before, and now ALSO sends
# foam a masked, identity-stamped COPY. Full contract + the REQUIRED loop step:
# see "Door 2 — the ingest entries" below.
from foam_otel import (
    create_foam_ingest_span_processor,
    create_foam_ingest_log_record_processor,
    create_foam_ingest_metric_reader,
)

# THEIR setup module — one added line per claimed signal:
their_tracer_provider.add_span_processor(create_foam_ingest_span_processor(
    token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV))
their_logger_provider.add_log_record_processor(create_foam_ingest_log_record_processor(
    token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV))
their_meter_provider = MeterProvider(          # THEIR constructor line — THE recipe
    resource=their_resource,
    metric_readers=[their_reader, create_foam_ingest_metric_reader(
        token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV)],
)
# If any signal is free, ALSO run init() — the doors compose (scenario 5).

# 5 — PER-SIGNAL COMPOSITION (traces claimed shown; symmetric over any subset).
# A foreign SDK owns traces only; metrics + logs are free.
init(name="checkout-api", environment=env, enabled=True, token=FOAM_OTEL_TOKEN)
# -> warns ONCE: "traces is owned by <owner> — foam is dark for traces", and the
#    warning names the one-line fix. metrics + logs register to foam on the FULL
#    contract. Trace ids still correlate through the shared W3C context.
their_tracer_provider.add_span_processor(create_foam_ingest_span_processor(
    token=FOAM_OTEL_TOKEN, environment=env))   # the tap rides THEIR tracer
# -> foam now ALSO receives their spans (tier-marked foam.ingest.tier=external,
#    carrying THEIR service identity); door-1 metrics/logs carry foam's identity.
# (Python has per-signal globals, so the doors compose — unlike Java's single
#  composite global.)

# 6 — TESTS / CI: fully inert. No token needed — token validates only when enabled.
init(name="checkout-api", environment="test", enabled=False)
# No SDK loads, no providers registered, no network, no warning. Helpers no-op.

# 7 — SERVERLESS (door 1 + the flush recipe). The platform freezes the process
# the instant the handler returns; flush() before it does, or buffered telemetry
# is lost. flush() never raises and is safe even if init() never ran.
from foam_otel import init, flush

init(name="orders-fn", environment="production", enabled=True, token=FOAM_OTEL_TOKEN)

def handler(event, context):
    try:
        return run(event)
    finally:
        flush()

Scenario 8 (browser) does NOT apply to Python. It exists only in the JS core's matrix — the browser package is a separate companion, build-time gated with a browser-scoped token, and this Python core has no browser surface.

init options

init() is keyword-only. Required arguments raise at boot (rule 10) so a misconfiguration fails on your machine, never silently in production. The unhappy path, shown (every other foam function no-ops instead of raising — init() is the ONE deliberate exception):

# MISCONFIGURED BOOT: a missing/blank required argument raises at init —
# the failure is a red CI run, never a dark production service.
from foam_otel import init

init(name="", environment="production", enabled=True, token="t0k")
# -> ValueError: [foam] init: name is required and must be a non-empty string
# Same shape for a blank/non-string `environment`, a non-bool `enabled`
# (TypeError), and a missing/blank `token` while enabled=True.

Option	Type	Required	Default	What it changes on the wire / why
name	str	YES	—	service.name on every export. Blank/non-string raises ValueError at init.
environment	str	YES	—	deployment.environment.name, exported VERBATIM. Values outside {production, staging, development, test} warn (typo guard) but export unchanged. Blank/non-string raises.
enabled	bool	YES	—	The ONE export switch. False = fully inert: no SDK import, no providers, no network, no warning, token untouched. True = export in that environment. Non-bool raises TypeError. No default because "on by accident in CI" and "off by accident in prod" are both incidents.
token	str	when enabled=True	None	Authorization: Bearer <token> on every OTLP export. Validated (and stripped) ONLY when enabled=True and the kill switch is unset; blank/missing there raises ValueError. Never read from the environment BY THE PACKAGE — you wire it from the service's real secret source. Fleet convention: read it from the FOAM_OTEL_TOKEN env var.
version	str	no	None	service.version, verbatim (git SHA recommended). Omitting warns — foam never detects a deploy id at runtime.
redact	Mapping	no	None	The PREFERRED grouped redaction surface (2.4.0): a mapping with up to three fields — "secrets" (same semantics as redact_keys: tail mask), "pii" (same semantics as redact_pii_keys: full [REDACTED]), and "detect" (the opt-in PII detection tier: entity names whose VALUE SHAPES are masked with typed placeholders like [EMAIL]). Absent = byte-identical to today. When a field AND its flat alias are both set, the lists UNION. LOUD at boot (rule 10): an unknown field, a non-list field value, or an unknown detect entity name raises at init().
redact_keys	Sequence[str]	no	None	Alias of redact={"secrets": [...]} (both set = union). OPT-IN secret masking, ADDITIVE on top of the always-on credential floor. With redact_keys unset/empty, NOTHING beyond the floor is masked — every other value (attributes, non-credential headers, bodies, URL query params, db.statement, logs) is captured RAW, including secrets under unlisted names. Listing a key TAIL-MASKS its value (********cafe); a matched key holding a dict/list masks in full. Normalized-token match: ["apiKey"] also catches api_key, x-api-key. A floor name listed here stays fully [REDACTED] — never downgraded to a tail.
redact_pii_keys	Sequence[str]	no	None	Alias of redact={"pii": [...]} (both set = union). YOUR OWN PII field names, FULL-ERASED to [REDACTED] (never a tail — last-4 of an email is the domain). Opt-in and additive like redact_keys: unset = nothing erased beyond the floor. Foam ships no customer PII preset and infers nothing — PII posture is your call.
secret_heuristics	bool	no	True	The ONE opt-out of the always-on value-pattern secret layer: False turns off the Tier-2 keyword+entropy HEURISTICS (H1/H2) only. The Tier-1 NAMED patterns (AWS/GitHub/JWT/PEM/DB-URL/…) and the credential floor CANNOT be disabled and are unaffected. Set False only if legitimate high-entropy telemetry (e.g. base64 content-addressed IDs) collides; the choice is logged.
capture_payloads	str	no	"off"	Inbound HTTP request/response body capture on the official ASGI server spans (fastapi/starlette) — see the payload-capture section. "off" (DEFAULT): nothing is wired, zero buffering, zero per-request overhead. "errors": bodies are buffered per request but attach ONLY when the request actually errored — an exception was recorded on the span, or the response status is >= 500; a clean 2xx/4xx attaches nothing. "always": attach on every request at response completion. The FOAM_CAPTURE_PAYLOADS env var OVERRIDES this option when set (the FDE clamp valve; precedence: env > init). Any other init value raises ValueError at boot; an invalid env value warns and falls back to this option. Bodies are capped at 8192 chars (…[truncated] marker; true byte sizes ride http.*.body.size), textual content types only, and masked by the same central redaction engine as everything else.
capture_locals	bool	no	False	Exception-time local variable capture — see the capture-locals section. False (DEFAULT): nothing runs — no frame walking, zero cost, wire byte-identical. True: when foam records an exception (the record_exception helper or the thrown-then-mapped gap-filler), the in-app frames of exc.__traceback__ are walked and ONE exception.local_variables attribute (a capped, scrubbed JSON snapshot, frames nearest the raise first) is merged into the existing exception span event — never a second event. The FOAM_CAPTURE_LOCALS env var (true/false) OVERRIDES this option in both directions when set (env > init; invalid env values warn and fall back). A non-bool init value raises ValueError at boot. Variable names on the credential floor and your redact lists mask in-process, every serialized string rides the value-pattern secret scan (reprs included, pre-truncation), and every cap overflow is explicitly annotated. Deliberately independent of capture_payloads (interior state vs wire data).
additional_instrumentations	Sequence[object]	no	None	Constructed instrumentor instances registered on foam's pipeline, fault-isolated: one throwing instance is skipped with a loud [foam] warning naming it, boot proceeds.
additional_span_processors	Sequence[object]	no	None	Tenant seam (traces). Your SpanProcessor joins foam's tracer provider AFTER the redaction stage — you see the masked view, byte-identical to what foam exports. A throwing instance is DISABLED and warned; foam's export is unaffected.
additional_log_record_processors	Sequence[object]	no	None	Same seam, logs (LogRecordProcessor).
additional_metric_readers	Sequence[object]	no	None	Same seam, metrics. NOTE: a MetricReader TRIGGERS collection on its own schedule/thread — that is your code on your thread; foam's never-throw guard covers only foam-invoked calls (flush/shutdown), not your reader's collect loop.
ignored_outbound_hosts	Sequence[str]	no	None	EXTENDS the outbound loop guard (rule 24), never replaces it. Explicit hostnames whose outbound calls produce no client spans. REQUIRED for any tenant/agent exporting over HTTP from inside your process (scenarios 2/3).
before_send	callable / Sequence	no	None	Export hook (2.8.0): one callable — or a sequence run as a pipeline — called with every span and log record at the export boundary, after batching, so nothing a hook drops is ever serialized or leaves the process. Return the record (or a same-type replacement) to export it — in-place mutation supported; return None to drop it; a raise or foreign return drops that record loudly, fail-closed, never a raise into your threads. Hook output is RE-masked by the credential floor + your redact lists (in this package the primary redaction runs BEFORE batching, at the processor seam) — a hook can never widen what ships. Spans and logs only; door 1 only. A non-callable (or a callable that cannot take one positional argument) raises at boot. Process-global. Hooks run on the export thread — keep them non-blocking. Full contract: "The before_send hook" below.
diagnostics	bool	no	False	Turns on [foam] self-narration at INFO (the init-ok line with per-signal ownership + instrumentation count, skipped-instrumentation notes). Off = the same messages log at DEBUG. Telemetry data never appears here.

Unknown keyword arguments are ignored with a single [foam] warning naming them (forward/backward-compat safety) — they never raise.

HTTP header capture (always-on, inbound AND outbound)

What is captured. With zero config — no init option, no env var — every HTTP request AND response header lands on the official spans as semconv http.request.header.<name> / http.response.header.<name> attributes (names lowercased, dashes folded to underscores; values are string arrays, multi-value headers per each client/server library's own header model):

• Inbound (SERVER spans): the ASGI/WSGI-backed frameworks — fastapi, django, flask.
• Outbound (CLIENT spans, since 2.4.x): the bundled requests, urllib3, and httpx instrumentations — plus any beyond-floor official client instrumentation you install that honors the same standard env vars (aiohttp-client and stdlib urllib do at the pinned contrib line).

The mechanism is entirely the upstream one: foam writes OTel's own documented OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_{SERVER,CLIENT}_{REQUEST,RESPONSE} vars to .* at activation (see the env table below) — no instrumentation internals are touched. gRPC metadata is not captured: the official grpc instrumentation documents no metadata-capture option (see the deliberately absent knobs section).

Floor masking, automatic. The seven credential headers — authorization, proxy-authorization, cookie, set-cookie, x-api-key, x-auth-token, www-authenticate — always export as [REDACTED], both directions, both span kinds, with zero config and no off switch (the credential floor below; the value-pattern layer additionally masks credential-SHAPED values such as Bearer … tokens under any header name). Everything else rides RAW by design.

How to narrow it. Three cooperating levers, none of which can shrink the floor:

• redact_keys / redact_pii_keys at init(): listing a header name masks its captured value(s) on every span — redact_keys=["x-internal-token"] catches http.request.header.x_internal_token too (bare-name matching is normalized).
• Your own OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_* allowlist: a narrower operator value set before init() is RESPECTED per direction — foam only writes .* where you set nothing, and never widens yours.
• Your own OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS: UNIONED with the floor's seven — your extra fields mask at the moment the attribute is set, the floor never shrinks.

HTTP payload capture (opt-in capture_payloads)

Off by default. Request/response bodies never leave the process unless you turn this on — and in the default "off" mode the capture hooks are not even wired (the instrumented middleware skips the empty hook slots: zero buffering, zero per-request overhead, wire byte-identical to 2.5.x).

import foam_otel

foam_otel.init(
    name="orders",
    environment="production",
    enabled=True,
    token="…",
    version="7f3c2e1",
    capture_payloads="errors",  # "off" (default) | "errors" | "always"
)

The three modes.

Mode	What attaches, and when
"off" (default)	Nothing, ever. No hooks wired, nothing buffered.
"errors"	Bodies are buffered per request and attach to the server span ONLY when the request actually errored: an exception was recorded on the span (foam's record_exception, the thrown-then-mapped gap-filler, or an unhandled raise the official middleware records), or the response status is >= 500. A clean 2xx/4xx attaches NOTHING — the buffer dies with the request.
"always"	Bodies attach on every request at response completion.

Env override (the FDE clamp valve). FOAM_CAPTURE_PAYLOADS=off|errors|always OVERRIDES the init option whenever it is set — precedence: env var > init option — so an operator can clamp (always → off) or widen (off → errors) a deployment without a code change. It is read once, at init(). An invalid env value warns loudly and falls back to the init option; an invalid init value raises ValueError at boot (fail in CI, not dark in prod).

What lands on the wire. Four attributes on the official ASGI server span (never a foam-invented name):

• http.request.body / http.response.body — the payload text, utf-8 decoded defensively (errors="replace"), capped at 8192 chars with a literal …[truncated] marker appended when the cap bit (fleet constant, identical to the browser package);
• http.request.body.size / http.response.body.size — the TRUE byte counts (counting continues past the cap), set whenever the stream was observed — including for payloads whose text is skipped.

Body TEXT is captured for textual content types only (text/*, JSON incl. +json, XML incl. +xml, x-www-form-urlencoded, graphql) and only when Content-Encoding is absent/identity; binary or compressed payloads record sizes only. A request body your handler never reads is never captured (foam observes the ASGI receive events the app itself triggers — it never reads, consumes, delays, or reorders the stream, so capture cannot change app behavior), and no size is fabricated from Content-Length. Two wire-truth special cases: HEAD responses record NO body attributes at all (the framework may generate the full payload at the ASGI seam, but the server strips it — the wire carries nothing, so foam reports nothing), while an actually-empty body (204/304/empty 200) records its truthful http.response.body.size of 0. Mounted instrumented sub-apps (app.mount() of another FastAPI/Starlette app, each carrying its own OTel middleware) capture ONCE, single-counted, on the outer SERVER span — nested layers relaying the same events are deduplicated, and replayed request streams (body-caching middleware between layers) are not double-counted.

Security posture. Captured bodies ride the SAME central redaction engine as every other attribute (see the sections below — there is no separate payload key list): a JSON body is parsed on export and deep-walked, so a real payload key on the credential floor (password, token, …) or on your redact_keys / redact_pii_keys / redact lists masks exactly as it would anywhere else. A JSON body the cap TRUNCATED (the …[truncated] marker breaks the parse) gets the same guarantee: the intact prefix is completed back into parseable JSON and deep-walked by the same engine — failing CLOSED to the full mask when it cannot be — so the floor holds past the cap too, never downgraded to a key-blind text scan. Non-JSON bodies (plain text, form, XML — truncated or not) take the standard string pass (always-on value-pattern secret scan, the form/query pair tokenizer with its per-pair name matching, and the opt-in detect tier). In "errors" mode remember the anti-fabrication contract cuts both ways: error payloads are exactly the ones most likely to carry user input, so review your redaction lists before widening to "always".

Scope. Inbound ASGI (fastapi, and starlette when its official contrib instrumentation is installed) only — the surfaces whose official instrumentation exposes a body-observing hook. Deliberately skipped (no standard seam; revisited when upstream grows one): WSGI (django/flask — the official hooks expose request/response objects, and reading their bodies would consume or force the app's stream), outbound client bodies (requests/urllib3/httpx), gRPC message payloads, and queue payloads. See GOTCHAS P26 for the operational hazards.

Local-variable capture (opt-in capture_locals)

Off by default. Frame-local variables never leave the process unless you turn this on — in the default False posture no capture code runs at all (no frame walking, no serializer on any path; the wire is byte-identical to a build without the feature). This transparency sentence is a tested claim, not prose. Binding design (vendor comparison, caps, rationale): docs/decisions/locals-capture-design.md.

import foam_otel

foam_otel.init(
    name="orders",
    environment="production",
    enabled=True,
    token="…",
    version="7f3c2e1",
    capture_locals=True,  # default False
)

What lands on the wire. ONE attribute, exception.local_variables, merged into the attributes of the EXISTING exception span event whenever foam records an exception — a JSON string of the in-app frames nearest the raise: [{"function", "file", "line", "vars": {...}}, ...]. Never a second event, never a foam.* name, existing exception dedupe untouched.

Which records carry it (trigger truth). Foam-authored exception records: the record_exception(err) helper and the thrown-then-mapped gap-filler. NOT carried on records foam never authors: the span() helper's automatic failure record and the OTel ASGI middleware's unhandled-raise record are written by the SDK/middleware themselves, so those events have the standard exception attributes but no locals.

Scope and caps (every truncation carries an explicit annotation like <truncated:len=N> — capped is always distinguishable from complete): IN-APP frames only (site-packages/dist-packages, the stdlib, foam_otel and opentelemetry frames are skipped); max 10 frames (nearest the raise), 10 variables per frame, 3 container levels, 256 chars per serialized value, 2048 bytes per frame, 16 KB total. Values are serialized with a guarded safe-repr: a raising __repr__ becomes "<repr failed>", cycles become "<cycle>" — capture can never break the exception being recorded.

Env override (the FDE clamp valve). FOAM_CAPTURE_LOCALS=true|false OVERRIDES the init option whenever it is set — precedence: env var > init option, both directions (clamp True → false, or widen without a deploy). Read once, at init(). An invalid env value warns loudly and falls back to the init option; an invalid init value raises ValueError at boot.

Security posture. Locals are the broadest capture class this package has (anything a frame ever touched), so the scrub pipeline runs IN-PROCESS, before the value is even placed on the span — and then the central redaction engine deep-walks the JSON again on export (the same treatment captured HTTP bodies get), as defense in depth:

1. the credential floor matches variable names AND nested dict keys recursively, with locals-specific normalization (lowercase + strip _-$@ — apiKey, API_KEY, api-key, pass$word all mask to [REDACTED]); no off switch;
2. the value-pattern secret layer scans EVERY serialized string — object/exception reprs included, variable names and dict keys included (a cache keyed by a token or a session map keyed by an email carries the secret on the NAME side of the pair, where name-list matching is blind), and BEFORE truncation — so a bearer token embedded in a repr under an innocuous variable name is masked (the classic locals-capture leak class; see GOTCHAS P27);
3. your redact_keys / redact_pii_keys / redact lists apply to variable names and nested dict keys with the same normalized matching and mask shapes as everywhere else — except that a credential-shaped string value under a redact_keys name masks in FULL ([REDACTED], the same upgrade the central engine applies): listing a variable never masks weaker than not listing it.

The collector backstop (org-wide off switch). Because the signal rides one attribute name, an operator can drop the whole class at the OTel Collector without touching any deployment:

processors:
  attributes/drop-locals:
    actions:
      - key: exception.local_variables
        action: delete

The default credential denylist (always-on)

One exception to opt-in redaction (fleet ruling 2026-07-26): the credential floor. The package always masks — in every signal, on door 1, every door-2 tap, and the view handed to tenant additional_* instances — the VALUES of a fixed, frozen list of credential/secret NAMES to the literal [REDACTED] (full, no tail):

• The seven credential headers, matched as header names wherever headers are captured (including the semconv http.request.header.<name> / http.response.header.<name> span attributes, masked per header instance with the string-array arity preserved): authorization, proxy-authorization, cookie, set-cookie, x-api-key, x-auth-token, www-authenticate.
• The frozen 52-entry key list in contract/credential-denylist.json — the union of sentry-python's default denylists and foam's documented credential roots (password, secret, token, api_key, jwt, ssn, session, …) — matched as attribute/field names at every depth the engine walks: span and span-event attributes, log attributes, structured log-body fields (nested dicts included), metric datapoint attributes and exemplar filtered_attributes, and URL query parameter names (?token=x&user=bob → token=[REDACTED]&user=bob).

The matching rule: case-insensitive, dash/underscore-normalized EXACT name-equality — never substring. Authorization, AUTHORIZATION, and x_api_key/X-Api-Key all match; authorization_url, secretary, and x-api-key-id never do. The floor masks NAMES only — but it is no longer the only always-on control: the value-pattern secret layer below masks credential-SHAPED VALUES (a Bearer token, an AWS key) even under an unlisted name.

There is no off switch. No init option, env var, or door-2 parameter can disable, shrink, or re-spell the floor; redact_keys/redact_pii_keys are additive on top, and listing a floor name yourself never weakens its full mask. (enabled=False remains fully inert — nothing exports at all.) Rationale: these names carry credentials, not debug signal — capturing them raw is a breach vector with no observability upside, so safety here is deliberately not configurable away.

Defense-in-depth on the header path: foam also derives OTel's own OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS from the same frozen list (anchored regexes, ^authorization$,…), so the ASGI/WSGI middlewares mask these header values at the moment the attribute is set. The central engine remains authoritative either way (see the env table below).

Migration note (2.2.0, minor — security-motivated wire change): before 2.2.0 these header/field values exported RAW by default. If a dashboard or alert keyed off a raw credential value (it should not have), it now sees [REDACTED]. Everything else is unchanged and still exports raw.

The value-pattern secret layer (always-on)

Second always-on control (fleet ruling 2026-07-27): value-shape masking. The credential floor above masks by NAME; this layer masks by VALUE SHAPE. foam now masks credential-SHAPED values — AWS/GCP/Azure keys, GitHub/GitLab tokens, Slack, Stripe, Anthropic/OpenAI keys, JWTs, private-key blocks (RSA/EC/OPENSSH/PGP + GCP service-account JSON), user:pass@ URI credentials, and Bearer/Basic tokens — to the literal [REDACTED] in every exported string value, regardless of field name: span/log/event attributes, log bodies, db.statement (SQL), LLM content (gen_ai.*), URL query AND fragment values, exception/status messages, baggage, and exemplar attributes. Only the matched span is masked (no partial reveal). Metric datapoint attributes (labels) are exempt — low-cardinality by construction.

Beyond the named patterns, a generic keyword+entropy heuristic (H1: a secret-looking key = <value> assignment; H2: a standalone high-entropy token) catches unnamed providers. It is the ONE part you can disable — secret_heuristics=False — for legitimate high-entropy telemetry that collides; the named patterns and the credential floor cannot be disabled.

The scanner is compile-once, anchor-prefiltered, ReDoS-bounded (a 256 KiB per-value cap; oversized values with a secret anchor are masked whole), and fail-closed (any scanner error masks the value, never leaks it).

Coverage also widened in the same release: redaction now reaches URL fragments (OAuth implicit-flow #access_token=…, hash-router #/reset?token=…), http.*.header.* attribute keys (your redact_keys names now match the bare header name), nested attribute containers (fail-closed — a credential in a kvlist/array log attribute no longer slips through), structured LLM content before serialization, and form-urlencoded bodies.

Migration note (2.3.0, minor — security-motivated wire change): foam now masks credential-SHAPED values (regardless of field name) and reaches URL fragments, http.*.header.* keys, nested containers, and form bodies. If a dashboard keyed off a raw token value (it should not have), it now sees [REDACTED]. Non-credential-shaped values are unchanged and still export raw; disable only the generic heuristic with secret_heuristics=False if it collides with legitimate high-entropy data.

PII detection (opt-in)

The third value control (fleet ruling 2026-07-28) — and the first that is fully OPT-IN. List an entity in redact={"detect": [...]} and foam masks that entity's VALUE SHAPE wherever the value-pattern secret layer runs (span/event/link attributes, log attributes and bodies, URL/query leaves, nested structures) — replacing ONLY the matched span of text with a typed placeholder, everything around it preserved verbatim. Nothing here runs unless you list the entity; with detect absent or empty the wire is byte-identical to 2.3.0.

The exact entity set (frozen fleet-wide in contract/pii-detect.json — these five names are the ONLY valid detect entries; anything else raises at init()):

Entity	Placeholder	Matches
email	[EMAIL]	RFC-shaped addresses with a real TLD (user@localhost does not match)
phone	[PHONE]	space/dash/parenthesis-delimited numbers, optional +country code (bare digit runs, versions, and clock times do not match)
ssn	[SSN]	delimited 3-2-4 US SSNs with a consistent delimiter (structurally invalid areas/groups/serials and undelimited runs do not match)
credit_card	[CREDIT_CARD]	13-19 digit PANs (spaced/dashed/contiguous) that pass Luhn — a checksum-failing digit string is never masked
ip	[IP]	IPv4 with valid octets, full and ::-compressed IPv6

from foam_otel import init
import os

init(
    name="support-api",
    environment="production",
    enabled=True,
    token=os.environ["FOAM_OTEL_TOKEN"],
    redact={
        "secrets": ["client_ref"],        # == redact_keys (tail mask); both set = union
        "pii": ["patient_name"],          # == redact_pii_keys (full [REDACTED])
        "detect": ["email", "ssn"],       # NEW: value-shape detection, typed placeholders
    },
)
# "reach me at ada@example.com, SSN 536-90-4399"
#   -> "reach me at [EMAIL], SSN [SSN]"

What to know before relying on it:

• Names and free-text prose are NOT detectable. A person's name, an address, a diagnosis in a sentence — anything without a machine-checkable shape — is server-side NER scope, not something a regex tier can promise. For fields you know carry PII, list the field name in redact["pii"] instead; that masks the whole value regardless of shape.
• Precedence: the credential floor stays terminal, the always-on secret value layer runs first (a JWT is [REDACTED], never [EMAIL]), then detect, then your listed keys. Detect never weakens any earlier layer.
• Placeholders are stable: running detection over already-masked output changes nothing, and only the matched span is replaced.
• Metric datapoint attributes (labels) are exempt — exactly where the secret value layer is exempt (low-cardinality by construction).
• Fail-closed: a detection engine error masks that whole value to [REDACTED] with a loud [foam] warning — never the raw value, never a crash. The scanner rides the same compile-once, bounded, capped discipline as the secret layer.
• Door-2 parity: all three ingest entries accept the same redact object with the same union semantics.

Migration note (2.4.0, minor — additive only): redact= and the detect tier are new, opt-in surface. With redact absent the wire is byte-identical to 2.3.0; redact_keys/redact_pii_keys keep working unchanged as aliases.

Cross-language param names — the same customer options, identical concept, per-language casing, on init() AND every door-2 tap (they never drift; the credential floor itself has NO init surface in any language). The grouped redact object ships in the Python and Ruby cores this wave; js/otel, js/browser, and java follow under the fleet-parity rule:

Concept	js/otel	js/browser	python	ruby	java
secret keys (tail-mask)	redactKeys	redactKeys	redact_keys	redact_keys:	.redactKeys(String…)
PII keys (full [REDACTED])	redactPiiKeys	redactPiiKeys	redact_pii_keys	redact_pii_keys:	.redactPiiKeys(String…)
grouped object (secrets/pii/detect)	follow-up wave	follow-up wave	redact	redact:	follow-up wave
export hook (per-record transform/drop)	follow-up wave	follow-up wave	before_send	before_send:	follow-up wave

Deliberately absent knobs (documented AS absent)

• endpoint — the fleet endpoint (https://otel.api.foam.ai) is pinned in code. The ONE override is the operator-level OTEL_EXPORTER_OTLP_ENDPOINT env var (warns loudly). Never an init option: a code change must not silently re-route telemetry.
• sampling / any sampler knob — none. Every span ships (ParentBased(ALWAYS_ON)); an unsampled inbound flag cannot drop foam's spans, and OTEL_TRACES_SAMPLER is inert. Cost control is server-side (rule 4).
• cadence / batch-delay knobs — not init surface. Batch cadence is delegated to the standard OTEL_BSP_SCHEDULE_DELAY / OTEL_BLRP_SCHEDULE_DELAY / OTEL_METRIC_EXPORT_INTERVAL env vars (operator tuning), read by the SDK's own processors.
• disabled_environments — gone. enabled is an explicit boolean; compute it from your environment in one line (scenario 1).
• exporter / processor injection — the only pipeline seam is the three additive additional_* options. They can never redirect, drop, or displace foam's export.
• credential-floor off switch — none, deliberately (see the default credential denylist): the floor is not part of any config object, so no option surface can build a floor-less engine. Safety must not be un-configurable-away.
• gRPC metadata capture — none, deliberately OUT OF SCOPE. The official opentelemetry-instrumentation-grpc package documents no metadata-capture option or env var (only generic interceptors), so there is no standard mechanism to enable; foam never reaches into instrumentation internals to invent one. A grpc instrumentation you install still activates through the sweep and emits its standard spans — just without metadata attributes.

Environment variables

Variable	Posture
OTEL_SDK_DISABLED=true	HONORED — the emergency kill switch (config-only, no redeploy). SUPERSEDES enabled=True: when set, foam is fully off and warns loudly. Parsed exactly like the SDK: literal true, case-insensitive, trimmed.
OTEL_EXPORTER_OTLP_ENDPOINT (+ OTEL_EXPORTER_OTLP_{TRACES,METRICS,LOGS}_ENDPOINT)	HONORED — the one operator-level override of the pinned fleet endpoint. Per-signal variant wins over the base (OTLP-spec precedence). Active override = one loud [foam] warning per signal naming the destination.
OTEL_BSP_SCHEDULE_DELAY, OTEL_BLRP_SCHEDULE_DELAY, OTEL_METRIC_EXPORT_INTERVAL	HONORED — batch/export cadence, delegated to the SDK's own batch processors and periodic reader. Positive integers only; anything else is ignored. Never customer (init) surface.
OTEL_PROPAGATORS=none	HONORED — the propagation kill lever: injection stops AND inbound extraction stops (the service becomes its own trace root), telemetry keeps flowing. Foam implements none itself and warns loudly. Any OTHER value warns "set but ignored" — foam's propagator set is fixed (W3C tracecontext + baggage).
OTEL_PYTHON_DISABLED_INSTRUMENTATIONS	INERT — set values warn loudly "set but ignored" and disable NOTHING: every installed instrumentation and both foam gap-fillers activate regardless. Deliberate: the honored table above is CLOSED, and an env lever that silently drops whole instrumentations' telemetry is exactly the class it bans. (Versions ≤ 1.1.x HONORED this var — see the migration notes below.)
OTEL_PYTHON_TRACER_PROVIDER / OTEL_PYTHON_METER_PROVIDER / OTEL_PYTHON_LOGGER_PROVIDER	INERT — foam registers by ATTEMPT (set-once, first wins), never by pre-reading a slot, so the API's entry-point provider loaders never fire on foam's account. Proven inert by a dedicated test per var.
FOAM_CAPTURE_PAYLOADS=off|errors|always	HONORED — one of the two foam-named env vars the package reads: the payload-capture clamp valve. Set, it OVERRIDES init's capture_payloads option in BOTH directions (clamp always→off or widen off→errors) — precedence: env > init. Read once at init(). An invalid value warns loudly and falls back to the init option (never crashes, never guesses).
FOAM_CAPTURE_LOCALS=true|false	HONORED — the other foam-named env var: the locals-capture clamp valve, same shape. Set, it OVERRIDES init's capture_locals option in BOTH directions — precedence: env > init. Read once at init(). An invalid value warns loudly and falls back to the init option.
TRACEPARENT / TRACESTATE	HONORED (inbound) — a launcher that sets these parents the whole process to its trace (spawn is an inbound call whose carrier is the environment). No-op when unset.
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_{REQUEST,RESPONSE}, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_CLIENT_{REQUEST,RESPONSE}, OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS	WRITTEN BY FOAM (raw-capture posture, never read as config) — foam sets GenAI content capture ON, header capture to ALL (.*) on inbound SERVER and outbound CLIENT spans, and sanitize-fields to the credential floor's seven headers as ANCHORED regexes (^authorization$,^proxy-authorization$,… — derived from the one frozen constant; anchored because util-http matches via unanchored re.search, and a bare stem would over-mask authorization-url), all at activation — both inside the strip window (for instrument()-time readers: openai-v2's GenAI switch, and the requests/urllib3/httpx client instrumentors' CLIENT header vars) and persisted while foam is active (for the ASGI/WSGI middlewares, which read the SERVER vars at middleware construction, httpx's per-client transports, which read the CLIENT vars at transport construction, and util-genai's request-time paths). The sanitize-fields write is defense-in-depth: it masks the seven header values at the ASGI/WSGI attribute-set moment with OTel's own [REDACTED] literal; the central engine enforces the same floor regardless (os.environ is host-mutable — the engine backstop is what makes the floor non-disableable). The GenAI var has TWO upstream grammars: legacy readers parse true, util-genai — the reader behind google-genai AND openai-v2 at the pinned window — parses a ContentCapturingMode enum; foam writes each reader class its own spelling at its instrument() moment and persists the enum (SPAN_AND_EVENT). Undone at shutdown() (an operator value that predated init is restored). Header vars are OPERATOR-RESPECTING (2.3.0, F-PY3 — all four SERVER/CLIENT names): foam's .* capture default and the floor's seven sanitize fields apply only where the operator set nothing — a narrower operator capture allowlist (Content-Type) is RESPECTED, never widened to .*, and operator sanitize fields are UNIONED with the floor's seven (never blanked/shrunk). The GenAI content switch stays foam-owned. The FDE clamp path is redact_keys/redact_pii_keys, the always-on value-pattern layer, and now a respected operator capture allowlist.
upstream URL sanitizers (redact_url — not an env var, an always-on util-http default)	DISABLED while foam is active — upstream unconditionally rewrites URL credentials and AWSAccessKeyId/Signature/sig/X-Goog-Signature query values to REDACTED on every recorded http.url, with no upstream knob. Foam replaces the sanitizers with identity at activation (restored at shutdown()), so URLs ride RAW — except query-parameter NAMES on the credential floor (token, sid, signature, …), which foam's own engine masks per pair; list additional keys in redact_keys to mask them too. See GOTCHAS P19.
Every other OTEL_* (OTEL_SERVICE_NAME, OTEL_RESOURCE_ATTRIBUTES, OTEL_TRACES_SAMPLER, OTEL_EXPORTER_OTLP_HEADERS/_TIMEOUT/_COMPRESSION, OTEL_SEMCONV_STABILITY_OPT_IN, OTEL_*_EXPORTER, attribute limits, …)	INERT — and inert BY CONSTRUCTION. Foam reads every honored var itself, then builds the whole pipeline inside a synchronous window that strips every OTEL_* var from os.environ (restored before init returns), so no un-consumed var reaches an exporter/provider/instrumentation constructor. Identity and auth come from init options only.

The before_send hook

before_send= is the last word your code gets on every span and log record before it is exported to foam. It runs at the export boundary — after the batch processor hands foam the finished records, immediately before the OTLP serialization — so a record your hook drops is never serialized and never leaves the process.

# The before_send export hook (2.8.0): transform or drop spans/logs right
# before they are exported to foam. Return the record (mutate in place
# freely) or None to drop; a raising hook drops ONLY its record, loudly,
# and never raises into your threads (fail-closed).
import os
from foam_otel import init

def scrub(record):
    if getattr(record, "name", None) == "GET /healthz":
        return None                              # drop the noise
    record.attributes.pop("internal.debug.blob", None)  # trim before it ships
    return record

init(
    name="checkout-api", environment=os.environ.get("APP_ENV", "development"),
    enabled=True, token=os.environ["FOAM_OTEL_TOKEN"],
    before_send=scrub,
)
# Or a sequence pipeline — hooks run in order, each receiving the previous
# hook's return; the first None drops the record and short-circuits:
# init(..., before_send=[drop_healthchecks, trim_debug_attrs, stamp_team])

The pipeline order is pinned — and it is the ONE deliberate ordering divergence from the Ruby core (where redaction lives at the exporter and the hooks run before it). In this package the primary redaction pass runs at the processor seam, before batching, so the documented Python order is:

session stamp → redaction (processor) → batch → before_send hooks →
RE-mask of hook output → OTLP wire

Two consequences of that order, both proven in the test suite:

• A hook can never widen what ships — identically to Ruby. Whatever your hooks return is RE-masked (credential floor + your redact lists; log bodies included) before the inner exporter, so an attribute a hook ADDS is masked exactly like one an instrumentation set.
• Hooks see the already-masked view, not the raw values (redaction ran before batching). And tenant additional_span_processors / additional_log_record_processors observe records at on_end/on_emit, BEFORE before_send — a record your hook drops was still seen by tenant processors (in Ruby, tenants sit at the exporter and see the post-hook view).

The per-record contract (Sentry beforeSend semantics):

• return the record — or a replacement of the same type → it continues to the next hook / to export. Mutating it in place is supported — record.attributes arrives as your own mutable plain-dict copy (sequence values as fresh lists), so record.attributes["k"] = v, record.attributes.pop("k") and (for logs) editing record.body — a deep copy — all just work without bleeding into the span your app still holds or the view a tenant processor saw. Span events/links/status are the exception: they are shared structures — never mutate them in place; return a replacement record instead. Foam captures the record's SDK-limit dropped-attribute count before your hooks run and restores it on the survivor, so adding/deleting attributes never corrupts the wire's dropped-count accounting.
• return None → the record is dropped, silently (that is the filter mechanism, not an error).
• raise, or return a foreign object → that record is dropped loudly — one aggregated [foam] warning per batch, error classes only (messages can carry your data) — and the batch reports failure: fail-closed, a faulted hook never ships a record you may have meant to scrub, and never raises into your application or export threads. Healthy records in the same batch still export. Because a fault marks the whole batch failed, a hook that faults deterministically also makes flush()/shutdown() report failure internally — fix faulting hooks promptly (GOTCHAS P28).

Hooks run on foam's export thread, inside the batch worker. Keep them fast and non-blocking: no network calls, no DB lookups, no sleeping — a blocked hook stalls the export pipeline. Anything a hook needs from the request/job context (tenant id, user identity) must be stamped onto the record as an attribute at capture time; context vars and thread-locals are gone by the time the hook runs.

Boot validation is strict (rule 10): a non-callable — or a callable whose signature cannot accept one positional argument (e.g. a Sentry-port lambda event, hint: ... with two required params, or a required keyword-only param) — raises ValueError at init(), because it would otherwise fault on every record and ship zero telemetry from a green boot. Make the extra Sentry hint param optional (lambda event, hint=None: ...) and it is accepted; uninspectable C callables get the benefit of the doubt.

Scope: spans and log records (the two record-shaped signals — span hooks receive a ReadableSpan-typed copy, log hooks the record-shaped API LogRecord with .body/.attributes/.severity_number; distinguish them with isinstance or hasattr(record, "body")). Metrics are aggregated state, not discrete records, and are deliberately not routed through before_send. Door-2 ingest taps are unchanged. Like enabled and the endpoint, before_send is process-global: the exporter chain is built at the first init() (and replayed to forked workers), and a second init() never changes it. With the option absent, no hook stage is wired at all — the default path is byte-identical to 2.7.0.

API — every public function

Every function below never throws and no-ops silently before init(), when enabled=False, or when the kill switch is active — the one exception is init() itself, which raises on developer error (missing/blank required options) so a misconfigured boot fails in CI, not dark in production. Import everything from the top-level foam_otel.

init(*, name, environment, enabled, token=None, version=None, redact_keys=None, redact_pii_keys=None, redact=None, capture_payloads="off", capture_locals=False, before_send=None, additional_instrumentations=None, additional_span_processors=None, additional_log_record_processors=None, additional_metric_readers=None, ignored_outbound_hosts=None, diagnostics=False) -> bool

Wire up foam telemetry (door 1). Validates arguments, then per signal ATTEMPTS to register foam's provider into OTel's set-once global and reads the outcome (attempt-then-read — foam never pre-reads a slot to predict, and never displaces an owner); a lost attempt warns once naming the owner and foam stays DARK for that signal only. Activates installed instrumentations (traces only), wires the loop guard, and installs the stdlib-logging root export. Idempotent: a second call warns and returns. Returns True when foam is exporting at least one signal, False when disabled / killed / every signal is foreign-owned or failed. See the options table for every parameter.

from foam_otel import init

owned = init(
    name="checkout-api",
    environment="production",
    enabled=True,
    token=os.environ["FOAM_OTEL_TOKEN"],
    version=os.environ.get("GIT_SHA"),
)
# owned is True when >= 1 signal reaches foam

flush() -> None

Force-flush every foam-owned signal without tearing anything down. Never raises; safe before init() (silent no-op). The serverless and pre-crash lifeline — call it before the process can freeze (scenario 7).

from foam_otel import flush

flush()  # batched spans/logs/metrics are on the wire when this returns

shutdown() -> None

Flush and stop export for every foam-owned signal, unregister the globals foam set (providers, the TRACEPARENT context token, the stdlib root handler), and reset to a pre-init state — so a later init() (e.g. a post-fork re-init) can run. Fail-open; safe without init; never raises. Helpers no-op afterward.

import signal, sys
from foam_otel import shutdown

def _graceful(*_):
    shutdown()
    sys.exit(0)

signal.signal(signal.SIGTERM, _graceful)

span(name, attributes=None)

A context manager: run a block inside a new ACTIVE span. On exception it records the exception, sets status ERROR, re-raises the IDENTICAL error, and ALWAYS ends the span. When foam is not active it yields None and runs the block uninstrumented — your code never behaves differently because telemetry is off. attributes, if given, are set on the span (masked).

from foam_otel import span

with span("price-cart", {"cart.items": len(cart)}) as s:
    total = price_cart(cart)   # if this raises, the error is recorded and re-raised

set_attribute(key, value) -> None

Set one attribute on the currently active span; no-op when none is active (never fabricates a span). The value rides the redaction engine, so a credential-floor name or a key you listed in redact_keys/redact_pii_keys is masked before it lands on the span; anything else is captured RAW (customer redaction is opt-in above the floor).

from foam_otel import set_attribute

set_attribute("order.id", order.id)
set_attribute("api_key", key)   # -> "[REDACTED]" (credential-floor name, always)
# with init(redact_keys=["client_ref"]):
set_attribute("client_ref", ref)  # -> "********" + last 4 (listed key). Un-listed, non-floor → RAW.

set_attributes(attributes) -> None

Set many attributes on the active span at once; each floor-named or listed value is masked, the rest ride RAW. No-op without an active span.

from foam_otel import set_attributes

set_attributes({"customer.tier": "gold", "password": "sup3rs3cr3tvalue"})
# -> password: "[REDACTED]" (credential-floor name); customer.tier RAW

add_event(name, attributes=None) -> None

Add a timestamped event to the active span; listed-key attributes masked, the rest RAW. No-op without an active span.

from foam_otel import add_event

add_event("cache.miss", {"key": cache_key})

record_exception(error, attributes=None) -> None

Record a standard OTel exception event (type/message/stacktrace) on the active span and set status ERROR. Accepts any BaseException, including your own error classes. NEVER notifies a vendor tracker — the bridge direction is tracker -> foam only. No-op without an active span. Listed-key attributes masked, the rest RAW. With the opt-in capture_locals on, this call also merges the exception.local_variables snapshot into the same exception event.

from foam_otel import record_exception

try:
    charge(card)
except PaymentError as err:
    record_exception(err)
    raise

get_trace_context() -> dict

The active span's ids for correlating foam traces with your OWN logs and outbound payloads: {"trace_id": <32 hex>, "span_id": <16 hex>}. Returns an EMPTY dict when no valid span is active — never fabricates ids.

from foam_otel import get_trace_context

ctx = get_trace_context()
if ctx:
    my_logger.info("charging card", extra={"trace_id": ctx["trace_id"]})

increment_counter(name, n=1, attributes=None) -> None

Add to a monotonic counter (create_counter). Instruments are lazily created and cached per name (rebound once if the provider swaps). Names pass through verbatim.

from foam_otel import increment_counter

increment_counter("orders_placed", 1, {"plan": "pro"})

record_histogram(name, value, attributes=None) -> None

Record a distribution observation (durations, sizes) via create_histogram.

import time
from foam_otel import record_histogram

record_histogram("checkout_duration_ms", (time.monotonic() - started) * 1000)

add_up_down_counter(name, n, attributes=None) -> None

Add to a counter that can decrease (in-flight counts, pool sizes), via create_up_down_counter. Pass a negative n to subtract.

from foam_otel import add_up_down_counter

add_up_down_counter("jobs_in_flight", +1)
# ... work ...
add_up_down_counter("jobs_in_flight", -1)

set_metric(name, value, attributes=None) -> None

Set a synchronous gauge — last value wins (queue depth, temperature-style readings), via create_gauge.

from foam_otel import set_metric

set_metric("queue_depth", queue.qsize())

Metrics — attribute discipline. Every distinct attribute combination is a live metric stream. increment_counter("requests", 1, {"user_id": uid}) is a memory leak and a flat dashboard (the SDK caps streams protectively). Keep attribute VALUES low-cardinality: plans, regions, status classes — never ids, raw paths, or emails.

log(severity, body, attributes=None) -> None

Emit a log record through foam's pipeline, trace-correlated to the active span. severity is one of trace|debug|info|warn|warning|error|fatal|critical (case-insensitive; anything unknown lands as INFO — never throws). The body and attributes ride the redaction engine — the always-on credential floor plus your opt-in lists: with no redact_keys everything beyond the floor's names is captured RAW; floor-named and listed keys are masked (dict/list bodies deep-redact by key, URL-query bodies redact the matching keys per key; free text without key shape always rides raw). Both halves are masked at write time, BEFORE emission — so the mask holds even when a foreign OTel SDK owns the logs global and the record rides the foreign pipeline (inert-mode emission; through 2.1.0 only the attributes were write-time masked and a listed key inside the body left the process raw on that path — corrected in 2.1.1).

from foam_otel import log

log("warn", "payment retry scheduled", {"attempt": 2, "gateway": "stripe"})

Most apps don't call log() directly — they use stdlib logging, which foam bridges automatically (see install_root_export below).

redact(value) -> str

Foam's masking on demand, for your OWN sinks (your logger, an audit trail). Same fail-closed engine foam uses internally, with no key context: a scalar string/number gets the tail mask, everything else masks in full. Empty string on hard failure — never the unredacted payload. Never raises.

from foam_otel import redact

my_logger.info(f"token used: {redact(api_token)}")
# redact("sk-live-0123456789abcdef") -> "********cdef"   (len >= 12, tail kept)
# redact("short")                    -> "********"        (len < 12, full mask)
# redact({"nested": "secret"})       -> "********"        (non-scalar, full mask)

Beyond the always-on credential floor (the ONE frozen name denylist, masked unconditionally) the package redacts nothing until you list keys in redact_keys/redact_pii_keys. Capturing other secrets/PII by default is a deliberate posture — you own the privacy/legal decision and opt in per key.

get_tracer(name=None) / get_meter(name=None) / get_logger(name=None)

The raw OTel Tracer / Meter / Logger on whatever pipeline owns the process globals (foam's when foam owns the signal; a foreign SDK's in inert mode). Everything the helper set doesn't wrap — span links, explicit span kinds, observable/async instruments, batch log emission — is reachable here; this is what keeps the helper set small and CLOSED. Default instrumentation scope is foam-otel (the package identity, matching the JS core's @foam-ai/otel; versions ≤ 1.1.x used app.custom — see the migration notes below). Before init they return safe no-op objects. NOTE: a span you start_span() here is yours to end() — a never-ended span leaks; the span() helper cannot leak by construction.

from foam_otel import get_tracer
from opentelemetry.trace import SpanKind

tracer = get_tracer("worker")
with tracer.start_as_current_span("drain-batch", kind=SpanKind.CLIENT, links=links) as s:
    drain()   # explicit kind + links — not wrapped by span(), reached via the passthrough

from foam_otel import get_meter
from opentelemetry.metrics import Observation

meter = get_meter("inventory")
def observe(options):
    return [Observation(current_stock())]
meter.create_observable_gauge("stock_level", callbacks=[observe])  # async instrument

install_root_export() -> bool

Attach a plain OTel LoggingHandler to the ROOT logger so every stdlib logger that propagates (the normal case) exports to foam, trace-correlated. init() calls this automatically when foam owns the logs signal — you rarely call it yourself. Idempotent: it stands down if any OTel export handler is already on root, and is RE-callable after logging.basicConfig(force=True) wipes root handlers. Returns True when a foam (or existing OTel) handler covers root, False when foam isn't active or owning logs. Fail-open.

import logging
from foam_otel import install_root_export

logging.basicConfig(force=True)   # your app resets root handlers, wiping foam's
install_root_export()             # re-attach foam's root export

export_handler(record_filter=None, transform=None)

Build a policy-drivable OTel export handler for loggers you attach it to YOURSELF — the path for libraries that set propagate=False and never reach the root handler. record_filter(record) -> bool gates each record; transform(copy) may scrub a COPY of the record (return None to drop it, so other handlers on that logger still see the original). Returns None when foam isn't active or doesn't own logs.

import logging
from foam_otel import export_handler

def only_warnings(record):
    return record.levelno >= logging.WARNING

handler = export_handler(record_filter=only_warnings)

attach_to_non_propagating(handler, *, skip_prefixes=("foam_otel", "opentelemetry")) -> list

Attach handler to every ALREADY-REGISTERED logger with propagate=False (libraries that detach from root, so the root export never sees them). Idempotent per logger by a marker attribute (not class identity), so a --reload re-import can't double-attach. Propagating loggers are never touched. Returns the list of logger names it attached to. Re-call after importing libraries that configure their loggers late.

from foam_otel import export_handler, attach_to_non_propagating

handler = export_handler()
if handler is not None:
    attached = attach_to_non_propagating(handler)   # e.g. ["browser_use", "some.detached.logger"]

Failure modes — what happens, and where the warning appears

All foam warnings are single-line, prefixed [foam], emitted on the foam_otel stdlib logger (visible on the root handler / your logging config).

Situation	Behavior
Missing/blank token while enabled=True (kill switch unset)	init() raises ValueError at boot — fail in CI, not dark in prod.
Blank/non-string name or environment	init() raises ValueError, naming the option.
Non-bool enabled	init() raises TypeError.
enabled=False	Fully inert and SILENT. No SDK import, no providers, no network, token never touched. Helpers no-op.
OTEL_SDK_DISABLED=true	Same inert posture, but LOUD: one [foam] warning that the kill switch superseded enabled=True.
environment outside {production, staging, development, test}	Exports verbatim, plus one [foam] warning (typo guard).
version omitted	Exports without service.version, plus one [foam] warning.
A foreign OTel SDK owns a signal's global	init() registers nothing there and warns ONCE per claimed signal naming the owner (see the coexistence guide); free signals still register to foam.
Export endpoint overridden via OTEL_EXPORTER_OTLP_ENDPOINT	One [foam] warning per signal naming the destination.
A broken exporter / instrumentation at runtime	That signal is disabled with a [foam] warning; the host app never crashes (fail-open per signal). Runtime detail is visible under diagnostics=True.
init() called twice	Second call warns [foam] init called twice … and returns the current ownership; the first init stands.

Coexistence guide — when foam is not alone

init() attempts to register foam's provider into each of the three global provider slots (traces, metrics, logs) — OTel's set-once, first-wins globals — and reads the outcome back: every FREE slot becomes foam's, and every CLAIMED slot warns once naming the owner (foam never pre-reads a slot to predict, and never displaces an owner). Three situations (rule 18):

• A — a proprietary APM agent beside foam (a vendor agent running its own private pipeline, no OTel globals). Disjoint pipelines; both run. Foam never touches the agent. Your job: add the agent's intake host to ignored_outbound_hosts (scenario 2) so foam never traces the agent's own export traffic. (If a modern agent ALSO registers an OTel global, that signal becomes situation B — the classifier decides per signal.)

• B — a foreign OTel SDK owns a signal's global. Foam's init() is dark for that signal — not degraded: NOTHING for that signal reaches foam through door 1. For each claimed signal, init() emits exactly one warning:

  [foam] traces is owned by <Owner> — foam is dark for traces; helper telemetry for it rides the foreign SDK to ITS backend. To also send that pipeline to foam, add one line in THEIR setup: create_foam_ingest_span_processor. See README "Door 2".

  What it means: door 1 registered nothing for that signal, and the fix is door 2 — the shipped create_foam_ingest_* entries ("Door 2 — the ingest entries" below): one line in THEIR setup taps their pipeline additively, so their export keeps working AND foam receives a masked, tier-marked copy. Your other options stay valid: remove the foreign SDK so foam owns the signal (scenario 1), or feed foam server-side from their collector. Your helpers keep working either way, but helper telemetry for the claimed signal rides the foreign provider and carries ITS resource identity, not foam's. (The owner name is read from the provider's class, MODULE-QUALIFIED — e.g. opentelemetry.sdk.trace.TracerProvider vs a vendor's ddtrace.opentelemetry.TracerProvider — because the bare class name is generic; a registered provider doesn't announce its vendor, so "an unknown OTel SDK" is the honest fallback.)

• C — a scoped tenant rides foam's pipeline (an LLM-eval / AI-observability SDK that attaches a processor/reader). Pass its CONSTRUCTED instance via additional_span_processors / additional_log_record_processors / additional_metric_readers (scenario 3). Guarantees: strictly ADDITIVE (foam's export is byte-identical with or without it), FAULT-ISOLATED (a throwing span/log instance is DISABLED loudly, foam unaffected), and ALREADY-MASKED (the redaction stage runs first, so the tenant only ever sees the masked view). REQUIRED: the tenant exporter's host in ignored_outbound_hosts — otherwise foam traces the tenant's export calls, which the tenant re-exports (a loop in the tenant's pipeline). A metric READER is different in kind: it collects on its own schedule/thread, so its collect loop is your code, not covered by foam's guard.

Claim severity is not uniform. A claimed MeterProvider costs foam little — RED metrics derive server-side from foam-owned spans; only custom metric-helper data diverts. A claimed TracerProvider costs the spans AND everything derived from them. Prioritize freeing traces first.

Late arrival. A foreign SDK can register AFTER foam boots. The same once-per-signal warning fires on the next helper use when the cached provider identity no longer matches the global — a coexistence warning is not only a boot-time event.

Door 2 — the ingest entries

When a foreign OTel SDK owns a signal (situation B), init() cannot serve it — door 2 does: one constructed instance per claimed signal, added by you — ONE LINE — to the CUSTOMER'S OWN OTel setup. Their pipeline keeps working exactly as before, and now ALSO sends foam a copy. The package provides the tap; you install the tap (consent is the line in their config — that is also the off-switch: there is no enabled param; to stop the tap, delete the line).

# signatures — identical shape across all three entries (keyword-only)
from foam_otel import (
    create_foam_ingest_span_processor,        # -> real SpanProcessor
    create_foam_ingest_log_record_processor,  # -> real LogRecordProcessor
    create_foam_ingest_metric_reader,         # -> real PeriodicExportingMetricReader
)

tap = create_foam_ingest_span_processor(
    token=os.environ["FOAM_OTEL_TOKEN"],  # REQUIRED — Bearer auth, same as init
    environment=APP_ENV,                  # REQUIRED — -> deployment.environment.name,
                                          #   stamped AT EXPORT TIME on foam's copy
    redact_keys=["internal_ref"],         # optional — OPT-IN tail-mask, additive on
                                          #   top of the always-on credential floor
    redact_pii_keys=["customer_email"],   # optional — YOUR PII fields, full [REDACTED]
    redact={"detect": ["email"]},         # optional — the grouped object (2.4.0):
                                          #   secrets/pii/detect, unions with the
                                          #   flat aliases above; detect = the
                                          #   opt-in PII tier, typed placeholders
    diagnostics=False,                    # optional — tap-scoped [foam] narration
)

Validation is loud and at construction (the same bar as init()): a blank/missing token or environment raises a [foam]-prefixed ValueError at boot — never a dark tap. The ONE exception is the operator kill switch: under OTEL_SDK_DISABLED=true each entry returns a truly inert instance of the correct type (zero threads, zero exporters, zero network — your setup file keeps one shape) with one loud warning, and the token is NOT validated — the operator's off wins and must never crash your boot. The switch is read ONCE at construction; unset it and restart to re-enable.

# unhappy paths, shown:
create_foam_ingest_span_processor(token="", environment="production")
# -> ValueError: [foam] create_foam_ingest_span_processor: token is required
#    and must be a non-empty string
#
# OTEL_SDK_DISABLED=true ->
# [foam] OTEL_SDK_DISABLED=true — this foam ingest entry is disabled by the
# operator kill switch and will send nothing. Unset the variable and restart
# to re-enable.

Per-signal recipes

Traces and logs — post-construction attach (Python's providers support it):

their_tracer_provider.add_span_processor(create_foam_ingest_span_processor(
    token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV))
their_logger_provider.add_log_record_processor(create_foam_ingest_log_record_processor(
    token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV))

Metrics — constructor placement is THE recipe. Foam's reader collects on its own schedule beside theirs (multiple readers per MeterProvider is spec-legal); their Views shape what foam receives — an external-tier caveat, never fought:

their_meter_provider = MeterProvider(     # THEIR constructor line
    resource=their_resource,
    metric_readers=[their_reader, create_foam_ingest_metric_reader(
        token=os.environ["FOAM_OTEL_TOKEN"], environment=APP_ENV)],
)

MeterProvider.add_metric_reader() exists at the pinned floor and works as a FALLBACK when their MeterProvider(...) line is out of reach — but it is lifecycle-incomplete upstream: the provider's force_flush() and shutdown() walk only constructor-time readers, so a fallback-attached reader never receives the provider's final flush (GOTCHAS P16). Foam mitigates with an idempotent shutdown() plus its own atexit hook, which delivers the final collection at normal interpreter exit. Serverless caveat: constructor placement is REQUIRED on serverless — the platform freezes the process, so atexit never fires and a fallback-attached reader's last window is lost. One more invariant: every create_foam_ingest_metric_reader() call returns a fresh instance and each instance goes to exactly ONE provider — the SDK itself raises in THEIR constructor if one reader is registered twice. Never share one.

Door 2 — the required loop step

The tap's exports to foam are outbound HTTP calls THEIR instrumentation will span, and their pipeline hands those spans back to the tap — a feedback loop in THEIR pipeline that foam's own rule-24 guard cannot reach. Three layers stop it:

1. Foam wraps every tap export in the SDK's suppression context — official contrib instrumentations (requests/urllib/urllib3/httpx/aiohttp) bail out.
2. REQUIRED — your step: instrumentation that ignores the suppression key (vendor SDKs, mesh/eBPF layers) still spans foam's endpoint. Add the foam export host — otel.api.foam.ai, or the active OTEL_EXPORTER_OTLP_ENDPOINT override host — to THEIR client-side exclusions. In contrib-Python that is the PER-CLIENT vars (OTEL_PYTHON_REQUESTS_EXCLUDED_URLS, OTEL_PYTHON_URLLIB_EXCLUDED_URLS, OTEL_PYTHON_URLLIB3_EXCLUDED_URLS, OTEL_PYTHON_HTTPX_EXCLUDED_URLS, OTEL_PYTHON_AIOHTTP_CLIENT_EXCLUDED_URLS) — never the global OTEL_PYTHON_EXCLUDED_URLS, which their SERVER instrumentation also honors and which would drop their inbound spans when the collector is on localhost.
3. The echo filter, foam's structural backstop: if both layers are defeated, a client-kind span describing foam's own export call is withheld from FOAM'S copy only (their pipeline keeps it), with one loud warning naming this missing step. The loop is bounded even with the README step forgotten — but the step stays REQUIRED: the filter withholds foam's echo, it cannot stop their pipeline from paying for those spans.

What the tap does — and never does

• Additive reader only: their data, resource, and export are NEVER mutated. Foam's labels merge at serialization onto foam's copy; your pipeline sees your data byte-identical. The tap masks foam's copy only — your pipeline still carries what it captured.
• Identity: foam's copy carries THEIR resource identity (service.name, service.version, service.instance.id untouched) plus exactly four stamp keys: deployment.environment.name (the argument), foam.ingest.tier: external, telemetry.distro.name: foam, telemetry.distro.version.
• Masking: every tap unconditionally enforces the always-on credential floor on foam's copies — zero tap-level configuration, no off switch. Beyond it, redaction is opt-in — with no redact_keys/redact_pii_keys the tap masks nothing else and forwards every other value RAW. Floor-named and listed keys are masked on foam's copy before it leaves — attributes, event/link attributes, metric datapoint attributes and exemplar filtered_attributes, and log attributes
  • body (per key). Your own pipeline is NEVER touched — it keeps its raw values. Fail-closed on foam's side: an item whose masked copy cannot be built is dropped from foam's batch with a loud counter, never exported raw.
• Never-throw: after construction, nothing in the tap can break your pipeline or your app — failures warn [foam] and foam's copy alone is lost.
• Self-ingest stand-down: wiring a tap onto foam's OWN door-1 pipeline is a wiring bug — the tap warns once and stands down (door 1 already exports that data; nothing is lost, nothing is doubled), and tears its own machinery down.
• Same-signal double-claim: if init() already exports a signal AND a tap sits on a separate foreign provider, both keep flowing (different pipelines; the tier marker distinguishes them) with one loud warning — if unintended, remove one.
• Transport: OTLP/HTTP protobuf to foam's fleet /v1/* endpoints with Authorization: Bearer <token> — same pins, same operator-override env var (honored + loudly warned) as door 1.
• Forbidden surface: no endpoint, cadence, sampling, temporality, or Views options — deliberately absent, same as init().
• Fork safety: taps built pre-fork self-heal in workers via the SDK's at-fork hooks (regression-pinned). Door-1's post-fork re-init does not govern the tap — door-2 identity is theirs, there is no foam instance id to re-mint.

Recipes

• Per-environment enable (the one-liner): enabled=os.environ.get("APP_ENV") not in ("test", "ci") — scenario 1. Other valid shapes: enabled=os.environ.get("APP_ENV") in ("production", "staging") (explicit allowlist), enabled=os.environ.get("APP_TELEMETRY") == "on" (your own app-named flag, compared EXPLICITLY — never bool(os.environ.get(...)), which is True for "false" and "0"), enabled=True (export everywhere, honestly labeled). Foam itself defines no on/off env flag — the only foam env key that exists is the token you wire yourself.

• Serverless flush: scenario 7 — try: … finally: flush(). The platform freezes the process before any batch timer fires; flush() is the fix. Import cost is lazy: importing foam_otel costs only the OTel API; the SDK and exporters load inside init() (and never with enabled=False).

• Uvicorn / ASGI graceful shutdown — wire shutdown() at lifespan shutdown. Uvicorn re-raises SIGTERM after its graceful stop, so the process dies BY SIGNAL and the SDK's atexit backstop never runs — telemetry buffered since the last batch timer is silently lost. The signal.signal(SIGTERM, ...) recipe above CANNOT be used under uvicorn (uvicorn owns the SIGTERM handler; replacing it disables graceful HTTP shutdown). The reliable path is the ASGI lifespan hook — proven by the conformance app under 60s batch delays:

# uvicorn/ASGI graceful shutdown: flush buffered telemetry before the process dies
from contextlib import asynccontextmanager
from fastapi import FastAPI
import foam_otel

@asynccontextmanager
async def lifespan(app):
    yield
    foam_otel.shutdown()   # flushes every buffered span/log/metric

app = FastAPI(lifespan=lifespan)

• Gunicorn / uvicorn / any pre-fork server — foam re-inits automatically. Foam registers a post-fork hook (os.register_at_fork), so under gunicorn --preload or a uvicorn worker fork each child re-inits automatically and re-mints a fresh service.instance.id — N workers report as N processes, not one. You do NOT need a manual post_fork hook or a shutdown()+init() dance. (A bare init() inside a hand-written post_fork would be a no-op — one init per process — but the automatic hook makes that unnecessary. On platforms without os.fork, the hook is skipped.) See GOTCHAS "Fork / pre-fork servers".

• gevent / eventlet workers: call monkey.patch_all() as line 1 of your entrypoint, BEFORE import foam_otel — patching after threading/ssl/socket are imported breaks context propagation and the export threads. See GOTCHAS.

• What foam deliberately does NOT automate: reading your git SHA (pass version yourself), choosing your PII fields (pass redact_pii_keys yourself), sampling (none exists), and named logger bridges beyond the generic stdlib bridge (structlog/loguru arrive via the update path — log() and the root-export bridge cover the mechanism today). LLM instrumentation IS automated for the floor providers: the default bundle carries openai-v2 (chat/embeddings), the Anthropic instrumentor, and the google-genai instrumentor (Gemini), and foam's own gap-filler covers the OpenAI Responses API (sync + async) — all with prompt/response content captured raw (see the dedicated section below). LLM libraries beyond the floor are covered by installing their official instrumentor (auto-activated) or by tier-3 spans through the helpers/passthroughs.

LLM instrumentation — content captured, raw

The LLM floor ships in the default bundle (provider parity: OpenAI + Anthropic + Gemini): opentelemetry-instrumentation-openai-v2 (chat/embeddings — foam switches its content capture ON at activation), the Anthropic instrumentor (content on by its default), and the official contrib google-genai instrumentor (Gemini — foam sets its content mode to SPAN_AND_EVENT at activation, so raw prompt/response content lands on the gen_ai span attributes AND the inference-details event; the same OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT var speaks a different grammar per reader, and foam writes each reader its own spelling at its instrument() moment), all activated by the sweep on presence.

Gemini provider value, canonicalized (2.1.1). The pinned google-genai instrumentor still derives gen_ai.provider.name as the DEPRECATED semconv values gemini / vertex_ai (semconv 0.65b0: "Replaced by gcp.gemini/gcp.vertex_ai"), while the JS, Ruby, and Java cores emit the current gcp.* well-known values — so a fleet dashboard filtering gen_ai.provider.name="gcp.gemini" silently dropped every Python service. Foam canonicalizes those two VALUES on every export path (span attributes, the inference-details event, metric datapoints, door-2 copies); the attribute NAME is untouched, no other value is rewritten, and the mapping drops out when the upstream fix lands and is re-pinned (GOTCHAS P23). Dashboards keyed on the deprecated gemini/vertex_ai values must re-key to gcp.gemini / gcp.vertex_ai (see the migration notes).

Foam additionally ships its own gap-filler for the OpenAI Responses API (what reasoning-model services call via client.responses.create(...), sync or awaited), which the official openai-v2 window does not cover. Know exactly what the gap-filler does in your process:

• What it patches: Responses.create AND AsyncResponses.create, via wrapt wrappers. Each target stands DOWN independently (no wrap) when openai is absent, or when upstream (openai-v2 or anyone) already wrapped that method — foam never double-instruments.
• What it emits: ONE CLIENT span per logical call, named chat <model>, carrying the gen_ai activity metadata — gen_ai.operation.name, gen_ai.provider.name: openai, gen_ai.request.model, gen_ai.response.model, gen_ai.response.id, gen_ai.usage.input_tokens / output_tokens — AND the content, raw (2026-07-26 ruling: LLM content is captured like any other signal, never gated or dropped), in the fleet-pinned structured shape (rule 8c GenAI-uniformity: ONE content dialect fleet-wide): gen_ai.input.messages / gen_ai.output.messages carry the semconv chat-message JSON — [{"role": ..., "parts": [{"content": ..., "type": "text"}]}], output messages with finish_reason observed from the response status — exactly what the bundled openai-v2 chat / Anthropic / Gemini paths and the other language cores emit; the content TEXT is verbatim, a bare-string input wraps as one user message, a message-list input maps role/content → role/parts (2.1.0 emitted bare strings on these keys — a second dialect, corrected in 2.1.1; see the migration notes). instructions land verbatim on gen_ai.system_instructions. Content VALUES are never pattern-scanned by the credential floor (it masks names, not content); content is masked ONLY when a credential-floor NAME appears as a field key inside it, or when you list a matching key in redact_keys/redact_pii_keys — same rule as everything else. On a failed call the span additionally carries the observed error.type and — when the provider error exposes one — http.response.status_code. A successful call carries NO status attribute (nothing is observed, so nothing is stated; versions ≤ 1.1.x emitted gen_ai.response.status and a fabricated http.status_code: 200 — see the migration notes below). The HTTP transport span underneath is suppressed so one call yields one span.
• Streaming: stream=True calls are captured too (since 2.1.1; 2.1.0 ended the span at connection-open and silently lost output/usage/id/model). The span stays open and foam wraps the returned Stream/AsyncStream with a transparent passthrough that only OBSERVES events as YOUR code pulls them — foam never drains, buffers, or consumes the stream. The terminal response.completed/response.incomplete/response.failed event lands the full response shape on the span, which ends when the stream is exhausted, closed, or exits its with block (abandon a stream without closing it and the span is never exported). The responses.stream() and responses.parse() helper APIs are NOT wrapped — a documented boundary of the gap-filler, not silent partial capture.
• Turning it off: there is no env lever — OTEL_PYTHON_DISABLED_INSTRUMENTATIONS is inert (env table above; versions ≤ 1.1.x honored the name openai_responses there). Restricting content for a compliance-sensitive deployment is the FDE clamp path: list the content keys in redact_keys/redact_pii_keys. Errors inside the wrapper never break your call (fail-open).
• History: 1.1.0 briefly emitted gen_ai.system (renamed to gen_ai.provider.name in 1.1.1 — dashboards keyed on gen_ai.system must re-key). 1.1.1 also removed content capture; that removal was OVERRULED by the 2026-07-26 capture ruling and content returned — additively, on the current-semconv content attributes above — in 2.1.0. The attribute-key set is frozen additively by the golden fixture — keys may be added within a major, never removed or renamed.
• Picking ecosystems: contrib, OpenLLMetry, and openinference all register opentelemetry_instrumentor entry points and sometimes wrap the SAME client under different package names; the double-instrument guard is per-instrumentor, not cross-ecosystem. Install ONE ecosystem per provider (the bundled floor already covers OpenAI + Anthropic + Gemini).

Thrown-then-mapped exceptions (FastAPI/Starlette). Also since this line: an exception your @app.exception_handler maps to a clean 4xx is handled BELOW the OTel middleware, so the official instrumentation alone would export the request span with no exception event. Foam records the exception on the server span at handler-lookup time — a throw is a genuine fault signal even when mapped (contract §6). Control-flow exceptions (HTTPException — every 404 — and request-validation errors) are never recorded, and unmapped exceptions are left to the official instrumentation (no double recording). Like the Responses gap-filler, it has no env disable lever (versions ≤ 1.1.x honored the name mapped_exceptions in the now-inert OTEL_PYTHON_DISABLED_INSTRUMENTATIONS).

Supported versions

Surface	Supported	Reason
Python	>= 3.10	init() uses importlib.metadata.entry_points(group=...), whose keyword form landed in 3.10.
opentelemetry-api / opentelemetry-sdk	>= 1.44, < 2	The logs SDK's processor contract is on_emit(ReadWriteLogRecord) at this line (the older emit(LogData) shape is a different, incompatible signature), and foam's redaction rides that hook. API and SDK move in lockstep. The < 2 ceiling is deliberate (rule 40): a future OTel 2.0 is NOT supported until the matrix re-proves it — an open floor would let a breaking major resolve into your install as a silent no-op.
opentelemetry-exporter-otlp-proto-http	>= 1.44, < 2	The OTLP-over-HTTP transport foam exports on. Same deliberate ceiling as the API/SDK.
opentelemetry-instrumentation	>= 0.65b0, < 0.66b0	The entry-point activation base foam sweeps — pre-1.0, so capped to the PROVEN minor (rule 40: 0.x lines break between minors); widening is a deliberate, re-proven bump.
wrapt	>= 1.0, < 3	Direct dependency of the gap-fillers' wrap_function_wrapper (declared, never assumed transitively).
Floor instrumentations (-fastapi, -django, -flask, -requests, -urllib3, -httpx, -psycopg, -asyncpg, -mysql, -pymysql, -redis, -pymongo, -celery, -system-metrics, -openai-v2, -anthropic, -google-genai)	>= 0.65b0, < 0.66b0 (contrib line; openai-v2 >= 2.4b0, < 2.5b0 — every earlier 2.x is dead against this core: 2.0/2.1 import the removed opentelemetry._events, 2.2/2.3 call wrapt with the renamed module= keyword; anthropic >= 0.62.1, < 0.63, google-genai >= 1.0b1, < 1.1b0)	The universal floor, BUNDLED in the default install (2.1.0): web + outbound HTTP + pg/mysql/redis/mongo + queue + system-metrics + LLM (OpenAI + Anthropic + Gemini; activity AND content, raw). Auto-activated at init(), presence-checked; FastAPI + the LLM content path are proven end-to-end by the conformance app; django + flask carry in-suite test-client wire proofs (SERVER span, http.route, raw headers) and system-metrics is wire-pinned (a system.* + process.* datapoint after init, both ownership postures) since 2.1.1. Same rule-40 tight caps as the instrumentation base.

The EXACT versions everything above was proven against (the rule-44 known-good set) are committed in constraints.txt; install with pip install -c constraints.txt foam-otel[...] to reproduce the proven environment byte-for-byte.

Foam builds on the OpenTelemetry project — the Python API/SDK, the OTLP exporters, and the contrib instrumentations — all Apache-2.0. Full credits in THIRD-PARTY-NOTICES.

MIGRATION — deliberate wire/surface corrections after 1.1.x (breaking; ships in the next MAJOR)

Three 1.1.x behaviors were corrected because they violated the wire contract (rule 26: the package never invents, renames, or fabricates a wire-visible name) or the closed env-var posture table. Each is a breaking change, shipped deliberately and gated by the golden fixture; check the following before upgrading.

1. OpenAI Responses gap-filler wire names (finding python-otelapi-4). The chat <model> gen_ai CLIENT span no longer carries gen_ai.response.status (an invented name — it exists nowhere in the gen-ai semantic conventions) or http.status_code (deprecated semconv, and on success it was a fabricated constant 200, never read from any response). Now, per the current GenAI span conventions (open-telemetry/semantic-conventions-genai, docs/gen-ai/gen-ai-spans.md): a successful call carries NO status attribute; a failed call carries error.type (the exception's qualified name, as the official openai-v2 instrumentation emits) plus the observed http.response.status_code when the provider error exposes one. Check your side: dashboards, alerts, or queries keyed on gen_ai.response.status or http.status_code on gen_ai spans will go empty — re-key error monitoring to error.type / span status ERROR (+ http.response.status_code where present); "success" is a span with status unset/OK, not http.status_code == 200.
2. OTEL_PYTHON_DISABLED_INSTRUMENTATIONS is now INERT (finding python-surface-6). 1.1.x honored it (skipping listed instrumentations, including the gap-filler names openai_responses / mapped_exceptions). The spec's honored env table is closed, and this lever silently dropped whole instrumentations' data — so it now warns loudly "set but ignored" and disables nothing. Check your side: any deployment setting this var will, after upgrade, emit telemetry from the instrumentations it used to suppress (more spans, possibly new span names/attributes on dashboards and in billing) and log one [foam] warning at boot. Remove the var, or uninstall the instrumentation package you need gone.
3. Helper telemetry scope renamed app.custom → foam-otel (finding python-surface-10). The instrumentation scope on every helper-emitted span, metric, and log is now the package identity (matching the JS core's @foam-ai/otel); app.custom was producer-classification stamp vocabulary the package must not ship (rule 4) — classification is foam's collector's job, server-side. Check your side: queries, dashboards, or collector/backend rules that filter on instrumentation scope app.custom (e.g. otel.library.name/otel.scope.name == "app.custom") must be re-pointed at foam-otel; scope-based routing in a customer-side collector should be updated the same way.

2.1.0 — the universal-floor bundle + raw-capture posture (ADDITIVE minor)

Non-breaking per the Area 0 floor-raise rule (new signals only; nothing already flowing is renamed or reshaped). What upgrading adds:

• The floor is bundled: the default install now pulls the instrumentors for django/flask, psycopg/asyncpg/mysql/pymysql, redis/pymongo, celery, system-metrics, openai-v2, anthropic, and google-genai (Gemini) (fastapi/requests/urllib3/httpx were already reachable via extras). Each activates only where its target library is present — expect NEW spans/metrics (and volume) for targets you have.
• Raw capture on by default: HTTP request/response headers (all), SQL bind params (psycopg), pymongo command bodies, raw redis statements, raw URLs (upstream credential/signature sanitizers disabled), and LLM prompt/response content (openai-v2 content switch ON; google-genai content mode SPAN_AND_EVENT; the Responses gap-filler extracts content; sync Responses.create newly covered). Everything is maskable per key via redact_keys/redact_pii_keys.
• Session stitching: spans and logs emitted under an inbound context whose W3C Baggage carries session.id (what @foam-ai/browser sends alongside traceparent) are stamped with session.id/session.previous_id attributes — no-op when absent, maskable per key like everything else. Your API's CORS allowlist must include baggage alongside traceparent for browsers to deliver it cross-origin.
• New foam-written env vars while active (removed at shutdown): the GenAI content switch + the three header-capture vars (table above).

2.1.1 — GenAI semconv alignment + capture/boot corrections (review findings 2026-07-26)

Fixes from the adversarial fleet review. Two are deliberate wire-value corrections ratified by the rule-8c GenAI-uniformity ruling (the fleet pins ONE GenAI vocabulary and ONE content dialect; Python was the sole deviator on both) — check your side:

1. Gemini gen_ai.provider.name → canonical gcp.* values. Spans, events, and metric datapoints that carried the deprecated gemini / vertex_ai now carry gcp.gemini / gcp.vertex_ai — the values the JS/Ruby/Java cores already emit. Re-key any dashboard/alert filtering the deprecated spellings.
2. OpenAI Responses gap-filler content is now the fleet-pinned structured dialect. gen_ai.input.messages / gen_ai.output.messages carry the semconv role/parts chat-message JSON (content text verbatim inside) instead of 2.1.0's bare strings/request-schema JSON — the same shape the openai-v2 chat, Anthropic, and Gemini paths emit. Parsers that assumed the bare string must parse the message list; the golden fixture now freezes the three content keys AND the structured dialect.

And the behavior fixes:

3. Streamed Responses calls captured (stream=True): output content, usage, response id/model, and true latency now land on the span via an observe-only stream passthrough — 2.1.0 ended the span at connection-open with none of them. responses.stream()/parse() remain out of the gap-filler's scope (documented boundary).
4. Bare-string redact_keys/redact_pii_keys raises at boot (rule 10). 2.1.0 swallowed the designed rejection inside the fail-open init path and booted the service fully DARK with only a warning; the typo now fails loudly at init(), exactly like door 2.
5. log() bodies are masked at write time (rule 14): with a foreign SDK owning logs (inert-mode emission), a customer-listed key inside a log() body used to leave the process RAW into the foreign backend while the same key in attributes was masked. Both halves now mask before emission.
6. Scenario 5 keeps the host-metrics floor: when a foreign SDK owns traces and foam owns metrics, the metrics-only floor item (system-metrics) now activates instead of dying with the trace sweep — and the system./process. floor is wire-pinned by tests on both postures.
7. django/flask wire proofs added (test-client SERVER-span tests with http.route + raw headers), closing the untested half of the bundled web-framework floor.

2.2.0 — the always-on credential floor (fleet ruling 2026-07-26; MINOR, security-motivated wire change)

2.x (minor): always-on credential masking. As of this version foam masks, by default and in every signal, the VALUES of a fixed list of credential/secret header and field NAMES (authorization, cookie, set-cookie, proxy-authorization, x-api-key, x-auth-token, www-authenticate, and the 52-name key list in contract/credential-denylist.json — Sentry-parity plus foam's documented roots) to the literal [REDACTED]. Matching is exact name-equality, case-insensitive, dash/underscore-insensitive — never substring: authorization_url is untouched. Everything else still exports RAW exactly as before; redact_keys/redact_pii_keys are unchanged and additive. There is no off switch — if a dashboard keyed off a raw credential value (it should not have), it will now see [REDACTED].

Details in the default credential denylist section above. What lands in this version, concretely:

• The frozen constants CREDENTIAL_HEADER_DENYLIST / CREDENTIAL_KEY_DENYLIST (_constants.py, gate-checked bit-for-bit against the fleet fixture) and a floor predicate inside the ONE central engine (_redaction.py) — door 1, all three door-2 taps, and the tenant seams inherit it with zero wiring.
• OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SANITIZE_FIELDS is now WRITTEN as the seven headers, anchored (^authorization$,…), instead of pinned empty — capture-time defense-in-depth on the ASGI/WSGI header path (the engine remains authoritative).
• No API surface change: no option added or removed, no semantics change to redact_keys/redact_pii_keys beyond the floor running first (a floor name you also list stays fully [REDACTED], never a tail). enabled=False full-inert semantics unchanged.

2.4.x — outbound CLIENT header capture (ADDITIVE minor)

Outbound HTTP client spans (requests, urllib3, httpx — the bundled floor) now carry request AND response headers by default, exactly like the inbound server spans have since 2.1.0. See HTTP header capture. What lands, concretely:

• Foam now also writes the standard OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_CLIENT_{REQUEST,RESPONSE} vars (default .*; operator-respecting like the SERVER pair — your narrower pre-init allowlist is honored, never widened; removed at shutdown()). New signal only: no existing attribute is renamed or reshaped.
• The credential floor masks the seven credential headers on the new client attributes automatically ([REDACTED], both directions) — the same frozen list, the same central engine, zero config.
• Beyond-floor official client instrumentations that honor the same vars (aiohttp-client, stdlib urllib) inherit the posture when you install them. gRPC metadata stays out of scope (no standard upstream mechanism — see the deliberately absent knobs section).
• Expect NEW http.request.header.* / http.response.header.* attributes (and volume) on CLIENT spans; clamp per key with redact_keys, or set your own narrower ..._CLIENT_REQUEST/..._CLIENT_RESPONSE allowlist before init().

2.8.0 — the before_send export hook (ADDITIVE minor)

• New init option before_send — one callable, or a sequence run as a pipeline, called with every span and log record at the export boundary (after batching, before serialization). Return the record (in-place mutation supported) or a same-type replacement to keep it; None to drop it; a raise or foreign return drops that record loudly and fail-closed (batch reports failure; healthy siblings still export). Fleet parity with the Ruby core's 1.8.0 before_send: (Sentry beforeSend semantics); see "The before_send hook".
• Ordering divergence, documented: this package's primary redaction runs at the PROCESSOR seam (pre-batch), so hooks run AFTER it and see the masked view; the hook wrappers RE-mask everything the hooks return (attributes and log bodies) before the inner exporter, so the fleet invariant — a hook can never widen what ships — holds identically. Tenant processors observe records BEFORE before_send here (they sit at on_end/on_emit), unlike Ruby where they sit at the exporter.
• Bookkeeping: SDK-limit dropped-attribute counts are captured before the hooks and restored on survivors, so hook adds/deletes never corrupt the wire's dropped_attributes_count (GOTCHAS P28).
• Zero default-path change: with before_send absent no hook stage is wired at all — the wire is byte-identical to 2.7.0. Metrics and door-2 taps are deliberately not routed through the hook.

License

Proprietary — see LICENSE. Use is permitted only by Foam and customers with an active Foam agreement. Third-party components are credited in THIRD-PARTY-NOTICES (OpenTelemetry is Apache-2.0).