opexia-trace 0.1.0a17

Pluggable reasoning-trace observability for LLM applications

opexia-trace

Pluggable reasoning-trace observability for LLM applications.

opexia-trace is the Python client SDK for the OpexIA Observability Layer. It captures reasoning-altitude spans — the decisions an agent made, the sources it used, what each step cost, and the inputs to a reliability score — and ships them to an OpexIA backend over OTLP. Unlike raw LLM logging, the unit of observation is the reasoning step, not the HTTP call.

The package now includes more than tracing. One pip install opexia-trace also gives you three tools that ship with the SDK — all covered below:

• opexia shipcheck — a zero-LLM pre-merge CI gate that catches prompt / model / cost / cache regressions from the traces you already sent.
• opexia audit — a local, zero-egress security audit + map of an agentic app (MCP servers, agents, tools, hooks), graded against the NSA's MCP security guidance.
• import pxcore (+ pxcore-mcp / pxcore-proxy) — deterministic image-as-context token compression: render dense reference context as images the model reads with native vision, keep exact content as text.

See What's new (0.1.0a13 → a16) and the Also in this package section.

Not a Python shop? OpexIA's ingest speaks plain OTLP/HTTP, so any language with an OpenTelemetry SDK works today — including Next.js / TypeScript and Vercel AI SDK apps. See Use from TypeScript / Next.js below. You don't need to spin up Python just to ship spans to us.

What OpexIA does with your spans (the intelligence layer)

opexia-trace is not a logging service. It is the client half of an AI-pipeline reconstruction engine. Once your spans land in our ingest, reconstruction engines run server-side on every batch:

Engine	Reads from your spans	Produces
Decomposition	parent-child tree + reasoning_role	how the query was broken down + decomposition quality score
Decision Trace	opexia.decision.* attributes per step	reasoning DAG — which branches were taken vs abandoned, and why
Sources	opexia.sources.used / consulted / dropped URL lists	sources matrix + live Exa-backed web verification + per-domain authority prior that learns over time per workspace
Reliability v2	LLM completions + cited sources	per-claim hallucination score
Correlation	model, tokens, cost, pipeline shape	over-provisioning signals — where a premium model ran a simple query, and the measured saving of a cheaper one
Savings Advisor	prompt/context token structure	prompt-caching and context-optimization recommendations (advice only; never actuates)
Alignment	completions + your workspace knowledge base	whether answers align with your own source-of-truth documents
External Cross-Check	extracted claims + live web	independent verification of stage-1 claims

Some engines are per-workspace toggles and some ship dark; ask your workspace owner what is enabled.

You do not call any "compute" endpoint. Spans go in → artifacts appear in the Read API seconds later. The semantic conventions on each span are what unlock this — a raw "LLM call, 1.2s, 800 tokens" span gives you cost and latency only; the rest of the magic needs the attributes documented in Semantic conventions below.

Install

pip install opexia-trace

Already have it? Upgrade to 0.1.0a16 or later — you get Ship Check, the agent audit, and pxcore, and every release before 0.1.0a12 silently discards its auto-instrumented spans (see What's new):

pip install --upgrade --pre opexia-trace

The --pre is required. Every published version is a pre-release (0.1.0a1 … 0.1.0a12) and pip will not upgrade between pre-releases without it. To see what you actually have versus what exists:

pip show opexia-trace                     # installed
pip index versions --pre opexia-trace     # available

What's new in 0.1.0a13 → a16

The SDK grew three capabilities beyond tracing, all shipped in this one package:

• 0.1.0a13 — Ship Check. opexia shipcheck: a pre-merge CI gate that makes no LLM call and never re-runs your agent. It projects a prompt/model/config change against the traces you already sent — cost per call / trace / month, prompt-cache safety, structure regressions — and a second 100%-local gate flags Claude/agent artifacts (CLAUDE.md, .mcp.json, …) shipping inside your production build. Also: prompt versioning + a Prompt Health Score, and the opexia.prompt_id / opexia.prompt_version envelope keys for exact baseline matching. See Also in this package.
• 0.1.0a14 — Agent Security Audit. opexia audit: local and zero-egress (no network, no LLM, nothing it finds leaves the machine). It maps an agentic app's topology — agents, subagents, MCP servers, tools, hooks — plus a granular reachability layer (capabilities, datastores, endpoints, secrets-by-name), and audits it against the NSA MCP Security CSI: tool-description injection (incl. hidden unicode), blanket OAuth scopes, unpinned npx -y boot-time code execution, tool-name collisions, cleartext credentials, and secret→internet exfiltration paths. Its headline is a committed .opexia/agentmap.lock that turns a silent capability change ("rug pull") into a reviewable PR diff.
• 0.1.0a15–a16 — pxcore token compression. import pxcore, plus the pxcore-mcp and pxcore-proxy commands. Deterministic image-as-context compression, folded into this distribution (no separate install). See pxcore below.

Also in this package — shipcheck, audit, pxcore

These ship with opexia-trace; nothing extra to install.

Ship Check — opexia shipcheck (zero-LLM PR gate)

pip install "opexia-trace[shipcheck]"     # the extra is just PyYAML, to read the policy file
opexia shipcheck --help

Declare a policy in .opexia/shipcheck.yml (prompt files, cost thresholds), then run opexia shipcheck in CI. Exit 0 = pass, 1 = a gate failed the policy, 2 = the check could not run (a backend outage is never a silent pass). It talks to your workspace with OPEXIA_API_KEY; the artifact-leak gate is fully local and needs no key (opexia shipcheck --skip-prompts). Full guide: docs/shipcheck.md.

Agent Security Audit — opexia audit (local, zero-egress)

opexia audit --map agentmap.html          # writes a self-contained, offline HTML map
opexia audit --approve-all                # bless the current agent map as the baseline

Runs automatically as a third gate of opexia shipcheck too (WARN by default). No API key, no network. Full spec: docs/agent-map-security-audit.md.

pxcore — token compression (import pxcore)

Cut input tokens by rendering dense, reference context as images the model reads with its native vision, while keeping exact content (ids, paths, hashes, code you edit) as text. No LLM in the compression path; nothing leaves the machine. It is default-off per model and images only where a calibration battery has proven that model reads imaged text back reliably — everything else stays text (safe).

Python — library (compress context you assembled, or give your agent tools):

import pxcore
from pxcore.calibration import load_profile

profile = load_profile("claude-fable-5")
blocks  = pxcore.to_anthropic(big_reference_text, profile)   # image+factsheet, or text
new_body, saved = pxcore.compress_anthropic(anthropic_request_body)   # whole-request

from pxcore import agent_tools                                # ready-to-register agent tools
result = agent_tools.read("large/log.txt")                   # returns Anthropic content blocks

TypeScript / Next.js — via the MCP server (no npm package needed):

pxcore-mcp --http --port 8765     # your TS code connects with @modelcontextprotocol/sdk

Claude Code — the in-path proxy (also images system prompt + history):

pxcore-proxy                      # then launch with ANTHROPIC_BASE_URL=http://127.0.0.1:8788

Two one-time gates before it actually compresses (until then it safely returns text): calibrate your model, and confirm image passthrough. Full guide: docs/frontend-pxcore-guide.md and docs/pxcore-readme.md.

What's new in 0.1.0a12

Bug fix — auto-instrumented spans were being discarded. init(auto_instrument=True) produced spans that the ingest API rejected and dead-lettered, silently, behind an HTTP 202. Two independent causes, both fixed:

1. The opexia.* envelope was never stamped. @observe and ReasoningTrace attached opexia.schema_version / org_id / workspace_id / project_id to each span; the auto-instrument wrappers did not. Ingest validates the span's own attributes, so every one of them failed as schema_version_unknown.
2. Three attributes the envelope schema forbids. OpexiaSpanAttributes is extra="forbid", so a single unknown opexia.* key rejects the entire span. opexia.cost.unpriced_model, opexia.instrumentation_error (auto-instrument) and opexia.domain (ReasoningTrace(domain=…)) are now emitted under the opexia_sdk. prefix, which ingest ignores rather than validates.

ReasoningTrace(domain=…) no longer destroys the span. It is still not a queryable dimension server-side — OpexIA does not index domain.

New — transport="direct". Export spans straight to backend_url over HTTPS. No OpenTelemetry collector, no gRPC:

init(..., transport="direct")     # or set OPEXIA_TRANSPORT=direct

The default is unchanged (transport="collector", OTLP/gRPC to collector_endpoint), so upgrading does not alter the behaviour of an existing deployment. If you keep the collector, set encoding: json on its otlphttp exporter — OpexIA's ingest answers OTLP/protobuf with a 415.

Fixed — init() raised on Windows. The WAL was opened for append before the previous run's WAL was replayed, and renaming an open file raises PermissionError on Windows (it silently succeeds on POSIX). Replay now happens first.

Fixed — a cold-connection timeout dropped a span batch. BatchSpanProcessor does not retry a failed export. The direct exporter now retries once on transport errors.

Quickstart

Call init() once at process startup:

from opexia.trace import init

init(
    org_id="pwc",
    workspace_id="chatpwc",
    project_id="due-diligence",
    backend_url="https://ingest.opexia.dev",
    api_key="opx_live_...",          # your workspace API key
    transport="direct",              # no collector to run
)

With auto_instrument=True (the default), every LLM call made through litellm / anthropic / openai is now traced. No other code changes required.

The three integration patterns

Pattern A — Auto-instrument (zero code changes)

init(auto_instrument=True) monkey-patches the litellm / anthropic / openai client methods at startup. Every completion call emits a gen_ai.* span carrying the model, token usage, and computed USD cost. This is the default; you get it just by calling init().

Pattern B — the @observe decorator

Wrap any function — sync or async — to emit a reasoning span for it:

from opexia.trace import observe

@observe(reasoning_role="decomposer", node_type="decomposer", name="plan.decompose")
async def decompose(query: str) -> list[str]:
    ...

Nested @observe calls auto-parent via OpenTelemetry context — a decorated function called inside another decorated function becomes its child span, so the reasoning tree falls out of normal call structure.

Pattern C — the ReasoningTrace context manager

For explicit, structured traces — when you want to record decisions, sources, and costs by hand:

from opexia.trace import ReasoningTrace

with ReasoningTrace(name="answer.query", reasoning_role="synthesis") as trace:
    trace.record_cost(model="claude-opus-4-6", input_tokens=1200, output_tokens=400)
    trace.record_sources(used=[...], consulted=[...], dropped=[...])
    trace.record_decision(selected="opt_a", rules_fired=[...], scores={...})
    with trace.subnode(name="retrieve", node_type="retriever") as node:
        node.record_sources(...)

Framework adapters

For agent frameworks, one line instruments a whole crew:

from opexia.trace.adapters.microsoft import instrument_microsoft_agents
instrument_microsoft_agents(crew)        # Microsoft Agent Framework

from opexia.trace.adapters.reasonix import instrument_reasonix
instrument_reasonix(orchestrator)        # Reasonix orchestrator

Adapters are duck-typed — they import nothing from the framework, so they never pin you to a version.

Per-employee attribution — opexia.end_user

When your app serves many employees, tag each trace with the actor who made the request. The alignment engine groups its on-/off-mission verdicts by this key (and buckets unattributed traces under (unattributed)). It's optional and per-request — set it wherever you already set reasoning_role:

# On the decorator:
@observe(reasoning_role="synthesis", end_user="[e-mail]")
def handle_query(...): ...

# On a ReasoningTrace / subnode:
with ReasoningTrace(name="chat.turn", end_user="[e-mail]") as trace:
    ...

# Or imperatively, once the actor is resolved mid-request:
with ReasoningTrace(name="chat.turn") as trace:
    trace.record_end_user(current_user.email)   # parity with TS setOpexiaEndUser

Distinct from the generic opexia.user_id envelope tag — end_user is the dedicated actor key the alignment feature reads. An empty/omitted value is a no-op (the trace stays unattributed). (TypeScript: use setOpexiaEndUser(span, endUser) — see the instrumentation snippet below.)

Agent Fleet Monitor — real-time topology & flags

The Agent Fleet Monitor is a separate, real-time capability from the batch reconstruction engines above. It folds your live span stream into one persistent topology map per fleet — nodes are your agents, edges are the hand-offs between them — and shows data-volume in/out (tokens + cache tokens) per node and edge, flagging latency spikes and errors live on the dashboard. It is deterministic and LLM-free: it never reads your prompt/response content, only counts and status.

It works with zero changes — it auto-discovers agents from the spans you already emit. But because it identifies a node by this ladder…

gen_ai.agent.name  →  gen_ai.system | opexia.reasoning_role | opexia.node_type  →  (unattributed)

…if you don't name your agents, distinct agents that share a role collapse into one coarse node (e.g. a fleet shows up as retrieval|retrieval instead of Planner, Researcher, Writer). To get a clean, correct topology, set a stable gen_ai.agent.name on each agent's span. That single attribute is the whole ask.

What the monitor reads (set these; most are already emitted)

Signal	Attribute(s)	Who sets it	Effect
Node identity	gen_ai.agent.name (fallbacks: agent.name, gen_ai.agent.id, agent.id)	you (explicit)	The node label. Keep it stable across runs.
Edge (A → B hand-off)	parent_span_id	OTel automatically	Every hand-off = one edge. Nest your spans (see below).
Volume in/out	gen_ai.usage.input_tokens / output_tokens	auto-instrument / your LLM span	tokens-in / tokens-out on the node + edge
Cache volume	gen_ai.usage.cache_read_input_tokens / cache_creation_input_tokens (OpenAI cached_tokens also read)	auto-instrument	cache-in / cache-out
Error flag	span status = ERROR	your try/except → record on span	node/edge lights red
Latency flag	span duration	OTel automatically	amber flag when a node runs slow vs its own learned baseline

Fleet grouping in v1 is automatic: each run is grouped by its entry (first-seen) agent, so naming the first agent in a run especially matters.

The one rule that determines the map: name the span that carries the tokens

The monitor folds every span into a node keyed by its identity, and a node's token/cache volume is the usage on the spans that share that identity. The in/out token counts live on your LLM-call spans (auto-instrument / traceLlmCall / Vercel AI SDK) — not on a wrapper span. So:

• Set gen_ai.agent.name on the LLM-call span itself → that call's volume folds onto the agent's node. This is what you want.
• Leave LLM-call spans unnamed → they group into a node named after the provider (anthropic, openai) via the gen_ai.system fallback, and the volume lands there. Still a valid view — just per-provider, not per-agent.
• Edges come from parent_span_id: nest spans so agent B's work runs inside agent A's span, and you get the A → B hand-off edge for free.

Python

Recommended — keep auto-instrument on (zero per-call code). Auto-instrument already emits one span per LLM call carrying the token/cache volume. Name your agents with @observe (or ReasoningTrace) and you get the hand-off topology for free — nested calls auto-parent into edges:

from opexia.trace import init, observe
from opentelemetry.trace import get_current_span

init(org_id="acme", workspace_id="chat", project_id="ui",
     backend_url="https://opexia.internal.example.com", api_key="opx_live_…")
     # auto_instrument=True is the default -> LLM calls emit their own spans.

@observe(reasoning_role="planner", node_type="planner", name="plan.step")
async def planner(question):
    get_current_span().set_attribute("gen_ai.agent.name", "Planner")  # STABLE label
    plan = await ask_llm(f"Plan: {question}")
    return await researcher(plan)     # child span -> Planner → Researcher edge

@observe(reasoning_role="retriever", node_type="researcher", name="research.step")
async def researcher(plan):
    get_current_span().set_attribute("gen_ai.agent.name", "Researcher")
    return await ask_llm(plan)        # auto-instrumented LLM call nests here

What you get: agent nodes Planner and Researcher with the hand-off edge between them, plus the LLM calls as provider nodes (anthropic, openai) that carry the token/cache volume, hanging off whichever agent made them. This is the low-friction path and it keeps cost + reliability + decomposition instrumentation fully intact.

Advanced — per-agent volume (fold the tokens onto the agent node). The token counts live on the LLM-call span, so to move volume off the provider node and onto the agent node, gen_ai.agent.name has to be on that span. Since auto-instrument owns it and doesn't set an agent name, you hand-instrument the call — and turn auto-instrument off (init(..., auto_instrument=False)) or skip it for these calls, so the same tokens aren't also counted on a duplicate anthropic node:

from opentelemetry.trace import get_tracer
tracer = get_tracer("my-ai-app")

def call_as_agent(agent_name, role, messages):
    with tracer.start_as_current_span(f"agent.{agent_name}") as span:
        span.set_attribute("gen_ai.agent.name", agent_name)   # node identity
        span.set_attribute("opexia.reasoning_role", role)
        r = anthropic_client.messages.create(
            model="claude-opus-4-6", max_tokens=800, messages=messages)
        span.set_attribute("gen_ai.usage.input_tokens",  r.usage.input_tokens)
        span.set_attribute("gen_ai.usage.output_tokens", r.usage.output_tokens)
        return r

Trade-off: auto_instrument=False is global — it disables the automatic cost/reliability spans everywhere, not just for the fleet map. Only reach for the advanced path if per-agent volume attribution matters more than zero-effort cost tracking. For most teams the recommended path (provider nodes carry volume, agent nodes carry structure) is the right call.

Errors flag automatically — a raised exception sets the span status to ERROR, which the monitor turns into a red flag. No extra code.

Next.js / TypeScript

Put gen_ai.agent.name on the token-bearing span. Extend the traceLlmCall helper from Pattern B with an agentName — one line — so identity and volume land on the same node:

// lib/opexia.ts — add agentName to the existing helper
export async function traceLlmCall<T>(
  fn: () => Promise<T>,
  opts: { system: System; model: string; agentName?: string },
): Promise<T> {
  return tracer.startActiveSpan(`gen_ai.${opts.system}.completion`, async (span) => {
    if (opts.agentName) span.setAttribute("gen_ai.agent.name", opts.agentName); // node label + volume
    span.setAttribute("gen_ai.system", opts.system);
    span.setAttribute("gen_ai.request.model", opts.model);
    // ...unchanged: attachUsage() stamps gen_ai.usage.* -> this node's volume,
    // recordException + ERROR status -> red flag, span.end() in finally.
  });
}

// app/api/answer/route.ts — two-agent fleet: Planner → Researcher
import { traceLlmCall } from "@/lib/opexia";
import Anthropic from "@anthropic-ai/sdk";

const anthropic = new Anthropic();

export async function POST(req: Request) {
  const { question } = await req.json();

  // Planner's call is the parent span...
  const plan = await traceLlmCall(
    () => anthropic.messages.create({
      model: "claude-opus-4-6", max_tokens: 200,
      messages: [{ role: "user", content: `Plan: ${question}` }],
    }),
    { system: "anthropic", model: "claude-opus-4-6", agentName: "Planner" },
  );

  // ...Researcher's call nests inside it -> Planner → Researcher edge.
  const answer = await traceLlmCall(
    () => anthropic.messages.create({
      model: "claude-opus-4-6", max_tokens: 800,
      messages: [{ role: "user", content: String(plan.content) }],
    }),
    { system: "anthropic", model: "claude-opus-4-6", agentName: "Researcher" },
  );

  return Response.json({ answer: answer.content });
}

Vercel AI SDK users: each model call already emits its own span with the token volume — wrap it in a tracer.startActiveSpan(...) on which you set gen_ai.agent.name, so the AI SDK's usage folds onto the named agent node.

That's the whole integration: name the token-bearing spans and keep the calls nested. Edges, volume, latency baselines, and error flags are derived for you. Read it back over REST + WebSocket per docs/fleet-monitor-dashboard-integration.md.

Use from TypeScript / Next.js

opexia-trace is the Python sugar. The transport is plain OTLP/HTTP, so your Next.js / TypeScript / Vercel-AI-SDK / Cloudflare-Workers app ships to the exact same ingest with the exact same backend intelligence — no Python process required.

Install (Next.js, Node runtime)

npm install \
  @opentelemetry/api \
  @opentelemetry/sdk-node \
  @opentelemetry/exporter-trace-otlp-http \
  @opentelemetry/resources \
  @opentelemetry/semantic-conventions

One-time setup — instrumentation.ts (Next.js native hook)

// instrumentation.ts (project root, sibling of next.config.js)
import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { Resource } from "@opentelemetry/resources";
import { SemanticResourceAttributes } from "@opentelemetry/semantic-conventions";

export async function register() {
  if (process.env.NEXT_RUNTIME !== "nodejs") return;

  const sdk = new NodeSDK({
    resource: new Resource({
      [SemanticResourceAttributes.SERVICE_NAME]: "my-ai-app",
      "opexia.org":       process.env.OPEXIA_ORG_ID!,
      "opexia.workspace": process.env.OPEXIA_WORKSPACE_ID!,
      "opexia.project":   process.env.OPEXIA_PROJECT_ID!,
    }),
    traceExporter: new OTLPTraceExporter({
      url: "https://ingest.opexia.dev/v1/traces",
      headers: { "x-opexia-api-key": process.env.OPEXIA_API_KEY! },
    }),
  });
  sdk.start();
}

Add to next.config.js:

experimental: { instrumentationHook: true }

That's it — your app is now wired to OpexIA.

Pattern A — Vercel AI SDK (recommended)

If you use the ai SDK (generateText, streamText, generateObject), just enable its native OTel telemetry. Every model call becomes a span automatically, and our reconstruction engines pick up cost / latency / model / token usage for free:

import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";

const result = await generateText({
  model: openai("gpt-4o"),
  prompt: userQuery,
  experimental_telemetry: {
    isEnabled: true,
    functionId: "answer-user",
    metadata: {
      "opexia.workspace": process.env.OPEXIA_WORKSPACE_ID!,
      "opexia.trace_kind": "answer",
    },
  },
});

Pattern B — raw @anthropic-ai/sdk / openai calls (auto-instrument equivalent)

Vercel AI SDK is great, but plenty of TS apps call @anthropic-ai/sdk or openai directly. The Python SDK auto-instruments these for you; in TS you write a tiny helper once, then every LLM call lights up the same way. The helper emits the same gen_ai.* semantic conventions the Python SDK emits (mirrors opexia_trace/opexia/trace/auto_instrument.py), so the backend treats both identically.

One-time helper — lib/opexia.ts:

import { trace, SpanStatusCode, Span } from "@opentelemetry/api";

type System = "anthropic" | "openai" | "litellm";

const tracer = trace.getTracer("opexia-trace-helpers");

export async function traceLlmCall<T>(
  fn: () => Promise<T>,
  opts: { system: System; model: string },
): Promise<T> {
  return tracer.startActiveSpan(
    `gen_ai.${opts.system}.completion`,
    async (span: Span) => {
      span.setAttribute("gen_ai.system", opts.system);
      span.setAttribute("gen_ai.request.model", opts.model);
      try {
        const resp = await fn();
        attachUsage(span, resp, opts.system, opts.model);
        span.setStatus({ code: SpanStatusCode.OK });
        return resp;
      } catch (err) {
        span.recordException(err as Error);
        span.setStatus({ code: SpanStatusCode.ERROR });
        throw err;
      } finally {
        span.end();
      }
    },
  );
}

function attachUsage(span: Span, resp: any, system: System, requested: string) {
  try {
    const usage = resp?.usage ?? {};
    const inputTokens  = usage.input_tokens  ?? usage.prompt_tokens     ?? 0;
    const outputTokens = usage.output_tokens ?? usage.completion_tokens ?? 0;
    const modelUsed    = resp?.model ?? requested;
    const finish =
      resp?.choices?.[0]?.finish_reason ??  // OpenAI / LiteLLM
      resp?.stop_reason ??                  // Anthropic
      "";
    span.setAttribute("gen_ai.usage.input_tokens",  Number(inputTokens));
    span.setAttribute("gen_ai.usage.output_tokens", Number(outputTokens));
    if (modelUsed) span.setAttribute("gen_ai.response.model", String(modelUsed));
    if (finish)    span.setAttribute("gen_ai.response.finish_reason", String(finish));
  } catch (err) {
    span.setAttribute("opexia.instrumentation_error", String(err));
  }
}

Usage with @anthropic-ai/sdk — wrap the call, no other changes:

// app/api/answer/route.ts
import Anthropic from "@anthropic-ai/sdk";
import { traceLlmCall } from "@/lib/opexia";

const anthropic = new Anthropic();

export async function POST(req: Request) {
  const { question } = await req.json();
  const resp = await traceLlmCall(
    () => anthropic.messages.create({
      model: "claude-opus-4-6",
      max_tokens: 100,
      messages: [{ role: "user", content: question }],
    }),
    { system: "anthropic", model: "claude-opus-4-6" },
  );
  return Response.json({ answer: resp.content });
}

Usage with openai — identical helper, only system changes:

import OpenAI from "openai";
import { traceLlmCall } from "@/lib/opexia";

const openai = new OpenAI();

const resp = await traceLlmCall(
  () => openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: question }],
  }),
  { system: "openai", model: "gpt-4o" },
);

What lands in OpexIA — the same span shape the Python SDK's auto-instrument produces, so cost/latency/token attribution and the reliability engine work out of the box:

Attribute	Value
gen_ai.system	anthropic / openai / litellm
gen_ai.request.model	claude-opus-4-6
gen_ai.response.model	claude-opus-4-6 (server-confirmed)
gen_ai.usage.input_tokens	12
gen_ai.usage.output_tokens	8
gen_ai.response.finish_reason	end_turn (Anthropic) / stop (OpenAI)

Backend cost reconstruction reads gen_ai.response.model + the token counts — no extra work needed on the client.

Pattern C — manual reasoning spans (decomposer, decision, sources)

To unlock the Decomposition / Decision Trace / Sources engines you have to set our semantic-convention attributes (see the reference table at the bottom). Example — a decomposer step that also records the URLs it consulted and the decision it made:

import { trace, SpanStatusCode } from "@opentelemetry/api";

const tracer = trace.getTracer("my-ai-app");

await tracer.startActiveSpan("plan.decompose", async (span) => {
  span.setAttribute("opexia.reasoning_role", "decomposer");
  span.setAttribute("opexia.node_type", "decomposer");

  try {
    const subqueries = await decomposeQuery(userQuery);
    const consulted  = await retrieveCandidates(subqueries);
    const cited      = pickCitations(consulted);

    // Sources engine consumes these JSON arrays of URLs:
    span.setAttribute("opexia.sources.used",      JSON.stringify(cited));
    span.setAttribute("opexia.sources.consulted", JSON.stringify(consulted));
    span.setAttribute("opexia.sources.dropped",   JSON.stringify([]));

    // Decision Trace engine consumes these:
    span.setAttribute("opexia.decision.selected",    "strategy_a");
    span.setAttribute("opexia.decision.rules_fired", JSON.stringify(["recency_boost"]));
    span.setAttribute("opexia.decision.scores",      JSON.stringify({ a: 0.82, b: 0.41 }));

    span.setStatus({ code: SpanStatusCode.OK });
    return subqueries;
  } catch (err) {
    span.recordException(err as Error);
    span.setStatus({ code: SpanStatusCode.ERROR });
    throw err;
  } finally {
    span.end();
  }
});

Nested startActiveSpan calls auto-parent the same way they do in Python — the reasoning tree falls out of normal control flow.

Edge runtime / Cloudflare Workers

Full OTel SDKs don't fit in Edge / Workers. Use the fetch-based exporter or fall back to a POST to /v1/traces on a Node API route. (Edge-native SDK is on the roadmap; ask if you need it sooner.)

Multi-provider chat apps (UI dropdown picks the model)

Most production chat apps let the end-user pick a model from a dropdown — Claude, GPT-4o, Gemini, etc. — at runtime. opexia-trace handles this without any extra config: the patches are at the SDK-method level, not at some global "current model" setting. Whichever SDK your dispatcher actually calls for a given turn is what gets tagged. Each turn → its own span → correctly attributed by provider and model.

UI dropdown → your dispatcher → SDK A.method() ──┐
                              ↘ SDK B.method() ──┼──► opexia-trace tags
                              ↘ SDK C.method() ──┘    each one individually

A single chat session with mixed turns shows up as N spans, one per call, with gen_ai.system and gen_ai.request.model set per-turn. Cost rolls up per provider, per model — no extra plumbing on your side.

Python — explicit dispatcher

from anthropic import Anthropic
from openai import OpenAI
from opexia.trace import init

init(org_id="acme", workspace_id="chat", project_id="ui",
     backend_url="…", api_key="opx_live_…")

anthropic_client = Anthropic()
openai_client    = OpenAI()

def chat(user_selected_model: str, messages: list[dict]) -> str:
    if user_selected_model.startswith("claude"):
        r = anthropic_client.messages.create(
            model=user_selected_model, max_tokens=1000, messages=messages)
        return r.content[0].text
    if user_selected_model.startswith("gpt"):
        r = openai_client.chat.completions.create(
            model=user_selected_model, messages=messages)
        return r.choices[0].message.content
    raise ValueError(f"unknown model: {user_selected_model}")

Each branch hits a patched method; spans land with the right gen_ai.system (anthropic or openai) and gen_ai.request.model.

Python — single-call dispatcher via litellm (recommended)

The cleanest pattern when you support many providers — one function, one patched call, ~100 providers covered out of the box:

import litellm

def chat(user_selected_model: str, messages: list[dict]) -> str:
    r = litellm.completion(
        model=user_selected_model,   # e.g. "anthropic/claude-opus-4-6", "openai/gpt-4o", "gemini/gemini-pro"
        messages=messages,
    )
    return r.choices[0].message.content

opexia-trace patches litellm.completion, so every dropdown selection routes through the same patched function. Spans get gen_ai.system="litellm" and gen_ai.request.model carries the actual provider+model (anthropic/claude-opus-4-6). Our cost engine handles the provider prefix.

TypeScript — explicit dispatcher with traceLlmCall

The traceLlmCall helper from Pattern B takes { system, model } per call, so the dispatcher writes the right tag per branch:

import Anthropic from "@anthropic-ai/sdk";
import OpenAI from "openai";
import { traceLlmCall } from "@/lib/opexia";

const anthropic = new Anthropic();
const openai    = new OpenAI();

export async function chat(selectedModel: string, messages: any[]) {
  if (selectedModel.startsWith("claude")) {
    return traceLlmCall(
      () => anthropic.messages.create({ model: selectedModel, max_tokens: 1000, messages }),
      { system: "anthropic", model: selectedModel },
    );
  }
  if (selectedModel.startsWith("gpt")) {
    return traceLlmCall(
      () => openai.chat.completions.create({ model: selectedModel, messages }),
      { system: "openai", model: selectedModel },
    );
  }
  throw new Error(`unknown model: ${selectedModel}`);
}

TypeScript — Vercel AI SDK provider lookup (recommended)

Each @ai-sdk/* provider package emits its own OTel span with gen_ai.system set correctly, so a provider lookup table gives you correct attribution across N providers with zero per-provider code:

import { generateText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { openai }    from "@ai-sdk/openai";
import { google }    from "@ai-sdk/google";

const providers = { anthropic, openai, google } as const;
type Provider = keyof typeof providers;

export async function chat(provider: Provider, model: string, prompt: string) {
  return generateText({
    model: providers[provider](model),
    prompt,
    experimental_telemetry: {
      isEnabled: true,
      metadata: { "opexia.workspace": process.env.OPEXIA_WORKSPACE_ID! },
    },
  });
}

Bonus — capturing the dropdown choice itself

Useful for debugging "I picked Claude but the answer feels like GPT-4o" — stamp the user's intent on a parent span so you can see both layers in the trace tree:

from opexia.trace import ReasoningTrace

with ReasoningTrace(name="chat.turn", reasoning_role="synthesis") as trace:
    trace._span.set_attribute("opexia.user_selected_model", user_selected_model)
    answer = chat(user_selected_model, messages)

The parent span carries what the user asked for; the child span carries what was actually sent. If they diverge, your dispatcher has a bug.

Cost for any model, any provider

opexia.cost.usd is computed for any model from any provider — there is no fixed allow-list. The SDK resolves a price through a layered strategy, first hit wins:

1. Provider-reported cost — if the response already carries a cost (e.g. OpenRouter usage accounting, or litellm's response_cost), it is used as-is. Zero maintenance, exact for every model the gateway prices.
2. Bundled snapshot — a version-pinned table of common models (GPT-4o, Claude, Kimi K2, Qwen2.5-72B, GLM-4.6, DeepSeek-V3, …), matched after normalizing the model id, so provider prefixes and variant suffixes resolve: openrouter/z-ai/glm-4.6:free → glm-4.6.
3. litellm price map — if litellm is importable, its continuously-updated map (thousands of models across ~100 providers) is consulted. No network.
4. OpenRouter live — last resort only: prices are fetched from OpenRouter's /api/v1/models and cached (6h). Disable with OPEXIA_PRICING_DISABLE_NETWORK=1.
5. If none match, cost is 0.0 and opexia.cost.unpriced_model is set — a missing price is always flagged, never a silent zero.

opexia.cost.model_pricing_version records which source produced the figure (provider, the snapshot version, litellm:<ver>, or openrouter-live), so the cost is reproducible server-side. This works identically through OpenRouter or any gateway — pass the model as usual (litellm.completion(model="openrouter/moonshotai/kimi-k2", …)) and the span carries an accurate opexia.cost.usd.

Reading the computed artifacts — Read API

Once spans are in, your Next.js dashboard / debug UI / analytics can fetch the reconstructed intelligence over plain REST. All endpoints are at https://api.opexia.dev/v1/observ/* and authenticated with the same x-opexia-api-key + x-opexia-workspace-id headers.

Endpoint	What you get back
GET /v1/observ/traces?workspace_id=…	paginated trace list
GET /v1/observ/traces/{tid}	full trace summary
GET /v1/observ/traces/{tid}/decomposition	reconstructed decomposition tree
GET /v1/observ/traces/{tid}/decision-trace	reasoning DAG
GET /v1/observ/traces/{tid}/sources	sources matrix (cited / dropped / recommended-unseen, each with a live web-verification verdict)
GET /v1/observ/traces/{tid}/reliability	per-claim hallucination score
GET /v1/observ/usage?workspace_id=…	cost + token usage rollups
GET /v1/observ/drift?workspace_id=…	daily drift signal

Quick Next.js example — a server component showing the sources matrix:

// app/traces/[tid]/sources/page.tsx
export default async function Page({ params }: { params: { tid: string } }) {
  const res = await fetch(
    `https://api.opexia.dev/v1/observ/traces/${params.tid}/sources`,
    {
      headers: {
        "x-opexia-api-key":      process.env.OPEXIA_API_KEY!,
        "x-opexia-workspace-id": process.env.OPEXIA_WORKSPACE_ID!,
      },
      cache: "no-store",
    },
  );
  const sources = await res.json();
  return <SourcesMatrix data={sources} />;
}

The OpenAPI / Swagger UI for every endpoint lives at https://api.opexia.dev/v1/observ/docs (and the ingest at https://ingest.opexia.dev/docs) — fully brand-styled, "Try it out" works in-browser.

Beyond your app — Claude Code enterprise telemetry

OpexIA also ingests telemetry from Claude Code running across your organization — no opexia-trace install required. Claude Code emits native OpenTelemetry metrics + events; an IT admin points every install at OpexIA with a single managed-settings file (distributed via MDM / Jamf / Intune / Group Policy) plus a workspace-scoped cck_live_… key. You then get, per employee:

• session count, active time, cost (USD), input / output / cache tokens
• git commits authored in Claude Code — SHA, message, files changed, ± lines (captured by a small PostToolUse hook)
• lines of code added / removed, edit accept/reject decisions, top files edited

This is a sibling capability to the SDK: the SDK traces your LLM app's reasoning, while Claude Code telemetry tracks your team's use of the Claude Code CLI. Both land in the same workspace and share one Read API + Swagger. Compliance is inherited from the same layer — per-workspace capture_text opt-in gates prompt/tool-content, every row is PII-redacted at ingest, and Read API access is RBAC-gated + audit-logged.

Read API (workspace_viewer role — Better-Auth session or workspace key):

Endpoint	What you get back
GET /v1/observ/orgs/{org}/workspaces/{ws}/claude-code/usage?period=day|week|month	per-employee rollup: tokens / cost / commits / PRs / lines / sessions / active-time / edit decisions
GET /v1/observ/orgs/{org}/workspaces/{ws}/claude-code/commits?limit=50	recent Claude Code-authored commits (SHA, message, files changed, ± lines)
GET /v1/observ/orgs/{org}/workspaces/{ws}/claude-code/top-files?period=week	most-edited files across the workspace

Full setup runbook — managed-settings template, the commit hook (.sh + .ps1), per-platform MDM rollout, and the PII / compliance posture — lives at docs/claude-code-enterprise-setup.md in the OpexIA repo.

Semantic conventions (for non-Python languages)

Set these on your spans (Python SDK does it for you). The reconstruction engines key off these names.

Attribute	Type	Purpose
opexia.org / opexia.workspace / opexia.project	resource attrs	tenancy stamp on every span
opexia.reasoning_role	string	decomposer / retriever / synthesizer / decision / guardrail
opexia.node_type	string	finer-grained role; free-form taxonomy
gen_ai.agent.name	string	Agent Fleet Monitor node identity — the agent's stable display name (fallbacks: agent.name / gen_ai.agent.id / agent.id). See Agent Fleet Monitor.
opexia.sources.used	JSON array of strings	URLs the step actually cited
opexia.sources.consulted	JSON array of strings	URLs the step looked at
opexia.sources.dropped	JSON array of strings	URLs the step considered then rejected
opexia.decision.selected	string	which option was chosen
opexia.decision.rules_fired	JSON array of strings	named rules that influenced the decision
opexia.decision.scores	JSON object {option: score}	per-option scoring
opexia.cost.usd	float	computed USD cost for any model (see Cost for any model); optional — we derive from tokens if absent
opexia.cost.model_pricing_version	string	which price source produced the cost (provider / snapshot version / litellm:<ver> / openrouter-live)
opexia.cost.unpriced_model	string	set only when no price source matched — the cost is 0.0 and this flags it (never a silent zero)
gen_ai.system / gen_ai.request.model / gen_ai.usage.input_tokens / gen_ai.usage.output_tokens	OTel GenAI conventions	LLM call metadata

JSON-encode arrays/objects as strings — OTel attributes don't support nested types natively. The backend parses them on the way in.

Configuration — init() parameters

Parameter	Default	Meaning
org_id / workspace_id / project_id	required	Tenancy identity stamped on every span.
backend_url	required	The OpexIA backend base URL.
api_key	required	Workspace API key (opx_live_… / opx_test_…).
transport	"collector"	"collector" = OTLP/gRPC to collector_endpoint. "direct" = HTTPS POST straight to backend_url, no collector, no gRPC. Also settable via OPEXIA_TRANSPORT; the argument wins.
collector_endpoint	http://localhost:4317	OTLP collector endpoint. Ignored when transport="direct".
wal_path	.opexia-wal/spans.jsonl	Write-ahead-log path (see Durability).
auto_instrument	True	Patch litellm/anthropic/openai at startup.
fail_open	False	See "fail_open scope" in the FAQ.
sampler_rate	1.0	Fraction of traces sampled.

Durability — the write-ahead log

Every span is fsync-ed to a local WAL (wal_path) before it is handed to the exporter, so a process that crashes mid-export leaves a durable on-disk record of what was in flight. On a clean shutdown the WAL is deleted; spans that never made it out survive on disk.

What the next init() does with that WAL, precisely: it counts the entries, logs a warning, and renames the file to <wal_path>.replayed-<timestamp>. It does not re-export them — full OTLP round-trip replay is not implemented yet. The renamed file is there so an operator can re-emit it manually. Do not read "WAL-backed" as "guaranteed at-least-once delivery to the backend"; read it as "a crash never silently loses the evidence."

Past max_wal_bytes (10 MB default) the WAL write is refused and a counter advances. The span is still exported — only its crash-durability is lost.

Troubleshooting

• Spans are accepted but never appear. The ingest API returns 202 even when it dead-letters every span in the batch — a rejected span is quarantined, not dropped, and the 202 refers to receipt, not validity. On transport="direct" the SDK reads the response's partialSuccess field and logs N span(s) were DEAD-LETTERED by ingest. If you are on 0.1.0a11 or earlier, this is almost certainly you — upgrade (see What's new).
• 415 OTLP/protobuf is not supported. Your collector is exporting protobuf. Set encoding: json on its otlphttp exporter. The SDK's transport="direct" path is unaffected.
• 404 on /v1/traces. You are pointed at an OpexIA backend that predates native OTLP support. Upgrade the backend, or use transport="direct" which posts to /v1/otlp/traces.
• No spans appear at all. Check backend_url / collector_endpoint reachability and that api_key is a live (not revoked) workspace key. A 401 is logged explicitly by the direct exporter.
• A custom opexia.* attribute made spans vanish. OpexiaSpanAttributes is extra="forbid": one unknown opexia.-prefixed key dead-letters the whole span. Put your own attributes under any other prefix — the SDK uses opexia_sdk. for its internal diagnostics for exactly this reason.
• init() has not been called. A @observe / ReasoningTrace ran before init(). Call init() once at process startup, before any traced code.
• Auto-instrument patched 0 clients. None of litellm/anthropic/openai are importable in the process — Pattern A has nothing to patch. Use Pattern B/C.
• PermissionError on init() (Windows). Fixed in 0.1.0a12; earlier releases renamed the WAL file while it was still open.

FAQ

Do I need Python to use OpexIA? No. Ingest accepts standard OTLP/JSON, so any language with an OpenTelemetry SDK (TypeScript, Go, Java, .NET, …) can point a stock exporter at it and get the full reconstruction stack. The API key alone identifies your org and workspace, so a stock span needs no opexia.* attributes at all:

export OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.opexia.dev
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json   # protobuf gets a 415
export OTEL_EXPORTER_OTLP_HEADERS="x-opexia-api-key=opx_live_…"

Both /v1/traces (what an exporter appends to a base endpoint) and /v1/otlp/traces work. Add the semantic conventions below to unlock the engines. See Use from TypeScript / Next.js.

Then why install this SDK? For auto_instrument (zero-code tracing of litellm/anthropic/openai), the @observe / ReasoningTrace reasoning-altitude helpers, the cost resolver, and the WAL. For transport alone, a stock OTel SDK is equivalent.

Is there a native @opexia/trace npm package? Not yet — on the roadmap. For now use the raw OpenTelemetry JS SDK with the semantic conventions documented above; Vercel AI SDK users get most of it for free via experimental_telemetry.

Does opexia-trace import my agent framework? No. The adapters are duck-typed and import nothing from the framework or from the OpexIA backend.

What does fail_open actually cover? fail_open=True wraps the whole init() body — exporter construction, TracerProvider setup, WAL replay, and auto-instrument registration. A bad collector_endpoint, a WAL-path I/O error, or an exporter-constructor failure all degrade to a logged exception instead of taking down your startup. set_config() runs before the guard, so get_config() still works in the degraded state (you simply get no spans). With the default fail_open=False, any of those failures raises.

Can I emit spans during process shutdown? No. Do not start new spans after your shutdown hook runs — a span started concurrently with the exporter shutting down can be dropped (this race exists in upstream OpenTelemetry's BatchSpanProcessor too). Finish traced work before tearing the process down. (Tracked as deferred note B2-N7.)

Is the SDK safe in production? Yes — span export is buffered and WAL-backed, and a traced function never fails because tracing failed: attribute-extraction errors are caught and recorded on the span, and the original return value passes through untouched.