Skip to main content

agent-watch-otel

Your production LLM spend in Agent Watch, hashed in your own process, and over-budget calls refused before they leave it.

One line beside your Phoenix registration. Prompt text, tool arguments and model responses never leave the process — only digests, token counts and ids do. Phoenix keeps receiving everything, unchanged.

pip install agent-watch-otel
from phoenix.otel import register
from agent_watch_otel import AgentWatchSpanProcessor

tracer_provider = register(project_name="Production", endpoint=..., batch=True)
tracer_provider.add_span_processor(AgentWatchSpanProcessor(token="..."))  # beside yours

The token is an Agent Watch SDK token scoped ingest:production. token and endpoint may also come from AGENT_WATCH_TOKEN and AGENT_WATCH_ENDPOINT.


What it sends

Finished spans whose openinference.span.kind is LLM, and nothing else. Per call:

Sent Not sent
sha256 of the system/developer prefix, and its 8-token window digests the prompt
sha256 of the tool definitions the tool definitions, and every tool argument
model, provider, token counts (input, output, cache read/write, reasoning) the response
span id, trace id, agent name (see "Which agent"), end time anything the span carries that is not in this table
whether the call failed and, if it reached the provider, an input estimate from its request body's size the request body
service.name and service.version from the OTel resource

Before a model call, the gate sends a decision request (see "Refusing over-budget calls"). It carries the agent name (see "Which agent"), service.name, the model and the deployment environment, plus counts of local refusals and fail-opens. Never prompt text, tool definitions or responses.

That last row is why this package exists rather than only the poller: Phoenix drops resource attributes at ingest, so a span processor is the only thing that can report which service and which version produced a call.

Digests are computed in on_end, on your thread, before anything is queued. The queue holds hashes; the sender thread never sees a prompt.

What it does to your application

One thing in it can stop a call: an over-budget refusal (see "Refusing over-budget calls"). Nothing else in the package reaches your application.

  • on_end never touches the network. It hashes and enqueues. A background daemon thread batches and POSTs.
  • on_end never raises. Every path out of it is wrapped. A bug here costs you a cost record, never a request.
  • The queue is bounded. When Agent Watch is unreachable or slow, calls are dropped and counted rather than buffered without limit inside your process.
  • It survives fork(). Under gunicorn --preload, uWSGI or Celery prefork, each child gets its own sender thread rather than silently dropping every call.
  • Shutting down is bounded. An endpoint that accepts the connection and never answers cannot stretch shutdown() past shutdown_seconds, however many times it is called — so stopping your process stays inside a short SIGTERM grace period. Call force_flush() instead when what you want is delivery rather than a prompt exit.
processor.stats()
# {'sent': 412, 'dropped_queue_full': 0, 'dropped_send_failed': 3,
#  'dropped_no_model': 0, 'dropped_error': 0}
processor.dropped   # 3

A call with no llm.model_name, llm.model or gen_ai.request.model is counted under dropped_no_model rather than sent under a guessed model, because spend priced against the wrong model is worse than spend that is visibly missing. A refused call has no spend, so it is sent with the model unknown instead.

Refusing over-budget calls

When a feature's production budget is spent, the agent's next model call is refused before it leaves your process. If httpx is installed (the OpenAI and Anthropic SDKs use it), registering the processor wraps httpx.Client.send and httpx.AsyncClient.send. The first request made inside an LLM span (openinference.span.kind = LLM, the spans this processor reports) asks Agent Watch about the agent the span names. Tool, agent and embedding spans, and spans that only set gen_ai.operation.name, are never gated. A refused call raises:

AgentWatchRefusal: An Agent Watch budget refused this call.

It names no cap, amount, feature or policy, and retryable is False: a retry gets the same answer from the held decision, with no request to Agent Watch or to the provider. Provider SDKs raise their own connection error with the refusal as its __cause__, and may retry with their own backoff first. Test for it with is_agent_watch_refusal:

from agent_watch_otel import is_agent_watch_refusal

try:
    client.chat.completions.create(**request)
except Exception as err:
    if is_agent_watch_refusal(err):
        return budget_spent_answer()
    raise

The cost. Answers are held per agent and deployment environment for as long as Agent Watch says (at most 60 s); a held answer adds no request. A fresh one waits at most 300 ms, with one request per agent in flight at a time, shared by threads and event loops: concurrent calls for the same agent wait on that one request, and every request in one span shares the span's decision. The 300 ms covers resolving the host too. A request still unanswered at 300 ms is not cut off: the call goes ahead as a fail-open, the request runs on in its daemon thread (at most 5 s, never holding the process open), and its answer is held for the agent's next calls, unless a newer request was sent meanwhile. So a gateway far away, slower than 300 ms on a fresh connection, still gets to refuse. A thread still alive past those 5 s (a resolver that hangs cannot be interrupted) counts as hung: no second one starts for that agent while it lives, and once four are hung, or 64 decision threads are alive at all, no new request starts until one ends. Many agents asking at once are all asked. An async wait never blocks the event loop, and a cancellation while it waits propagates at once.

Which agent. The agent is the first of the span attributes agent.name, gen_ai.agent.name and llm.task that is set and not blank, else the span name: the same rule Agent Watch uses when it reads your spans from Phoenix or Langfuse, so both name one agent. OpenInference's OpenAI instrumentation names every span ChatCompletion, so set agent.name (or wrap the call in a span named for the agent), or all such calls are one agent.

Fail-open, always. If Agent Watch is slow, unreachable, answers anything but 2xx, or answers something it cannot read, the call goes ahead, and the fail-open is reported on the next decision request for that agent. What is owed travels as one batch with a report_id, resent unchanged until a request carrying it is answered, so Agent Watch records it once. After a fail-open the agent is allowed without asking for 30 s, or for 5 minutes after a 401, 403 or 404 (the route is missing or the token lacks the scope); each call let through meanwhile is counted as a fail-open too. There is no fail-closed mode and no switch for one. Nothing inside the package reaches your application as anything but the refusal.

What passes ungated. Requests outside an LLM span; requests to metadata.google.internal, oauth2.googleapis.com and sts.googleapis.com, so a workload-identity token fetch is never refused; and the package's own requests. A refused call is still reported, with refused: true, and counts no spend; when its span names no model it is reported as model unknown.

A call that failed. A span ended with an error status is reported with errored: true. When the httpx request reached the provider and failed (a 5xx status, or a timeout or broken connection after it was sent; a 4xx such as 429 is not billed and is not estimated), the package records the request body's length on the span and reports the input as that length over 4, rounded up, with input_estimated: true and output 0; the provider's own count wins when the span has one. A DNS or connection failure never reached the provider and carries no estimate. Only the length is read, never the body, and nothing is measured on a call that succeeds.

Known limit: sampled-out spans are never gated. A span your sampler drops carries no name or attributes, so a call inside it goes ahead unjudged and is not counted anywhere. Keep LLM spans sampled wherever you rely on the gate. The span must also be current on the thread or task making the request, which is where instrumentation puts it.

Other transports. requests, urllib, gRPC and other clients are not intercepted. Call guard(agent_name) (or await guard_async(agent_name) in a coroutine) before such a call; it raises AgentWatchRefusal when refused, and returns at once when no processor is registered. It states the agent and the deployment environment from OTEL_RESOURCE_ATTRIBUTES; with neither environment key set there, it is judged as production.

Dev and CI. Set the resource attribute deployment.environment.name (or the older deployment.environment, read when the first is unset) to anything but production or prod and calls are allowed without judging; unset, they are judged as production.

Behind a proxy. Requests to Agent Watch, ingest and decisions alike, honour HTTPS_PROXY, HTTP_PROXY and NO_PROXY from the environment (lower case too). An https endpoint is tunnelled with CONNECT, so TLS still ends at Agent Watch; credentials in the proxy URL are sent to the proxy only. Redirects are never followed, proxy or not.

Options

Argument Default What it is
token $AGENT_WATCH_TOKEN SDK token scoped ingest:production
endpoint $AGENT_WATCH_ENDPOINT Ingest base URL; the route is v1/runtime/calls
instance_id the resource's service.name This deployment's name, as you know it
max_queue 2048 Calls held while the sender is behind
flush_seconds 5.0 How long a partial batch waits
timeout_seconds 10.0 Network timeout for one POST
shutdown_seconds 3.0 The whole budget shutdown() may spend, across every call

A missing token or endpoint raises at construction — on your startup path, where a misconfiguration is visible — never later from on_end.

Requirements

Python 3.9+ and opentelemetry-sdk. Nothing else, by policy: this code runs inside your application, so every dependency it took would become a version you have to resolve. httpx is gated when it is installed and never required.

Development

python -m venv .venv && .venv/bin/pip install opentelemetry-sdk
.venv/bin/python -m unittest              # the whole suite
.venv/bin/python -m unittest tests.test_vectors   # the fingerprint parity check
.venv/bin/pip install httpx && .venv/bin/python -m unittest tests.test_gate_conformance   # the gate parity check

conformance/runtime/gate-cases.json is the gate's behaviour (gateway answers in, outcomes out), replayed by tests/test_gate_conformance.py with sync and async httpx, and by the Node package. The suite runs without httpx too; the tests that need it are skipped.

conformance/runtime/fingerprint-vectors.json (at the repository root) is a byte copy of core's golden fingerprint vectors (packages/fingerprint/test/vectors.json, AWT-263). tests/test_vectors.py replays every case through this port, and the Node package replays the same file, so the implementations cannot drift apart without CI going red. It also covers a case no shared vector can express — see MAX_TOOL_DEPTH in agent_watch_otel/fingerprint.py.

Licence

MIT.

Metadata

Release files for agent-watch-otel 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for agent-watch-otel 0.2.0
File Size Uploaded Provenance Actions
agent_watch_otel-0.2.0.tar.gz 69.1 kB Without provenance Details for agent_watch_otel-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for agent-watch-otel 0.2.0
File Interpreter ABI Platform Actions
agent_watch_otel-0.2.0-py3-none-any.whl Python 3 none any Details for agent_watch_otel-0.2.0-py3-none-any.whl

Total release size: 111.1 kB

Release files / agent_watch_otel-0.2.0.tar.gz

Download URL agent_watch_otel-0.2.0.tar.gz
Size 69.1 kB
Tags Source
SHA-256 checksum
How to use checksums
875c29b556174941aa9682c281fd69d7c90d9f5e6ff9bf47b67d5d90ecf770d6
BLAKE2b-256 checksum
How to use checksums
f48d2cbe6de3ff370a765f347bd9f5d6addff3db0d6a071b9684a9c463036631
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Release files / agent_watch_otel-0.2.0-py3-none-any.whl

Download URL agent_watch_otel-0.2.0-py3-none-any.whl
Size 42.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
57129cefd6b744a608d85647915cc5b1335ec077c4cd1889b5e142d0e6ad649f
BLAKE2b-256 checksum
How to use checksums
b1cda49b0b983e896450082d621dc3db7b254bd76ec9fe767c53e26fb480937c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.6

Security for agent-watch-otel 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page