WeHarness tracing for Python

Send agent, model and tool spans to WeHarness using OpenTelemetry and OTLP/HTTP. Requires Python 3.10 or newer and a project ingest key from the WeHarness Tracing page.

import weharness_tracing as wh

# Set WEHARNESS_INGEST_KEY in your server environment, never in browser code.
wh.init(service="my-agent")
with wh.agent_run("support"):
    with wh.tool_call("lookup_order"):
        pass  # Call your tool here.
wh.shutdown()

The default endpoint is https://api.weharness.io/otlp/v1/traces. Override it with WEHARNESS_OTLP_ENDPOINT if needed. Set WEHARNESS_REVISION to correlate traces with a source revision.

Known prompt, output, tool-payload and stacktrace attributes are filtered by default. This filter is not a general secret detector: do not put secrets or personal data in span names, error messages or custom metadata. Content capture requires explicit configuration and appropriate project settings. An ingest key sends traces; it does not authorize reading them.

An existing OpenTelemetry tracer provider is reused. Background batches are bounded; call shutdown() to flush pending spans before the process exits.

Local evaluation runner (0.1.3)

The Python package includes weharness-run. Save the cases returned by the WeHarness MCP evaluate_local_start operation as cases.json, then run:

weharness-run --cases cases.json --command "python agent.py" --timeout 300 --out report.json

The command runs your agent locally, sends each case on stdin (or substitutes {input} as an argument), and records output, exit code and duration for each repetition. Give the report to evaluate_local_report through your coding host. The runner itself uploads nothing. Its exit code indicates report generation, not that the agent passed the behavior checks; inspect the evaluation verdict.

Your agent's provider credentials remain available. WeHarness credentials named WEHARNESS_TOKEN, WEHARNESS_INGEST_KEY, WEHARNESS_API_KEY, WEHARNESS_MCP_TOKEN, and variables starting with WEHARNESS_SECRET are removed from the child environment. This is a local process runner, not a security sandbox; run only code you trust. On POSIX, timed-out attempts receive a process-group termination; Windows currently terminates the direct process only.