latitude-telemetry-hermes

Stream Hermes Agent (Nous Research's open-source agent harness) sessions to Latitude as OTLP traces — user prompts, model turns, tool calls and results, the tools the agent was offered, memory reads and writes, delegated subagents, token usage, cost and timing, and the real system prompt — so you get full-fidelity observability of your agent runs in Latitude.

This is the Hermes counterpart to the other harness integrations (@latitude-data/claude-code-telemetry, @latitude-data/pi-telemetry, @latitude-data/openclaw-telemetry). Hermes is a Python harness and loads plugins from Python, so this connector ships as a pip package rather than npm.

Install

pip install latitude-telemetry-hermes

Then enable the plugin by adding latitude to ~/.hermes/config.yaml, plus Hermes's reasoning-delta forwarding so time-to-first-token is measurable:

plugins:
  enabled:
    - latitude
  stream_reasoning_deltas: true

stream_reasoning_deltas: true is required for TTFT. It is Hermes's own setting, off by default. Hermes streams a turn's visible text through a path that does not notify plugins when the turn ends in a tool call — most turns, for a coding agent — so reasoning deltas are the only ones a plugin reliably sees, and they are not forwarded until this is on. With it on, TTFT landed on 51 of 53 model calls in our dogfood session.

Hermes discovers the plugin through the hermes_agent.plugins entry point — there are no files to copy.

Enable via config.yaml, not hermes plugins enable latitude. Hermes's runtime loads pip/entry-point plugins, but its hermes plugins list/enable/disable commands scan only bundled and ~/.hermes/plugins/ directory plugins — so they report a pip-installed plugin as "not installed or bundled" even though it loads fine (hermes-agent#23802). The config.yaml entry above is the reliable way to turn it on.

Plugin not loading at all? It must be installed into the same Python that runs Hermes. The official installer puts Hermes in its own venv (~/.hermes/hermes-agent/venv) that ignores your shell's Python — so a plain pip install from another interpreter (system, pyenv, mise, …) won't be discovered. Install into Hermes's venv instead:

~/.hermes/bin/uv pip install --python ~/.hermes/hermes-agent/venv/bin/python latitude-telemetry-hermes

Set your credentials in the environment (or in ~/.hermes/.env, which Hermes loads at startup):

export LATITUDE_API_KEY=lat_xxx
export LATITUDE_PROJECT=my-project

That's it — every session is now streamed to Latitude as OTLP traces. To export structure and timing without prompt/response/tool content, set LATITUDE_NO_CONTENT=true. Secret redaction is on by default: content passes through Hermes's own redactor on the way out, so a token echoed by a terminal tool is masked before it leaves the machine.

Self-hosted / local Latitude? The plugin defaults to Latitude Cloud (https://ingest.latitude.so). Point it at your own ingest by setting LATITUDE_BASE_URL to the ingest origin only — e.g. http://localhost:3002 for a local dev stack — without the /v1/traces suffix (the plugin appends it). Your LATITUDE_API_KEY and LATITUDE_PROJECT must come from that same instance.

How it works

Hermes loads pip-installed plugins via the hermes_agent.plugins entry point and calls the module's register(ctx) function, which subscribes to the lifecycle hooks — pre_api_request / post_api_request / api_request_error, pre_llm_call / post_llm_call, pre_tool_call / post_tool_call, on_stream_start / on_stream_delta / on_stream_end, on_session_start / on_session_end / on_session_reset / on_session_finalize, and subagent_start / subagent_stop. The *_api_request family is the LLM-call span boundary; the *_llm_call pair frames the turn; the stream hooks supply time-to-first-token; the session hooks bound the memory read and flush the exporter so short / one-shot runs (hermes -z "…") ship before the process exits.

The plugin owns the OTLP mapping for that hook stream. It assembles one trace per turn:

interaction (one user turn)
├── search_memory  (what the agent remembered coming in, once per session)
├── llm_request    (one per model call: model, tokens, cost, TTFT, system prompt, tool definitions)
├── tool_call:*    (one per tool call: arguments, result, success, duration)
│   └── upsert_memory  (a memory write, under the call that made it)
└── tool_call:delegate
    └── interaction    (a delegated subagent, nested under the call that spawned it)

Spans follow Latitude's GenAI semantic conventions (gen_ai.*), so they render natively in the Latitude trace viewer. Content-bearing attributes are tagged :gated and dropped before export when content capture is disabled. Spans are shipped as they finish, coalesced into batches under a size ceiling, and retried on transient ingest errors. The plugin is fail-open — a telemetry error never affects the agent — and depends only on the Python standard library (plus certifi, which Hermes already ships, for TLS verification).

Hermes internals are imported only for what no hook exposes (memory paths, the tool snapshot, the secret redactor, cost status), always guarded and always with a working fallback, so a Hermes upgrade degrades one attribute rather than the plugin.

Design notes, hook payload traps, the usage reconciliation and the full attribute tables live in dev-docs/hermes-telemetry.md.

Configuration

Every setting is readable from two places, the environment winning: the env var below, or plugins.entries.latitude.settings.<key> in the active profile's config.yaml. Both config.yaml and .env are profile-scoped, so one profile per agent gives each agent its own credentials, tags and metadata. The ~/.hermes/… paths here are the default profile's; a named profile keeps its own pair under ~/.hermes/profiles/<name>/.

# ~/.hermes/config.yaml
plugins:
  enabled: [latitude]
  entries:
    latitude:
      settings:
        api_key: lat_xxx
        project: my-project
        agent: { name: alescript, version: 2.1.0 }
        tags: [prod, eu-west]
        metadata: { deployment: staging }

Telemetry stays off until both an API key and a project are set.

Derived tags (hermes, the platform, the agent name, the agent version, cron:<job>, subagent:<role>) make several agents in one project distinguishable, and make comparing two versions of one agent a single analytics breakdown or a two-variant experiment. See the public docs for the recipe.

Development

uv sync
uv run pytest

License

MIT