thrdi 0.2.2

Trace every agent session on your machine — Claude Code, Codex, Cursor, GitHub Copilot CLI — into one history you and your agents can manage, search, and evaluate.

Install

Windows support is experimental. The test suite runs on Windows in CI, and Claude Code tracing is the verified integration there. The Codex CLI, Cursor, and Copilot CLI installers are implemented but have not been live-certified against those tools on Windows. Please report Windows problems at the issue tracker. See docs/windows.md for the deliberate platform differences.

The easiest installation approach is to use homebrew. This will package all thirdeye extras as a globally available CLI tool. Homebrew is macOS / Linux only — on Windows, install with pipx or uv (below).

brew install duncankmckinnon/tap/thirdeye    # macOS / Linux

Alternatively, you can get the same result with pipx

pipx install 'thrdi[ui,logfire]' # global install

For local installation, you can also use uv or pip with the optional ui and logfire extras that add the ability to navigate thirdeye as a webpage and export traces via OpenTelemetry to Pydantic Logfire.

uv tool install thrdi # or: uv tool install 'thrdi[ui,logfire]'

pip install thrdi # or: pip install 'thrdi[ui,logfire]'

Interactive setup

Configure tracing, bundled agent skills, and optional Pydantic Logfire export in one guided flow:

thirdeye setup

The wizard prompts for each supported agent, installs skills in the matching agent directories, and can sign in to Logfire in the browser to mint a write token if you choose to enable remote export. The commands below remain available when you want to configure each piece separately.

Install agent skills locally

Install thirdeye's bundled skills for agents working in the current project:

thirdeye skills list                                 # show bundled skill names
thirdeye skills add                                  # all skills → .agents/skills/
thirdeye skills add --claude --codex                 # both agent-specific folders
thirdeye skills add -p path/to/folder                # custom parent folder
thirdeye skills add --only thirdeye-review           # just one
thirdeye skills add --force                          # replace existing entries

On macOS and Linux, skills install as symlinks, so upgrading thirdeye (brew upgrade thirdeye or pipx upgrade thrdi) automatically refreshes them. On Windows without Developer Mode, skills are copied instead; after upgrading, rerun thirdeye skills add --force to refresh them. See Windows support for details.

Enable tracing

thirdeye add --claude        # also: --cursor, --codex, --copilot

To detach: thirdeye remove --claude (also --cursor, --codex, --copilot).

Copilot CLI capture

GitHub Copilot CLI capture keeps a durable, local V1 archive and derives V2 views from that archive. V1 source records are immutable: V2 never rewrites them, and it can rebuild its derived turns, usage accounting, and export queue after the original Copilot files have gone away.

thirdeye add --copilot
thirdeye copilot status --source-home "$COPILOT_HOME"
thirdeye copilot sync --source-home "$COPILOT_HOME"
thirdeye copilot reconcile
thirdeye copilot reconcile --session-id <stored-session-id> --rebuild
thirdeye copilot watch --source-home "$COPILOT_HOME" --interval 1
thirdeye remove --copilot

--source-home is optional. Resolution is --source-home, then COPILOT_HOME, then ~/.copilot. Capture reads only that home's session-state/**/events.jsonl, workspace.yaml, and session-store.db (sessions, turns, assistant_usage_events). It does not read credentials, token-bearing config, or other Copilot files.

thirdeye add --copilot writes user-level hooks at $COPILOT_HOME/hooks/thirdeye.json (default ~/.copilot/hooks/thirdeye.json). The 1.0.83 live probe used repository hooks; user-level installation is not a claim that every Copilot runtime invokes them. watch is an explicit foreground poller and is not started by add or setup. It can import transcripts and SQLite rows even when hooks are absent. sync and status print recoverable source diagnostics (locations and reasons, not prompt bodies).

Normal sync and reconcile refresh local V2 projections only; they do not initialize export eligibility. To mark already-completed retained history as export-eligible, opt in explicitly:

thirdeye copilot sync --export --source-home "$COPILOT_HOME"
thirdeye copilot reconcile --export

Watch and hook follow-up activate live-style export without that history opt-in. Their first activation records a durable eligibility boundary even if remote export is not presently configured: interactions already complete stay local-only, while interactions still open at activation may be queued later if they complete and remote export is configured. --export instead activates with retained completed history included, or later removes those identities from an existing boundary. Jobs are dispatched only when remote export is configured. --rebuild resets only reproducible derived state; it preserves the raw archive and the separate delivery ledger. See Copilot CLI capture and reconciliation for the archive schema, attribution rules, correction behavior, and delivery limits.

Passing unit tests is not live certification of Copilot CLI, native VS Code, or Copilot cloud integrations.

Read your history

thirdeye list                          # every session, every platform
thirdeye events <id>                   # one session, terse
thirdeye tail <id> -n 5                # last few events
thirdeye event <id> <seq>              # one event, fully expanded
thirdeye search "migration"            # substring across all sessions
thirdeye stats                         # totals

Tag and filter

thirdeye tag <id> <seq> --add bug,review     # tag an event
thirdeye tag <id> --list                     # list tagged events in a session
thirdeye tag <id> <seq> --remove bug         # untag
thirdeye tags                                # global tag inventory
thirdeye search "migration" --tag review --platform claude --since 2026-05-01

Add --json for parseable JSONL, --tree for human-readable, --platform / --cwd / --tag / --since / --until to filter. Session IDs accept any unique prefix. Run thirdeye --help for the full reference.

Per-turn usage

thirdeye captures model name and token counts per turn into an append-only sidecar (usage.jsonl) and a global SQLite index (usage.db). Capture starts automatically on the next agent run after thirdeye add.

thirdeye usage                          # global rollup, sessions by token spend
thirdeye usage <id>                     # per-turn detail for one session
thirdeye usage --top 5 --since 2026-05-01
thirdeye usage <id> --json              # parseable JSONL rows
thirdeye usage reindex                  # rebuild SQLite from sidecars
thirdeye usage errors                   # tail the capture audit log

Filters: --platform / --harness, --model SUBSTR, --since / --until, --top N, --sort total|input|output|ts.

Export to Pydantic Logfire

Mirror every captured session into Logfire live, as traces — no separate export step. Once enabled, each thirdeye session becomes one Logfire trace: tool calls appear as spans with real durations (paired from PreToolUse/PostToolUse, or Codex's call_id), everything else (messages, notifications, compaction, ...) as timeline markers, all searchable by gen_ai.conversation.id.

On Claude Code, each individual model call within a turn gets its own chat <model> span. Codex reconstructs calls from its rollout JSONL. Cursor reconstructs each generation from its IDE/CLI hooks, pairing shell and MCP callbacks and recording file and generic tool events. All three use OpenTelemetry GenAI semantic conventions for agent invocation, chat, tool execution, messages, models, and token/cache usage; no OpenInference span taxonomy is used.

Within a trace, subagents dispatched in local Cursor IDE and CLI sessions are exported beneath their dispatching Task span — including background, parallel, and nested children — and each child's tool calls are attributed to it by the child hook generation derived from its Task call identity, never by timing, nearest-turn heuristics, or tool-completion order.

thirdeye logfire enable                                           # reuse a saved token, or sign in to mint one
thirdeye logfire enable --auth                                    # force a fresh Logfire login, then mint
thirdeye logfire status
thirdeye logfire disable                                          # keeps the saved key

Or from thirdeye ui, under settings: paste the gateway key and hit Enable — persisted the same way, in ~/.thirdeye/config.yaml.

Export is dispatched from the same Claude Code, Codex, and Cursor hooks that already capture events, but the actual Logfire call (including a flush, a real network round trip) runs in a detached background process — the hook itself never waits on the network, so enabling this adds no network latency to your tool calls.

From the sessions page, you can also send the currently filtered sessions to Logfire as a named managed dataset. Configure a separate project API key with project:write_datasets scope under Settings → Pydantic Logfire, apply the session filters you want, enter a dataset name, and choose Send to Logfire. Each session becomes one case containing its metadata and ordered event stream. The managed-datasets feature must be enabled for the Logfire project. Choose one case per turn to export each captured user-to-assistant turn as its own case. A turn-content query searches every turn in every session selected by the broader filters; comma-separated terms are ANDed within the same turn. An optional exact selector in the form <session-id>:<platform-turn-id> remains available for direct lookup.

Exporting environment context

Set THIRDEYE_CAPTURE_ENV=WB_* and individual variables such as WB_PLAN, WB_AGENT, WB_TASK, WB_STEP, and WB_WAVE on the agent process. Thirdeye captures matching variables into each export job and attaches them to every exported span for Claude Code, Codex, and Cursor. For example:

THIRDEYE_CAPTURE_ENV='WB_*' WB_PLAN=my-plan WB_TASK=task-1 claude

The patterns can also be persisted so they do not depend on the launching shell exporting THIRDEYE_CAPTURE_ENV (an agent dispatched from a context that never sources your shell rc would otherwise capture nothing):

thirdeye capture-env set 'WB_*'    # writes capture_env to ~/.thirdeye/config.yaml
thirdeye capture-env show          # what is active, and where it came from

THIRDEYE_CAPTURE_ENV still overrides the persisted value when set, so a one-off run can change it.

Each captured name becomes a dotted span-attribute key — lowercased, with _ mapped to . — so WB_PLAN is wb.plan and BUILD_LABEL is build.label. No pattern is privileged; the transform is uniform. Values remain strings with their original case and contents; nonempty values are also exported directly as Logfire tags on the top-level session span only, with duplicates removed. Child spans retain the named attributes without repeating these session tags. For example, WB_PLAN=my-plan adds the my-plan tag alongside wb.plan="my-plan". Logfire tags preserve the original case and contents. Local session tags retain their existing key prefixes, sanitization, and 64-character limit. Existing span attributes take precedence on a name collision.

Patterns match variable names, not fields inside a composite value. This path requires no OTEL_RESOURCE_ATTRIBUTES, which was observed to be absent from Claude Code hook environments. Workbench must supply the individual WB_* variables in each child's environment; changing thirdeye alone cannot extract values the launcher does not supply. Export jobs snapshot context before worker dispatch, keeping concurrent agents' values separate. Only opt in variables you intend to export to Logfire.

Browse in a browser

For a richer experience than the CLI, install the UI extra and launch:

thirdeye ui      # or: thirdeye serve

The local browser UI covers:

• Sessions list with platform / cwd / status / date filters and a tag multi-select drawn from every tag in your history, defaulting to the last 7 days, newest first.
• Ask panel — type "find sessions about the workbench plan" or "long-lasting claude runs this week" and a CLI agent of your choice (claude / codex) auto-fills the filter form. Review the populated fields and hit Search / Filter to run.
• Saved filter views — name a filter combination and pin it to the sidebar; restored across browser sessions via local storage.
• Session view — collapsible event tree color-coded by event type, inline tag editing, live-tail via Server-Sent Events for open sessions.
• Evals — author and edit YAML rubrics; dispatch a run on one session or a batch on a selection. Two complementary tables: per- definition cross-cut (/evals/defs/<name>/results) for comparing a rubric across sessions, and a per-(session, definition) panel showing the directive text plus every run on that session with parsed verdict and score columns.
• Usage charts — daily tokens-over-time and sessions-per-day with a platform filter and totals cards.

The server binds loopback only.

See docs/ui.md for full reference.

Evaluations

Grade a recorded session by dispatching one of your installed CLI agents (claude / codex) as an LLM-as-judge. Eval definitions are named rubrics — directive text shipped with sensible defaults and editable per-user.

thirdeye eval def list                                          # available rubrics
thirdeye eval def show default                                  # see the directive
thirdeye eval def create my-rubric --directive "<text>"         # custom rubric

thirdeye eval run <id> --agent claude                           # foreground
thirdeye eval run <id> --agent codex --using token-efficiency --background

thirdeye eval show <id>                                         # latest result
thirdeye eval list --since 2026-05-01 --verdict warn            # history
thirdeye eval status                                            # background jobs

Per-turn findings are stored with the event seq they anchor to, and thirdeye events <id> annotates the timeline inline by default (suppress with --no-findings, filter with --eval NAME). The eval invocation itself is a thirdeye-traced session, so every grading run has its own audit trail.

Dispatched agents run in read-only mode (Claude --allowedTools allowlist, Codex --sandbox read-only). No new Python deps — thirdeye shells out to the agent binaries you already have installed.

Agent

Dispatch an AI agent (Claude Code or Codex) against your thirdeye history directly from the CLI. The agent is pre-loaded with its analysis and evaluation skills and runs in read-only mode by default.

thirdeye agent "review my sessions from the last week"
thirdeye agent "find sessions where token usage spiked" --stream
thirdeye agent "fix inefficient tool use in session abc123" --fix
thirdeye agent "summarize eval findings" --agent codex

Flags:

New sessions opened by the agent are automatically tagged thirdeye-agent so you can filter them with thirdeye list --tag thirdeye-agent.

Skills used by thirdeye agent

Four bundled skills are injected into every thirdeye agent run by default:

• use-thirdeye — basic CLI fluency: enable tracing, search sessions, debug tool calls, analyze token usage.
• thirdeye-evals — eval workflow: create rubrics, dispatch evaluators, view per-turn findings.
• thirdeye-review — audit other agents' traces to find inefficiencies and propose invocation, skill, or convention changes (cohort scoping, tool-mix patterns, token spikes, recommendation templates).
• thirdeye-filter — directive used by the browser UI's Ask panel to translate natural-language queries into filter JSON. Installed alongside the others; not invoked directly by agents.

Pass --skill path/to/skill.md to inject additional skills from local files alongside the defaults. Run thirdeye agent --skills to see the default list.

License

MIT.