thirdeye

Trace every agent session on your machine — Claude Code, Codex, Gemini, Cursor — into one history you and your agents can manage, search, and evaluate.

Install

brew install duncankmckinnon/tap/thirdeye    # macOS / Linux
pipx install thrdi                           # or: uv tool install thrdi

Enable tracing

thirdeye add --claude        # also: --cursor, --codex

To detach: thirdeye remove --claude.

Read your history

thirdeye list                          # every session, every platform
thirdeye events <id>                   # one session, terse
thirdeye tail <id> -n 5                # last few events
thirdeye event <id> <seq>              # one event, fully expanded
thirdeye search "migration"            # substring across all sessions
thirdeye stats                         # totals

Tag and filter

thirdeye tag <id> <seq> --add bug,review     # tag an event
thirdeye tag <id> --list                     # list tagged events in a session
thirdeye tag <id> <seq> --remove bug         # untag
thirdeye tags                                # global tag inventory
thirdeye search "migration" --tag review --platform claude --since 2026-05-01

Add --json for parseable JSONL, --tree for human-readable, --platform / --cwd / --tag / --since / --until to filter. Session IDs accept any unique prefix. Run thirdeye --help for the full reference.

Per-turn usage

thirdeye captures model name and token counts per turn into an append-only sidecar (usage.jsonl) and a global SQLite index (usage.db). Capture starts automatically on the next agent run after thirdeye add.

thirdeye usage                          # global rollup, sessions by token spend
thirdeye usage <id>                     # per-turn detail for one session
thirdeye usage --top 5 --since 2026-05-01
thirdeye usage <id> --json              # parseable JSONL rows
thirdeye usage reindex                  # rebuild SQLite from sidecars
thirdeye usage errors                   # tail the capture audit log

Filters: --platform / --harness, --model SUBSTR, --since / --until, --top N, --sort total|input|output|ts.

Export to Pydantic Logfire

Mirror every captured session into Logfire live, as traces — no separate export step. Once enabled, each thirdeye session becomes one Logfire trace: tool calls appear as spans with real durations (paired from PreToolUse/PostToolUse, or Codex's call_id), everything else (messages, notifications, compaction, ...) as timeline markers, all searchable by gen_ai.conversation.id.

On Claude Code, each individual model call within a turn gets its own chat <model> span. Codex uses a turn-level chat <model> span reconstructed from the completed turn in its rollout JSONL, with execute_tool <name> children for every paired tool call. Both use OpenTelemetry GenAI attributes for model, messages, and token usage; Codex repeated token reports are deduplicated by their cumulative watermark before turn totals are calculated.

pip install 'thrdi[logfire]'
thirdeye logfire enable --token pylf_v1_... --project my-project   # gateway key from Logfire
thirdeye logfire status
thirdeye logfire disable                                          # keeps the saved key

Or from thirdeye ui, under settings: paste the gateway key and project name and hit Enable — persisted the same way, in ~/.thirdeye/config.yaml.

Export is dispatched from the same Claude Code / Codex hooks that already capture events, but the actual Logfire call (including a flush, a real network round trip) runs in a detached background process — the hook itself never waits on the network, so enabling this adds no latency to your tool calls.

Browse in a browser

For a richer experience than the CLI, install the UI extra and launch:

pip install 'thrdi[ui]'
thirdeye ui

The local browser UI covers:

• Sessions list with platform / cwd / status / date filters and a tag multi-select drawn from every tag in your history, defaulting to the last 7 days, newest first.
• Ask panel — type "find sessions about the workbench plan" or "long-lasting claude runs this week" and a CLI agent of your choice (claude / codex / gemini) auto-fills the filter form. Review the populated fields and hit Search / Filter to run.
• Saved filter views — name a filter combination and pin it to the sidebar; restored across browser sessions via local storage.
• Session view — collapsible event tree color-coded by event type, inline tag editing, live-tail via Server-Sent Events for open sessions.
• Evals — author and edit YAML rubrics; dispatch a run on one session or a batch on a selection. Two complementary tables: per- definition cross-cut (/evals/defs/<name>/results) for comparing a rubric across sessions, and a per-(session, definition) panel showing the directive text plus every run on that session with parsed verdict and score columns.
• Usage charts — daily tokens-over-time and sessions-per-day with a platform filter and totals cards.

The server binds loopback only.

See docs/ui.md for full reference.

Evaluations

Grade a recorded session by dispatching one of your installed CLI agents (claude / codex / gemini) as an LLM-as-judge. Eval definitions are named rubrics — directive text shipped with sensible defaults and editable per-user.

thirdeye eval def list                                          # available rubrics
thirdeye eval def show default                                  # see the directive
thirdeye eval def create my-rubric --directive "<text>"         # custom rubric

thirdeye eval run <id> --agent claude                           # foreground
thirdeye eval run <id> --agent gemini --using token-efficiency --background

thirdeye eval show <id>                                         # latest result
thirdeye eval list --since 2026-05-01 --verdict warn            # history
thirdeye eval status                                            # background jobs

Per-turn findings are stored with the event seq they anchor to, and thirdeye events <id> annotates the timeline inline by default (suppress with --no-findings, filter with --eval NAME). The eval invocation itself is a thirdeye-traced session, so every grading run has its own audit trail.

Dispatched agents run in read-only mode (Claude --allowedTools allowlist, Codex --sandbox read-only, Gemini --approval-mode plan). No new Python deps — thirdeye shells out to the agent binaries you already have installed.

Agent

Dispatch an AI agent (Claude Code, Codex, or Gemini) against your thirdeye history directly from the CLI. The agent is pre-loaded with all built-in skills and runs in read-only mode by default.

thirdeye agent "review my sessions from the last week"
thirdeye agent "find sessions where token usage spiked" --stream
thirdeye agent "fix inefficient tool use in session abc123" --fix
thirdeye agent "summarize eval findings" --agent gemini

Flags:

New sessions opened by the agent are automatically tagged thirdeye-agent so you can filter them with thirdeye list --tag thirdeye-agent.

Agent skills

Four bundled skills are injected into every thirdeye agent run by default:

• use-thirdeye — basic CLI fluency: enable tracing, search sessions, debug tool calls, analyze token usage.
• thirdeye-evals — eval workflow: create rubrics, dispatch evaluators, view per-turn findings.
• thirdeye-review — audit other agents' traces to find inefficiencies and propose invocation, skill, or convention changes (cohort scoping, tool-mix patterns, token spikes, recommendation templates).
• thirdeye-filter — directive used by the browser UI's Ask panel to translate natural-language queries into filter JSON. Installed alongside the others; not invoked directly by agents.

Pass --skill path/to/skill.md to inject additional skills from local files alongside the defaults. Run thirdeye agent --skills to see the built-in list.

You can also install the skills into your project for use outside of thirdeye agent:

thirdeye skill list                                  # show bundled skill names
thirdeye skill install                               # all skills → .agents/skills/
thirdeye skill install .claude/skills                # custom parent dir
thirdeye skill install --only thirdeye-review        # just one
thirdeye skill install --force                       # replace existing entries

Skills install as symlinks, so upgrading thirdeye (brew upgrade thirdeye or pipx upgrade thrdi) automatically refreshes the skill content in every repo where they're installed.

License

MIT.