llmly 0.1.1

Interactive viewer for Copilot and Codex coding-session token usage

llmly

Interactive web viewer for VS Code Copilot and Codex coding-session token usage. It reads local Copilot debug logs and Codex rollout logs, then turns each session into a small-multiple chart of cumulative token use over time, with a drill-in modal showing every LLM call and the tool invocations that fed it.

Copilot data comes from the debug logs that the Copilot extension writes under VS Code's workspaceStorage directory (<workspaceStorage>/<wsid>/GitHub.copilot-chat/debug-logs/). Codex data comes from CODEX_HOME (or ~/.codex), using sessions/**/*.jsonl for per-turn usage and state_5.sqlite for thread titles, source metadata, and spawned-subagent parentage.

The log location is auto-detected per platform:

Platform	workspaceStorage
macOS	~/Library/Application Support/Code/User/workspaceStorage
Linux	$XDG_CONFIG_HOME/Code/User/workspaceStorage (default ~/.config/...)
Windows	%APPDATA%\Code\User\workspaceStorage

Using VS Code Insiders, VSCodium, or a portable install? Point the COPILOT_USAGE_STORAGE env var at the equivalent workspaceStorage directory. Using a non-default Codex home? Point CODEX_USAGE_HOME or CODEX_HOME at that directory.

Visuals

These are the same annotated examples from the in-app guided tour (the ? Help button in the toolbar), which renders a live demo session — one main agent that spawned two sub-agents.

Small multiples, one per session

The default view: a grid of cards, one per session, on a shared y-scale so the heaviest hitter fills the frame. Click any card to drill into its detail.

Anatomy of a chart

Each card is one session. The big band plots cumulative input tokens over time; the band below plots per-turn input. Dot size ∝ that turn's input; dot color = cache hit rate.

Detailed view, session

Click any card or row to open the per-turn detail — every LLM call, its reasoning level, input/cached/output, cache-hit bar, and the tools that fired before that turn.

Table rollup — a day's worth of usage

The table toggle swaps the grid for a sortable rollup, one row per thread: model, reasoning level, request and sub-agent counts, input/cached/uncached/output tokens, cost, and duration. Click any column header to re-sort.

Usage calendar

The 📅 calendar is a GitHub-style year heatmap. In Copilot-only mode it shows daily AIC spend, and the unit toggle switches between AIC credits and dollars. In Codex or mixed-source mode it shows daily input tokens, because Codex rollout logs do not expose exact AIC or dollar cost.

Colors & marks

Run

Requires uv.

From PyPI:

uvx llmly

The command starts the local Flask server and opens the viewer in your browser. If port 5057 is busy, llmly automatically tries the next available port.

From a checkout:

git clone https://github.com/byronwall/data-viz-copilot-usage.git
cd data-viz-copilot-usage
uv run app.py

Then open http://localhost:5057 if the browser does not open automatically.

CLI flags:

uvx llmly --port 8000 --host 0.0.0.0 --debug
uvx llmly --no-open

uv will create .venv/ and install Flask on the first run; subsequent runs are instant.

Publish

Build and publish with uv:

./publish-llmly.sh

The script loads .env, increments the patch version, runs checks, builds the wheel/sdist, and publishes. Use PYPI_TOKEN for production PyPI and TEST_PYPI_TOKEN for TestPyPI:

./publish-llmly.sh test
./publish-llmly.sh prod

Test

Install dev dependencies and run the unit suite with:

uv run pytest

The tests live in tests/ and use hermetic fixtures for Codex state instead of reading live local agent history.

What you see

• Default view: top 50 sessions from the last 24h, sorted by total input tokens.
• Filters in the header: data source (all / Copilot / Codex), time window (1h … 90d), sort key, top-N cap, minimum-token floor.
• Each card: a small line chart. y = cumulative input tokens (shared scale across all cards in the result set so the heaviest hitter fills the frame). x = wall-clock time from first activity (per-card scale). Solid blue = the foreground panel/editAgent chat. Dashed colored lines = sub-agents (runSubagent-*). Orange diamonds = compaction events (summarizeConversationHistory*, summarizeVirtualTools). Dot size encodes per-turn input tokens. Dot color encodes cache hit on that turn (blue ≥70% / amber 30–70% / red <30%).
• Click a card → fullscreen modal with the chart on the left and a per-turn detail table on the right. Hover any dot to highlight (and scroll to) the matching row, and vice versa. Each row shows debugName, reasoning level, input/cached/output, cache-hit bar, and the tools that fired before that turn.
• charts / table toggle (below the controls): switch the result set between the small-multiples grid and a tabular rollup — one row per thread (session), with columns for model, reasoning level, request count, input/cached/cache%/output tokens, available cost, and duration. Click a column header to sort; click a row to open the same detail modal as a card. The active view persists in the URL (?view=table).
• Reasoning level is the requested effort, not a token count. Copilot reads it from each request's requestOptions; Codex reads it from the rollout turn_context.effort. Codex reasoning-output token counts are included in detail payloads, but not shown as a separate default table column.
• Codex subagents are folded into parent Codex threads by default using thread_spawn_edges; guardian/internal review threads are hidden from the default Codex view.

Endpoints

• GET / — UI
• GET /api/sessions?source=all&since_hours=24&min_tokens=0&limit=50&sort=total_input — list of session summaries (no per-tool detail)
• GET /api/session/<source:sid> — full detail for one session (every LLM call and its preceding tool invocations)
• GET /api/daily_usage?source=all — daily selected metric (AIC for Copilot-only, input tokens for Codex/all)
• GET /api/stats?source=all&since_hours=24 — high-level rollup

source values: all · copilot · codex. sort values: total_input · recent · uncached · requests · duration · aic (aic maps to usage for Codex, since exact cost is unavailable).

How it works

analyzer.py discovers Copilot main.jsonl files via the workspaceStorage glob. For each session it parses the foreground log plus any sibling *.jsonl files (those are child sessions — runSubagent-* and title-*). For Codex, it uses state_5.sqlite as the thread index and parses rollout JSONL token_count events for per-turn usage. Files are cached in-memory keyed by mtime where useful, and daily Codex token totals are disk-cached by rollout mtime/size. The frontend renders SVG directly in the browser from the JSON payload so filtering is responsive.

For the implementation write-up and the repeatable process for adding more agent sources, see docs/agent-source-integration-guide.md.

Notes

• VS Code Copilot only logs caching info for some sessions/models. gemini-3-flash-preview shows ~17% cache; gpt-5.x typically 88–95%.
• Sessions reopened across multiple days will have a duration that includes the idle gap.
• "Find relevant code snippets for: …" sessions are standalone subagent search sessions (separate session dirs, no parent linkage in the log files), so they show up as their own cards.