opd-viz 0.1.0

Live per-token distillation-loss (SDPO) heatmap server for any vLLM-served chat model

opd-viz

Live per-token distillation-loss (SDPO) heatmap server for any vLLM-served chat model.

opd-viz visualizes the gold-hint SDPO per-token distillation loss: for each rollout token it compares the student-policy prediction against the gold-procedure-reprompted teacher prediction as a top-k KL, and colors that divergence onto the realized tokens in a browser heatmap. It is model-agnostic (you pick the endpoint, model, and tokenizer at runtime from the browser) and brings-your-own-MCP (register any MCP server for the agentic generate loop).

Install / run

Zero-checkout, via uv:

uvx --from opd-viz opd-viz --port 8088

or into an environment:

pip install opd-viz
opd-viz --port 8088

Then open http://127.0.0.1:8088. The server binds loopback and is single-user; there is no auth on the opd-viz server itself.

Flags: --host (default 127.0.0.1), --port (default 8088). Everything else is set in the browser.

Requirements

• A served, OpenAI-compatible vLLM endpoint. It must be started with --max-logprobs 100 so the teacher-forcing pass can return top-100 prompt_logprobs; opd-viz preflights this on Connect and reports a clear error if it is missing.

• The tokenizer matching your model is pulled from the Hugging Face Hub by the model id at Connect time. For gated repos, set HF_TOKEN in the environment before launching:

  HF_TOKEN=<paste your hf_… token> uvx --from opd-viz opd-viz
  

  (Reference a secret manager rather than pasting tokens into shared shells.)

Use it (connect, generate, recompute)

1. Connect. Fill in the vLLM endpoint (e.g. http://host:port/v1), optional API key, the model id (e.g. org/Model-Name), an optional system prompt, and sampling params (temperature plus any of top_p / top_k / min_p / presence_penalty / repetition_penalty; blank fields are not sent). Click Connect — this loads the matching tokenizer, runs the vLLM preflight, and runs a chat-template self-check, surfacing any warnings. Changing the model and clicking Connect again re-pulls the matching tokenizer and re-preflights; no restart needed.
2. Generate. Type a question and click Generate rollout. Without tools this is a single assistant turn (the student = its own tokens, so gen == score holds). The generation logprobs are the student distribution.
3. Recompute. Paste the gold hindsight (and optionally tweak the reprompt template), then click Recompute & visualize. The exact generated token ids are teacher-forced under the gold-hint prompt via vLLM prompt_logprobs, and the per-token KL (JSD / reverse / forward) is colored onto the realized tokens. Hover any token for the student/teacher top-k and the top-1 (dis)agreement.

A rollout is stamped with the config that produced it; if you reconnect with a different model/config and then score a stale rollout, the server refuses (regenerate instead of mis-coloring).

MCP tools (optional, agentic)

Register MCP servers in the Connect panel to make their tools available to an agentic generate loop:

• remote: a URL (e.g. https://host/mcp) plus an optional auth header.
• stdio: a command + args (+ optional KEY=val env), e.g. uvx some-mcp-server.

After connecting, the use tools toggle enables once at least one MCP tool is discovered. With it on, Generate runs the model through tool calls (executed live) until it answers without a tool call; scoring colors each assistant turn and greys the tool results.

opd-viz contacts only your vLLM endpoint, the MCP servers you register, and the HF Hub (for the tokenizer). Nothing else.