graphsight-langgraph 0.1.1

X-ray for LangGraph retrieval: one callback handler captures every retriever call, score, and relational path — rendered as an interactive graph.

graphsight-langgraph

See exactly why your LangGraph agent picked the context it picked.

Retrieval in an agent is a black box: documents go in, an answer comes out, and when the answer is wrong you're left guessing which context misled it. This package opens the box. One callback handler records a LangGraph run — every node, every retriever call, per-document scores, and (for graph-aware retrievers) the relational paths between retrieved entities — and one command renders it as an interactive graph in your browser.

your LangGraph agent ──▶ LangGraphTracer ──▶ AgentTrace (v0.1) ──▶ graphsight viewer
                          (callbacks)         neutral JSON          interactive graph

Only dependency: langchain-core. No engine, no backend, no account — nothing leaves your machine.

Install

pip install graphsight-langgraph            # the tracer
pip install "graphsight-langgraph[example]" # + langgraph, for the CLI & demo
pip install graphsight                      # the local viewer (recommended)

Python ≥ 3.10. Verified against langchain-core 1.5.0 + current langgraph.

60-second demo: trace a real GitHub repo

No setup, no API keys — public repos don't even need a token:

graphsight-github-trace langchain-ai/langgraph "who fixed the recent streaming bugs?"
graphsight graphsight_out/trace_state.json

Your browser opens on a live graph of that repo's recent activity: the PRs that matched your question, the people who authored them, the issues they resolve — every node clickable, scores shown, execution timeline included.

Under the hood it fetches recent PRs + issues, builds a corpus with relational edges (person AUTHORED pr, pr RESOLVES issue, pr TOUCHES repo), and runs a traced two-node LangGraph agent over it. Ranking is lexical seeds + 1-hop graph expansion — simple on purpose, and labeled as such: the scores you see are exactly the scores computed, nothing dressed up as semantic. Private repos: pass --token or set GITHUB_TOKEN.

Trace your own agent

The whole API is four lines:

from graphsight_langgraph import LangGraphTracer, to_tracestate

tracer = LangGraphTracer()
result = graph.invoke(inputs, config={"callbacks": [tracer]})

trace = tracer.finish(query="why is checkout failing?", answer=result["answer"])
to_tracestate(trace)     # viewer-ready JSON — save it, then: graphsight trace.json

trace.to_dict() gives you the framework-neutral AgentTrace if you'd rather feed your own tooling.

⚠ The one gotcha: config propagation

LangChain only propagates callbacks into runnables that receive the run's config. Inside a LangGraph node, pass the node's config through to sub-runnables, or the tracer will see the node but not what happened inside:

def retrieve(state, config):                               # ① accept config
    docs = retriever.invoke(state["q"], config=config)     # ② pass it through
    return {"docs": docs}

If your trace shows node spans but no retrievals, this is why.

What gets captured

Source in the run	Captured as
Each LangGraph node execution	Span(kind="node") with monotonic-clock timing
Framework internals (RunnableSequence, ChannelWrite, __start__, …)	filtered out — one span per user node, no noise
on_retriever_end documents	one RetrievedItem per doc: id, label, kind, score, content, source
Document.metadata score keys (score, relevance_score, similarity, …)	item scores
Document.metadata["edges"] = [{source, target, relation, weight}]	relational edges (deduplicated)
LLM / tool calls	plain spans in the execution timeline
Edges present in a retrieval?	arm = "graph", else "vector" — auto-detected

Honest degradation — stated plainly

• Vector-only retriever (no edges in metadata) → you get a flat scored-retrieval view instead of relational path highlighting. Still useful; just not graph-aware.
• No recognized score keys → scores stay None and no score chips render. Nothing is ever fabricated.
• The emitted confidence.rationale explicitly says scores came from your retriever and Graphsight does not recompute them — an imported trace never masquerades as an engine-computed one.

Schema (v0.1)

AgentTrace is the stable contract; future adapters (LlamaIndex, raw OTel) emit the same shape and render in the same viewer.

{
  "schema_version": "0.1",
  "framework": "langgraph",
  "query": "…",
  "spans": [
    { "id": "…", "name": "retrieve", "kind": "node",       // node | retriever | llm | tool
      "parent_id": null, "start_ms": 0.0, "end_ms": 0.3, "status": "ok" }
  ],
  "retrievals": [
    {
      "span_id": "…", "query": "…",
      "arm": "graph",                                       // "vector" | "graph" (auto: edges present?)
      "items": [
        { "id": "pr_4821", "label": "PR #4821", "kind": "pull_request",
          "score": 0.94, "vector_score": null, "graph_score": null,
          "content": "…", "source_uri": "https://…", "metadata": {} }
      ],
      "edges": [                                            // optional — the graph-aware view
        { "source": "pr_4821", "target": "svc_checkout", "relation": "TOUCHES", "weight": 0.9 }
      ]
    }
  ],
  "answer": "…",
  "latency_ms": 3.9
}

Free-form kind strings are normalized to viewer entity types (pull_request → PR, service → Service, person/author → Person, ticket/issue/jira → Ticket, … fallback Document). Node positions are computed client-side, so emitters never deal with coordinates.

Offline example agent

The repo ships a fully offline two-node agent (retrieve → answer, no API keys) that demonstrates the whole loop, including edge-carrying documents:

git clone https://github.com/Kcodess2807/graphsight
cd graphsight/graphsight-langgraph
python example/demo_agent.py
graphsight example/out/trace_state.json

spans    : 3 (retrieve, CodebaseMemoryRetriever, answer)
retrieved: 4 items · 4 edges · arm=graph
latency  : 3.9ms

Status & roadmap

v0.1 — verified end-to-end against langchain-core 1.5.0 + current langgraph: spans, scored items, edges, and arm detection all captured from a real run and rendered in the viewer.

Known limits, honestly:

• Score-key coverage is the pragmatic short list above; unrecognized keys degrade to None (visible, not wrong).
• Edges are read from a documented metadata convention (metadata["edges"]) — retrievers must opt in to the graph-aware view.
• Async (ainvoke / astream) uses the same callback events but hasn't been exercised in the verification run yet.

Planned next, in order: LlamaIndex adapter emitting the same AgentTrace, then a raw OTel span ingestor. The schema is the contract; the adapters are thin.

Links

• Source & issues: github.com/Kcodess2807/graphsight
• The viewer: graphsight on PyPI
• Trying the beta? BETA.md