graphsight-langgraph 0.1.0

Capture LangGraph agent runs as Graphsight traces — see exactly what your agent retrieved and why.

graphsight-langgraph

See exactly why your LangGraph agent picked the context it picked.

One callback handler captures a LangGraph run — every node, every retriever call, per-document scores, and (for graph-aware retrievers) the relational paths between retrieved entities — as a framework-neutral AgentTrace. One function maps it to a JSON file the Graphsight Studio renders with zero backend.

your LangGraph agent ──▶ LangGraphTracer ──▶ AgentTrace (v0.1) ──▶ to_tracestate() ──▶ Studio /memory/import
                          (callbacks)         neutral JSON           Studio JSON          full canvas render

This package is standalone. It does not require the TraceRAG engine, database, or SaaS stack — its only dependency is langchain-core.

Install

pip install -e .            # the tracer (depends only on langchain-core)
pip install -e ".[example]" # + langgraph, to run the demo agent

Python ≥ 3.10. Verified against langchain-core 1.5.0 + current langgraph.

Quickstart

from graphsight_langgraph import LangGraphTracer, to_tracestate

tracer = LangGraphTracer()
result = graph.invoke(inputs, config={"callbacks": [tracer]})

trace = tracer.finish(
    query="why is checkout failing?",
    answer=result["answer"],
)

trace.to_dict()          # framework-neutral AgentTrace (schema v0.1)
to_tracestate(trace)     # Studio-ready JSON → drop into /memory/import

That's the whole API surface: construct, pass as a callback, finish(), map.

⚠ The one gotcha: config propagation

LangChain only propagates callbacks into runnables that receive the run's config. Inside a LangGraph node, pass the node's config through to sub-runnables, or the tracer will see the node but not what happened inside it:

def retrieve(state, config):                               # ① accept config
    docs = retriever.invoke(state["q"], config=config)     # ② pass it through
    return {"docs": docs}

If your trace shows node spans but no retrievals, this is why.

Try it in 60 seconds (offline, no API keys)

python example/demo_agent.py

This builds and runs a real two-node LangGraph agent (retrieve → answer) over an in-memory "codebase memory" corpus — a PR, a service, a person, and a ticket, connected by edges — and traces it:

answer   : The regression traces to PR #4821 in checkout-service, authored by Priya N. …
spans    : 3 (retrieve, CodebaseMemoryRetriever, answer)
retrieved: 4 items · 4 edges · arm=graph
latency  : 3.9ms

Two files land in example/out/:

File	What it is
agent_trace.json	the neutral AgentTrace — what any consumer would ingest
trace_state.json	Studio-ready — drag-and-drop (or paste) into http://localhost:5173/memory/import

In the Studio you'll see the full canvas: four typed entity nodes with score chips, the traced relational path (person → PR → service, PR → ticket), the execution timeline with per-span timings, and the retriever's arm.

Tip: with graphsight installed, skip the manual drop — graphsight example/out/trace_state.json serves the UI locally and opens the trace directly.

Trace your own GitHub repo

The package ships a dead-simple ingest CLI — one command from repo to trace:

graphsight-github-trace langchain-ai/langgraph "who fixed the recent streaming bugs?"
graphsight graphsight_out/trace_state.json

It fetches recent PRs + issues (public repos need no token; --token / GITHUB_TOKEN for private), builds a corpus with relational edges (person AUTHORED pr, pr RESOLVES issue, pr TOUCHES repo), and runs a traced two-node LangGraph agent over it. Retrieval is two-arm in miniature: lexical seeds + 1-hop graph expansion along the edges — simple on purpose, and the scores shown are exactly the lexical scores computed, nothing dressed up as semantic. Requires the example extra (langgraph).

What gets captured

Source in the run	Captured as
Each LangGraph node execution	Span(kind="node") with monotonic-clock timing
Framework internals (RunnableSequence, ChannelWrite, __start__, …)	filtered out — one span per user node, no noise
on_retriever_end documents	one RetrievedItem per doc: id, label, kind, score, content, source
Document.metadata score keys (score, relevance_score, similarity, …)	item scores
Document.metadata["edges"] = [{source, target, relation, weight}]	relational edges (deduplicated)
LLM / tool calls	plain spans in the execution timeline
Edges present in a retrieval?	arm = "graph", else "vector" — auto-detected

Honest degradation — stated plainly

• Vector-only retriever (no edges in metadata) → the Studio renders a flat scored-retrieval view instead of relational path highlighting. Still useful; just not graph-aware.
• No score keys → scores stay None and no score chips render. Nothing is ever fabricated.
• The emitted confidence.rationale explicitly says scores came from your retriever and Graphsight does not recompute them — an imported trace never masquerades as an engine-computed one.

Schema (v0.1)

AgentTrace is the stable contract; future adapters (LlamaIndex, raw OTel) emit the same shape.

{
  "schema_version": "0.1",
  "framework": "langgraph",
  "query": "…",
  "spans": [
    { "id": "…", "name": "retrieve", "kind": "node",       // node | retriever | llm | tool | chain
      "parent_id": null, "start_ms": 0.0, "end_ms": 0.3, "status": "ok" }
  ],
  "retrievals": [
    {
      "span_id": "…", "query": "…",
      "arm": "graph",                                       // "vector" | "graph" (auto: edges present?)
      "items": [
        { "id": "pr_4821", "label": "PR #4821", "kind": "pull_request",
          "score": 0.94, "vector_score": null, "graph_score": null,
          "content": "…", "source_uri": "https://…", "metadata": {} }
      ],
      "edges": [                                            // optional — the graph-aware view
        { "source": "pr_4821", "target": "svc_checkout", "relation": "TOUCHES", "weight": 0.9 }
      ]
    }
  ],
  "answer": "…",
  "latency_ms": 3.9
}

to_tracestate() maps this onto the Studio's TraceState (frontend/src/types/trace.ts). Free-form kind strings are normalized to Studio entity types (pull_request → PR, service → Service, person/author → Person, ticket/issue/jira → Ticket, … fallback Document). Node positions are recomputed client-side by the Studio's layout engine, so emitters never deal with coordinates.

Package layout

graphsight-langgraph/
├── graphsight_langgraph/
│   ├── schema.py         # AgentTrace v0.1 — zero-dependency dataclasses
│   ├── tracer.py         # LangGraphTracer (BaseCallbackHandler)
│   ├── mapper.py         # to_tracestate() — AgentTrace → Studio TraceState
│   └── github_ingest.py  # `graphsight-github-trace` CLI (repo → trace)
├── example/
│   └── demo_agent.py     # offline end-to-end demo (writes example/out/*.json)
└── pyproject.toml

Status & roadmap

v0.1 — verified end-to-end against langchain-core 1.5.0 + current langgraph: spans, scored items, edges, and arm detection all captured from a real run and rendered in the Studio.

Known limits, honestly:

• Score-key coverage is the pragmatic short list above; real-world retrievers vary, and unrecognized keys degrade to None (visible, not wrong).
• Edges are read from a documented metadata convention (metadata["edges"]) — retrievers must opt in to the graph-aware view.
• Async (ainvoke / astream) uses the same callback events but hasn't been exercised in the verification run yet.

Planned next, in order: LlamaIndex adapter emitting the same AgentTrace, then a raw OTel span ingestor. The schema is the contract; the adapters are thin.