needlepath-langchain

Needlepath context selection for LangChain agents — and, in this same package, the LangGraph recipe.

pip install needlepath-langchain

One line

from langchain.agents import create_agent
from needlepath_langchain import NeedlepathMiddleware

agent = create_agent(
    model,
    tools,
    middleware=[NeedlepathMiddleware(operating_point="np-2026-07-r2")],
)

The key comes from NEEDLEPATH_API_KEY, the operating point is pinned explicitly, and if the service is slow, down, or stands down, the agent runs on its original context and says so in message metadata.

The two seams

wrap_tool_call — the sweet spot

The only documented, first-class, mutate-the-tool-result seam in the ecosystem. A tool returns 40 KB of JSON; what enters the message list is the part of it that answers the current step.

Engages only when a single tool result exceeds tool_result_max_tokens (default 2000), which is also the budget. Below it, no call is made and nothing is touched.

wrap_model_call — selection over the accumulated history

Before every model call, the tool results already in the history are selected against the current task. A result that mattered three steps ago and does not matter now collapses to a placeholder; the one that matters is excerpted verbatim. Engages only when the prunable history exceeds history_max_tokens (default 8000).

Non-destructive by construction: request.override(messages=…) returns a new request and never touches request.state, so the graph keeps the full history and only the model sees the selection.

Both seams are on by default and independently switchable (select_tool_results=, select_history=).

Configuration

Any other keyword is forwarded to the core client (base_url, api_key, timeout, max_retries, …).

Two invariants, and why they cost tokens

A message is never removed, only rewritten. LangGraph's _validate_chat_history raises when an AIMessage carrying tool_calls loses its matching ToolMessage, and every provider rejects the same shape:

ValueError: Found AIMessages with tool_calls that do not have a corresponding ToolMessage.

A selector that drops messages has to reason about tool-call pairing on every path and get it right every time. One that only rewrites content in place cannot break the pairing at all — the message objects, their ids and their tool_call_ids are exactly what they were. We take the second option and give up the savings from deleting message envelopes.

A rewrite never grows a message and never empties one. An excerpt longer than what it replaces, or an empty excerpt applied to a non-empty message, is refused here rather than trusted upstream.

By default only ToolMessages are rewritten. SystemMessage never is, on any setting. An AIMessage carrying tool_calls never is: its content is usually empty and the part that matters is the call list.

Fail open

Every error path passes the original context through, with the reason recorded in response_metadata["needlepath"]:

{
    "applied": False,
    "reason": "engine_fallback",
    "request_id": "np-…",
    "tokens_before": 8400,
    "tokens_saved": 0,
    "gate_reason": "standdown:flat_gap",
}

reason is an open enum. Treat an unrecognized value as "not applied".

middleware.stats.as_dict() aggregates the same counters — the input to a shadow report.

Sync and async

Both variants of both hooks are implemented, and that is not optional: create_agent puts a middleware in both the sync and async hook lists if it implements either, specifically so the base class's NotImplementedError fires. A middleware that implements only the sync hooks raises under ainvoke().

If you pass your own client= without an async_client=, the async hooks run it on the default executor rather than blocking the event loop.

Lifecycle

A middleware is normally wired up once and lives as long as the agent, so you never need to close it. If you construct one per request or per test, call middleware.close() / await middleware.aclose() to release the connection pools. Clients you passed in with client= / async_client= are yours and are never closed for you.

LangGraph

Same package, not a second distribution.

from langgraph.prebuilt import create_react_agent
from needlepath_langchain.langgraph import needlepath_pre_model_hook

agent = create_react_agent(
    model,
    tools,
    pre_model_hook=needlepath_pre_model_hook(operating_point="np-2026-07-r2"),
)

Prefer create_agent + NeedlepathMiddleware if you can. pre_model_hook exists only on create_react_agent, which is deprecated since LangGraph v1.0 and slated for removal in v2.0.

add_messages is the only writer of the messages channel and its removal path physically drops entries. Once it runs, they are gone from the live thread and every checkpoint after it. That is a history compaction primitive — what SummarizationMiddleware uses it for — not a selection primitive. Selection is per-call by definition: the record that does not matter at step 7 may be the one that matters at step 9.

Four sharp edges, all handled

1. llm_input_messages is a persisted channel, not a transient. It lands in the checkpoint, so every turn adds a full copy of the selection to the checkpoint blob. Real storage on long threads.
2. Stale-selection leak. It is a LastValue channel that survives between invoke() calls. A hook that conditionally skips writing it silently reuses the previous turn's selection. This hook writes it unconditionally, every pass.
3. An empty selection silently degrades to the full history. The reader does state.get("llm_input_messages") or state.get("messages"), and [] is falsy. This hook never emits an empty list.
4. _validate_chat_history runs on your selection. Covered by the never-remove-a-message invariant above.

One caveat this package cannot fix

With response_format on create_react_agent, the structured-output node reads state["messages"] directly and does not consult llm_input_messages. That call sees the full history, not your selection. Price your savings accordingly.

Positioning

LangChain ships ContextEditingMiddleware with a ClearToolUsesEdit strategy in-tree and free, which clears the oldest tool results once a token trigger trips. This is not a competitor to "can something prune tool results" — it is a different answer to which bytes should survive. We select verbatim against the current task rather than clearing by age, stand down when trimming would not pay, and report what was actually saved.

If clearing by age is enough for your workload, use the free one.

Tested against

langchain==1.3.14, langchain-core==1.5.3, langgraph==1.2.10. Floor >=1.2.9 (the first release where ToolCallRequest, ModelCallResult and ExtendedModelResponse are all importable from the public langchain.agents.middleware package and ModelRequest.system_message is stable). Capped at <2.0.0; CI runs against the newest 1.x minor.