needlepath-langchain

Needlepath context selection for LangChain agents — and, in this same package, the LangGraph recipe.

pip install needlepath-langchain

One line

from langchain.agents import create_agent
from needlepath_langchain import NeedlepathMiddleware

agent = create_agent(
    model,
    tools,
    middleware=[NeedlepathMiddleware(operating_point="np-2026-08-r4")],
)

The key comes from NEEDLEPATH_API_KEY, the operating point is pinned explicitly, and if the service is slow, down, or stands down, the agent runs on its original context and says so in message metadata.

The two seams

wrap_tool_call — the sweet spot

The only documented, first-class, mutate-the-tool-result seam in the ecosystem. A tool returns 40 KB of JSON; what enters the message list is the part of it that answers the current step.

Engages only when a single tool result exceeds tool_result_max_tokens (default 2000), which is also the budget. Below it, no call is made and nothing is touched.

wrap_model_call — selection over the accumulated history

Before every model call, the tool results already in the history are selected against the current task. A result that mattered three steps ago and does not matter now collapses to a placeholder; the one that matters is excerpted verbatim. Engages only when the prunable history exceeds history_max_tokens (default 8000).

Non-destructive by construction: request.override(messages=…) returns a new request and never touches request.state, so the graph keeps the full history and only the model sees the selection.

Both seams are on by default and independently switchable (select_tool_results=, select_history=).

Configuration

Parameter	Default	What it does
operating_point	—	Required. Immutable label. Also NEEDLEPATH_OPERATING_POINT.
shadow	False	Measure and report; never apply.
enabled	True	Kill switch. Needs no credentials when False.
select_tool_results / select_history	True	The two seams.
tool_result_max_tokens	2000	Trigger and budget for one tool result.
history_max_tokens	8000	Trigger and budget for the history.
include_ai_messages / include_human_messages	False	Widen beyond tool results.
placeholder	see source	Replaces an unselected tool result.
name	class name	Must be unique if you register two instances.

Any other keyword is forwarded to the core client (base_url, api_key, timeout, max_retries, …).

Two invariants, and why they cost tokens

A message is never removed, only rewritten. LangGraph's _validate_chat_history raises when an AIMessage carrying tool_calls loses its matching ToolMessage, and every provider rejects the same shape:

ValueError: Found AIMessages with tool_calls that do not have a corresponding ToolMessage.

A selector that drops messages has to reason about tool-call pairing on every path and get it right every time. One that only rewrites content in place cannot break the pairing at all — the message objects, their ids and their tool_call_ids are exactly what they were. We take the second option and give up the savings from deleting message envelopes.

A rewrite never grows a message and never empties one. An excerpt longer than what it replaces, or an empty excerpt applied to a non-empty message, is refused here rather than trusted upstream.

By default only ToolMessages are rewritten. SystemMessage never is, on any setting. An AIMessage carrying tool_calls never is: its content is usually empty and the part that matters is the call list.

Fail open

Every error path passes the original context through, with the reason recorded in response_metadata["needlepath"]:

{
    "applied": False,
    "reason": "engine_fallback",
    "request_id": "np-…",
    "tokens_before": 8400,
    "tokens_saved": 0,
    "gate_reason": "standdown:flat_gap",
}

reason is an open enum. Treat an unrecognized value as "not applied".

No preserve_recent — what the engine returns is what gets applied

An earlier version had a preserve_recent option ("never rewrite the last N messages") and it was removed (see CHANGELOG.md): a client-side tail protection is a selection decision made locally, and it made .stats misreport what was actually rewritten whenever the engine's own selection favored the protected window. Passing preserve_recent today raises at construction, naming the migration.

Two stats objects, two truths

middleware.stats reports what this middleware actually did to the message(s) it touched — real before/after token counts. middleware.engine_stats reports what the engine's raw response claimed, unmodified. They differ because the engine cannot see the placeholder text this middleware substitutes for a record it declined to select. Use .stats for anything downstream that has to be true of the message list; use .engine_stats to reconcile against what the service itself measured or billed. In shadow mode .stats.tokens_saved is always 0 (nothing is ever applied); .engine_stats.tokens_saved carries the engine's prediction.

Sync and async

Both variants of both hooks are implemented, and that is not optional: create_agent puts a middleware in both the sync and async hook lists if it implements either, specifically so the base class's NotImplementedError fires. A middleware that implements only the sync hooks raises under ainvoke().

If you pass your own client= without an async_client=, the async hooks run it on the default executor rather than blocking the event loop.

Lifecycle

A middleware is normally wired up once and lives as long as the agent, so you never need to close it. If you construct one per request or per test, call middleware.close() / await middleware.aclose() to release the connection pools. Clients you passed in with client= / async_client= are yours and are never closed for you.

LangGraph

Same package, not a second distribution.

from langgraph.prebuilt import create_react_agent
from needlepath_langchain.langgraph import needlepath_pre_model_hook

agent = create_react_agent(
    model,
    tools,
    pre_model_hook=needlepath_pre_model_hook(operating_point="np-2026-08-r4"),
)

Prefer create_agent + NeedlepathMiddleware if you can. pre_model_hook exists only on create_react_agent, which is deprecated since LangGraph v1.0 and slated for removal in v2.0.

Mechanism	Graph state	Verdict
RemoveMessage → add_messages	destroys the removed messages	incompatible with selection
pre_model_hook → llm_input_messages	messages untouched	compatible
create_agent + wrap_model_call	nothing written at all	preferred

add_messages is the only writer of the messages channel and its removal path physically drops entries. Once it runs, they are gone from the live thread and every checkpoint after it. That is a history compaction primitive — what SummarizationMiddleware uses it for — not a selection primitive. Selection is per-call by definition: the record that does not matter at step 7 may be the one that matters at step 9.

Four sharp edges, all handled

1. llm_input_messages is a persisted channel, not a transient. It lands in the checkpoint, so every turn adds a full copy of the selection to the checkpoint blob. Real storage on long threads.
2. Stale-selection leak. It is a LastValue channel that survives between invoke() calls. A hook that conditionally skips writing it silently reuses the previous turn's selection. This hook writes it unconditionally, every pass.
3. An empty selection silently degrades to the full history. The reader does state.get("llm_input_messages") or state.get("messages"), and [] is falsy. This hook never emits an empty list.
4. _validate_chat_history runs on your selection. Covered by the never-remove-a-message invariant above.

One caveat this package cannot fix

With response_format on create_react_agent, the structured-output node reads state["messages"] directly and does not consult llm_input_messages. That call sees the full history, not your selection. Price your savings accordingly.

Positioning

LangChain ships ContextEditingMiddleware with a ClearToolUsesEdit strategy in-tree and free, which clears the oldest tool results once a token trigger trips. This is not a competitor to "can something prune tool results" — it is a different answer to which bytes should survive. We select verbatim against the current task rather than clearing by age, stand down when trimming would not pay, and report what was actually saved.

If clearing by age is enough for your workload, use the free one.

Tested against

langchain==1.3.14, langchain-core==1.5.3, langgraph==1.2.10. Floor >=1.2.9 (the first release where ToolCallRequest, ModelCallResult and ExtendedModelResponse are all importable from the public langchain.agents.middleware package and ModelRequest.system_message is stable). Capped at <2.0.0; CI runs against the newest 1.x minor.