Caveman Python middleware

Native framework adapters over caveman_cloud.middleware. Your framework keeps its models, tool loop, retries, streams and stored conversation. Caveman only changes the text of selected tool results in the outbound model view, and only when the local runtime answers in time.

Alpha. Install the extra for the framework you use:

pip install "caveman-middleware[langchain]"

from caveman_cloud.middleware import MiddlewareRuntime, Scope
from caveman_middleware.langchain import with_caveman_agent
from langchain.agents import create_agent

runtime = MiddlewareRuntime(endpoint="http://127.0.0.1:8787")
runtime.ready()
scope = Scope("my-app", session_id="conversation-1", branch_id="main", cache_epoch="0")

agent = create_agent(**with_caveman_agent(existing_agent_options, runtime=runtime, scope=scope))

Call runtime.close() at shutdown. Nothing in your stored message history is rewritten; only the request handed to the provider is.

Supported frameworks

One extra per adapter. The range is the band the adapter's version gate accepts; the left-hand value is the version its wire handling was written against.

Outside its range an adapter keeps the caller's native input unchanged and reports unsupported_version. It never raises.

Extras carry no provider pins, so most combine: pip install "caveman-middleware[langchain,agno,litellm]" resolves. Three pairs cannot, and the cause is upstream, not this package: strands-agents and crewai both require mcp<2.2 while the MCP adapter needs >=2.2, and agno and crewai disagree over litellm. Install those families in separate environments.

What you lose

Read this before turning on compress.

• Markers are scoped. A replacement marker is bound to namespace + session_id + branch_id + cache_epoch and to the authenticated principal that created it. Replaying it under any other scope returns nothing.
• Markers expire. A scope lives 24 hours, renewed each time it is used. After that the stored original is released.
• Persisted compressed history is a trap. If you save the transformed messages and replay them later — under a different scope, or after the scope expired — that text is gone for good. Caveman does not rewrite your stored history for this reason: keep the originals, let Caveman transform the outbound copy on every call.
• Recovery needs a registered executor. In compress mode a lossy transform is only used when your framework really holds the caveman_retrieve tool. Without it, only recovery-free transforms apply.
• Eligible content is narrow. Successful tool-result text and explicitly passed document bodies. Never system prompts, user messages, assistant reasoning, errored tool results, images or other non-text parts, anything marked protected, and never a payload the adapter cannot parse exactly.
• Failure is silent and safe. Runtime down, over deadline, out of capacity, expired scope: the original request goes to the provider and the decision is reported as skipped. Set strict=True only if you want a pre-inference error instead.

Reports

on_report= receives one immutable record per native call: status, reason, transform ids, replacement and reuse counts. No original content, no provider credentials. runtime.last_report holds the most recent one. A callback that raises cannot affect inference.

Token counts in reports are estimates from the runtime's tokenizer over the segments it saw. They are not measured billing savings, and this package does not claim a reduction figure.