ToolRecall — Deterministic Tool Cache for LLM Agents

Your agent reads the same file 10 times in a session. Each read goes to disk, returns to the LLM, and inflates your context window. That's 10× the tokens for the same content.

ToolRecall sits between your agent and the OS (or your API provider). On repeat calls it serves cached results from local SQLite instead of re-executing commands or re-sending requests. Byte-identical outputs mean every API call qualifies for provider prefix-caching discounts (up to 90% at Anthropic/OpenAI).

1 tick instead of 4: A file read normally needs stat → open → read → close. ToolRecall needs only stat (mtime check) — on cache hit the bytes come from memory, bypassing disk entirely.

⚠️ Best fit: stateless & open-source agents (Hermes, OpenCode, Cline, Aider, herdr)

ToolRecall excels where agents have limited context budgets and benefit from deterministic cache + MCP multiplexing. If you run Claude Code or Codex CLI, the shim and MCP bridge can cause stale-state issues — those agents manage their own in-memory tool tracking natively. See Agent Compatibility.

Zero pip dependencies. Python 3.11+ stdlib only. ~132 KB install. Everything starts automatically.

pipx install toolrecall
toolrecall setup          # One-shot: config → systemd → shim → daemon start
# Done — every agent on this machine now benefits

Zero config mode: After toolrecall setup, every command like toolrecall status, toolrecall mcp, or toolrecall serve auto-starts the daemon if it isn't running. You never need to think about it.

What It Does

Dynamic commands (git, ls, curl) and state-changing operations always execute live.

Measured effect

In a 13-hour session (Hermes + Gemini 3.1 Pro, 386 messages, 13 project files):

• 89% hit rate (91% file cache): 827 tool calls served from SQLite instead of OS
• 73% fewer file-read tokens at 3× re-read (~204K → ~55K unique)
• ~81% fewer at 10× re-read (~630K → ~55K unique)
• ~20 min less wait time — each cache hit avoids ~1.5s subprocess fork
• Provider prefix-caching becomes reliable: byte-identical payloads qualify for Anthropic/OpenAI's up-to-90% discount on every call

Real-agent debug loop (10 turns, 5 writes): A Hermes agent fixing bugs in ToolRecall's own code shows 36.4% input token savings — 63,326 input tokens without TR → 40,270 with TR. Write-invalidation resets the cache on every edit, so savings are lower than read-only benchmarks (98%+) but reflect actual edit-heavy sessions. At 50 turns with the same write frequency, estimated savings climb to ~68%. Full methodology.

Source: Benchmark

Agent Integration — zero-config for any agent

ToolRecall's daemon provides three agent-agnostic caching layers. None require per-agent configuration.

Layer 1: Python Shim (transparent, any Python agent)

After toolrecall setup, Python processes with the .pth shim installed auto-cache open() and subprocess.run() through ToolRecall. Hermes, Aider, Cline, Google ADK — all benefit once the shim is active (toolrecall shim --install).

pipx install toolrecall
toolrecall setup              # One-shot: shim + daemon
toolrecall shim --install     # Enable .pth shim (opt-in)
# Done — every Python process now transparently caches

Node.js agents (Claude Code, Codex CLI, OpenCode) are unaffected by the shim — see Agent Compatibility for their recommended integration.

Layer 2: MCP Bridge (any MCP-compatible agent)

Connect any MCP agent by registering one server. The same config works for all agents.

// ~/.claude/settings.json  or  ~/.cursor/mcp.json  or  ~/.config/cline/mcp_settings.json
// or any other MCP agent config
{
  "mcpServers": {
    "toolrecall": {
      "command": "toolrecall",
      "args": ["mcp"]
    }
  }
}

For OpenCode (v1.17+), toolrecall setup writes this automatically to ~/.opencode/opencode.jsonc:

// ~/.opencode/opencode.jsonc
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "toolrecall": {
      "type": "local",
      "command": "toolrecall",
      "args": ["mcp"],
      "enabled": true
    }
  }
}

Hermes Agent: Hermes already ships with ToolRecall built in — the tools cached_read, cached_terminal, mcp_call, etc. are available directly in your toolset.

Aider:

aider --mcp-toolrecall

All agents share one daemon and one cache — no duplication, no conflict.

Layer 3: Go Client (tr binary) — for any language or shell

For OpenCode, Claude Code, Codex CLI, herdr panes, or any non-Python agent: The tr binary connects directly to the ToolRecall daemon over UDS. Cached file reads, terminal commands, and status checks — all from the shell, no Python runtime needed.

tr read main.py            # Cached file read
tr cat /etc/os-release     # Alias for read
tr term "hostname"         # Cached terminal command
tr status                  # Daemon health & cache stats
tr ping                    # Fast connectivity check
tr read --bypass file.py   # Force fresh read
tr read --refresh file.py  # Alias for bypass
tr write /tmp/test.txt "hello"  # Write (invalidates cache)

Use it when: herdr panes (every agent in any pane uses tr directly), CI/CD pipelines, Rust/Ruby/Java agents, any shell script.

# Build from source
cd go-client && go build -o /usr/local/bin/tr .

See Go Client for full details.

⚠️ Claude Code users: Adding ToolRecall as an MCP server can cause stale-state issues in code edit loops. See Agent Compatibility before configuring.

Architecture

flowchart TB
    subgraph EntryPoints["Entry Points"]
        S["Python Shim<br/>open() → UDS"]
        B["MCP Bridge<br/>stdio → UDS"]
        F["Forward Proxy<br/>HTTP :8569"]
    end
    subgraph Daemon["ToolRecall Daemon"]
        LRU["In-Memory LRU"]
        SQ["SQLite Cache"]
        MP["MCP Multiplexer"]
        SG["Security Gate"]
        CT["Context Tracker"]
    end
    subgraph OSLayer["OS Layer"]
        O["Filesystem / Disk / Network"]
    end

    S --> Daemon
    B --> Daemon
    F --> Daemon
    Daemon --> O
    LRU <--> SQ

Shim layer (at the OS level): When tr_shim.pth is in site-packages, Python processes auto-patch builtins.open() and subprocess.run() — no imports needed. Hermes, Aider, Cline transparently benefit. (Claude Code, Codex CLI, and OpenCode are Node.js — the Python shim doesn't apply.)

Daemon layer (process level): Holds the hybrid in-memory LRU + SQLite WAL cache, the MCP Multiplexer (manages subprocesses for external MCP servers), the Forward Proxy (caches full API responses via body hash), and the Security Gate (path allowlist, sensitive file blocklist, cognitive scan, AST injection check).

How they work together:

1. Python process calls open("file.py") → Shim intercepts → cached_read() via Daemon UDS → returns cached bytes or reads from disk
2. Agent calls cached_read() via MCP → Daemon → same cache (shared with Shim)
3. Any SDK sends API request to localhost:8569 → Forward Proxy hashes body → checks same SQLite cache

MCP Multiplexer

When running multiple agents on the same machine (5 Claude Code sessions + 3 Cursor instances), each one normally spawns its own subprocess for every MCP server (GitHub, Postgres, time…). That's 10× the RAM for the same tool.

The daemon's multiplexer shares one subprocess per server across all agents:

• Lazy loading: servers boot on first call, not at daemon start (~0.01s vs ~1.7s per server)
• Idle timeout: inactive subprocesses killed after 15 min (configurable)
• Failure isolation: one server crash doesn't affect others (auto-reconnect, max 3 attempts)
• Secrets: API tokens loaded from ~/.toolrecall/.env, never exposed to the LLM
• Auto-resolution: Server names auto-resolve from the built-in registry — no command/args needed for common servers

All agents connect to one MCP server in their config: toolrecall mcp.

Quick Config Example

# ~/.config/toolrecall/toolrecall.toml
[mcp_multiplex]
servers = ["time", "github", "fetch"]

Built-in Servers (zero deps)

External Servers (needs uvx)

See MCP Multiplexer for full configuration details.

One-Time Setup

ToolRecall should be installed once per machine, then it works transparently for all agents.

pipx install toolrecall         # installs CLI + Shim (.pth file activates on next Python start)
toolrecall setup                # config → systemd service → shim → daemon start

That's it. Now opt-in Python processes (with the .pth shim installed) transparently cache file reads and terminal commands through ToolRecall. To enable the shim: toolrecall shim --install.

What toolrecall setup does

What happens on every CLI command

Every toolrecall command that needs the daemon (status, mcp, serve, stats, etc.) automatically:

1. Checks if the shim is installed — auto-installs it if missing
2. Checks if the daemon is running — auto-starts it if not

This means you can run toolrecall status on a fresh install and it "just works" — no extra steps.

Daemon auto-start (fallback chain)

Forward Proxy (API-level caching)

Cache API responses before they leave your machine. The forward proxy starts automatically with the daemon — no extra command needed.

# Point any OpenAI-compatible SDK at the forward proxy
export OPENAI_BASE_URL=http://localhost:8569/v1

Supported providers: OpenAI, Anthropic, Google Gemini, DeepSeek, xAI, Mistral, Groq, Together, OpenRouter. See Forward Proxy docs for the full provider list and usage examples.

FTS5 Knowledge Base — Query via MCP or HTTP

The SQLite FTS5 index built by toolrecall index is queryable by the agent itself:

• MCP tool (active when MCP bridge is connected): mcp_toolrecall_docs_search(query="...") — returns BM25-ranked results with snippets
• HTTP endpoint (active when Forward Proxy is running): GET http://localhost:8569/__docs/search?q=<query> — returns JSON, any HTTP-speaking client can use it

This means the agent can search its own cached docs, memory stores, and indexed files without leaving the tool loop. Index with toolrecall index. See Knowledge DB.

Security

ToolRecall doesn't prevent prompt injection — it cages the consequences:

• Default-deny path allowlist: Without config, NO paths are readable. toolrecall init prompts for paths interactively.
• Sensitive file blocklist: .env, .ssh/, .pem, .aws/, etc. are blocked even inside allowed paths.
• allow_terminal (default: false): allows read-only commands matching the regex allowlist (27 patterns for ls, cat, git status, etc.). Set true to enable terminal caching.
• os.path.realpath(): catches ../../../etc/shadow traversal before OS is touched.
• Cognitive Pre-Fight: Deterministic regex scan on MCP tool arguments for override instructions, jailbreak tags, exfiltration URLs. Zero LLM, ~0.001ms hot path.
• AST injection check: Parses tool arguments as Python AST — blocks exec(), eval(), __import__() calls.
• Daemon IPC via UDS: No open ports (POSIX), immune to SSRF. The forward proxy listens on TCP :8569 for HTTP API caching — intentional, separate from daemon transport.
• Fail-closed fallback: If the daemon is unreachable, the client refuses gated operations (terminal, unrestricted reads) instead of silently allowing them.

See Security Architecture for the full trust boundary.

Quick Reference — CLI

toolrecall setup          One-shot: config + systemd service + shim + daemon start  [required once]
toolrecall init           Create default config.toml and .env
toolrecall status         Cache status and stats               [auto-starts daemon]
toolrecall stats          Detailed cache statistics (JSON)     [auto-starts daemon]
toolrecall invalidate     Clear all caches                     [auto-starts daemon]
toolrecall restart        Health check + clean daemon restart  [auto-starts daemon]
toolrecall mcp            Start MCP Bridge                     [auto-starts daemon]
toolrecall serve          Forward proxy (cache API responses)  [auto-starts daemon]
toolrecall serve --port 9000  Forward proxy on custom port
toolrecall debug          Start debug/demo server              [auto-starts daemon]
toolrecall index          Build/update FTS5 knowledge database [auto-starts daemon]
toolrecall config-set     Set a config value
toolrecall daemon         Start/stop/manage cache daemon
toolrecall shim           Install/uninstall OS-level cache shim (.pth file)
toolrecall nginx          Generate nginx config

Configuration

TOML (stdlib tomllib) or YAML (optional, requires pyyaml).

# ~/.config/toolrecall/toolrecall.toml (created by toolrecall init)
[norm]
# Cache key normalization (v0.9.0) — deterministic JSON sorting + noise stripping.
# When enabled, tool call arguments are normalized before cache key generation:
# keys sorted, whitespace stripped, timestamps/session IDs removed.
# This broadens cache hits when agents rephrase or reorder arguments.
# ⚠️ Changes existing cache keys — existing entries become orphans.
enabled = false

[mcp]
allowed_paths = ["/home/user/projects"]  # Add your project dirs — default-deny!
allow_terminal = false
allow_invalidate = false

[cache]
# Terminal cache default TTL (seconds) — commands matching the terminal
# command allowlist will be cached for this duration.
terminal_default_ttl = 60

[mcp_multiplex]
enabled = true
servers = ["time", "sequential-thinking"]

[forward_proxy]
# Forward proxy starts on :8569 automatically with the daemon

TOOLRECALL_* environment variables override TOML.

Platform Support

Contributing

git clone https://github.com/whiskybeer/toolrecall.git
cd toolrecall
make setup      # one-time: install dev deps
make test       # run tests
make check      # lint + format check

See the Testing Guide and Makefile for all targets.

Uninstall

toolrecall shim --uninstall          # Remove .pth from site-packages
systemctl --user stop toolrecall-daemon
systemctl --user disable toolrecall-daemon
pipx uninstall toolrecall
rm -rf ~/.toolrecall ~/.config/toolrecall

Documentation

• Agent Compatibility — per-agent value, config, and caveats
• Architecture — daemon design, layers, IPC
• Architecture Diagram — system and sequence diagrams, token costs, Context Tracker
• CLI Reference — all subcommands explained
• Configuration Reference — config.toml, config.py, all env vars
• Context Tracker — checkpoint-based dirty-file tracking, O(n²) breakdown
• How It Works — quick technical overview
• MCP Multiplexer — single-daemon MCP management, server registry
• Testing Guide — test philosophy, organization, per-file coverage
• Benchmark — measured performance, token savings
• Real-Agent Debug Loop — edit-heavy session benchmark
• Knowledge DB — FTS5 indexing guide
• Normalizer — cache key normalization, deterministic JSON sorting
• Replay Mode — record/replay tool calls for deterministic CI testing
• Docker Deployment — containerized stack
• Forward Proxy — cache API responses by body hash, provider list, usage
• Security Architecture — WAF details, trust boundary
• Troubleshooting — common fixes
• Appendix — comparison tables, OSI model, ROI, vision, audit
• Hermes Transparent Cache — auto-patching for Hermes Agent
• Framework Adapters:
  • Google ADK — @cached_tool decorator + forward proxy + runtime patch
  • LangChain / LangGraph — ToolRecallCache BaseCache + callback handler
  • herdr — tr binary + MCP bridge for any agent in any pane