recall-py 0.1.1

Local-first conversation cache, RAG, and routing to reduce cloud LLM spend.

---

RecallPy is a Python library and CLI that helps you cut cloud LLM costs by caching conversations, retrieving relevant context via RAG, and routing queries to local models when appropriate. It integrates natively with Cursor, Claude Code, and any OpenAI-compatible client.

Key features

• RAG memory — every conversation is chunked, embedded, and stored in SQLite for semantic retrieval
• Local draft answering — uses a cheap local model (Ollama) to answer common questions
• Intelligent routing — auto-escalates to cloud models when local confidence is low
• OpenAI-compatible proxy — drop-in caching + compression for existing clients
• IDE-native MCP tools — handle_query, ingest_turn, escalate_pack for Cursor / Claude Code
• Token savings tracking — see exactly how many cloud tokens you avoided

Quick start

python3.12 -m pip install recall-py
recall-py onboard
recall-py doctor

Library usage

import sqlite3
from recall_py.engine import handle_query, ingest_turn
from recall_py.ollama_client import OllamaClient
from recall_py.settings import AppSettings

settings = AppSettings.load()
conn = sqlite3.connect(settings.resolved_db_path())
ollama = OllamaClient(settings.ollama)

result = await handle_query(
    conn, settings, ollama,
    query="How does the routing policy work?",
    thread_id=None,
)

# result.context_pack.citations has the relevant context

await ingest_turn(
    conn, settings, ollama,
    thread_id=result["thread_id"],
    role="assistant",
    content="The routing policy...",
    title="my-session",
    workspace_fingerprint=result["workspace_fingerprint"],
    provider="my-app",
)

CLI

Command	Purpose
recall-py onboard	First-time setup: config, DB, Ollama, MCP snippet
recall-py serve	HTTP API (health + optional proxy)
recall-py mcp-stdio	MCP server for IDE integration
recall-py doctor	Check DB, migrations, Ollama
recall-py metrics	Token savings overview
recall-py index	Index workspace docs into RAG memory

Documentation

Full documentation is available at rehanpunjwani.github.io/TokenGuard.

Installation options

See the Getting Started guide for pipx, uv, and source installs.

Requirements

• Python 3.12+
• Ollama with nomic-embed-text and llama3.2 (configurable)

Architecture

RecallPy stores every user and assistant message in SQLite, chunks them, and embeds each chunk via Ollama. On each query, it retrieves the most semantically relevant chunks and optionally drafts a local answer. When the local draft is too uncertain, it signals escalation — you paste the context pack into your cloud model instead. All token operations are tracked for savings reporting.

Read the architecture docs

Docker

bash scripts/docker-up.sh

Starts RecallPy + Ollama with a single command. See docker-compose.yml.

Contributing

See CONTRIBUTING.md for guidelines and the Development guide for technical setup.

License

MIT