_{virtual-context cloud: a 3 million token virtual window served at 80K real tokens}

virtual-context

Virtual memory for LLM context. Give your agent a context window of tens or hundreds of millions of tokens; the model only ever sees the part that matters.

Your client sets contextWindow: 20000000 (or 200,000,000). Your model's real window is 200K. virtual-context sits between them and makes it work, the same way an operating system lets a process address more memory than physically exists. The client keeps sending its whole conversation; virtual-context stores all of it, organizes it by topic, and forwards the model a compact window of what matters for the current turn, at a size you choose. Nothing is discarded, and anything older can be paged back in at full fidelity. The dashboard above shows a live 3 million token conversation served at 80K real tokens.

It runs as an HTTP proxy, so integration is a base-URL change; a Python SDK and an MCP server are there for direct use.

Why

• Memory that lasts. What the user said at turn 12 is still recallable at turn 1,000, across sessions, clients and models.
• Better answers from less context. A focused window of relevant summaries and facts beats a raw window of everything, where the one fact you need is buried.
• Lower cost. Smaller payloads, arranged to keep providers' prompt caches warm.
• Group chats that know who said what. Every message keeps its sender, facts keep their author, and search can be scoped to one person.
• One memory everywhere. Build context in Claude Code, continue it from Telegram, query it from Cursor.

How it works

client ──> virtual-context proxy ──> model provider
             │   before the call: retrieve topics and facts for this turn,
             │                    assemble a bounded window
             │   after the call:  store the turn, tag it, compact when needed
             v
           store (SQLite or PostgreSQL)
             canonical turns   every message verbatim, with sender and channel
             segments + facts  per-topic summaries and structured facts
             topic summaries   one digest per topic, the bird's-eye view

Recent turns stay verbatim. Older turns are grouped by topic, summarized and mined for facts. Each turn, the model gets recent turns plus the topics and facts relevant to the question, and a topic index telling it what else it can open with built-in tools (vc_expand_topic, vc_find_quote, vc_recall_all, and others).

Quick start

pip install "virtual-context[proxy,embeddings]"    # Python 3.11+

virtual-context uses a language model of your choice to tag turns and write summaries. Pick where it runs:

# A local model server (Ollama, LM Studio, llama.cpp, vLLM): nothing leaves your machine
virtual-context init recommended-local

# Or a hosted model through OpenRouter
virtual-context init recommended
export OPENROUTER_API_KEY=...

Either writes virtual-context.yaml; set tag_generator.model and summarization.model to the model you want. Then start the proxy and point your client at http://127.0.0.1:5757 instead of the provider:

virtual-context proxy --upstream https://api.anthropic.com

A dashboard runs at http://127.0.0.1:5757/dashboard.

The two presets are the hosted service's configuration on one machine and differ only in where the model runs: SQLite storage, paging tools on, fact curation on. Answer quality depends on the model you pick. scripts/smoke_recommended.sh checks a preset end to end from a clean install. The optional parts (PostgreSQL vector search, the judgment layer, group attribution) are each a section of docs/configuration.md; virtual-context presets list shows every preset, and docs/install.md covers install options and running it as a service.

The proxy accepts Anthropic Messages, OpenAI Responses, OpenAI Chat Completions and Gemini requests. The paging tools are injected for Anthropic, OpenAI Responses and Gemini; Chat Completions clients get retrieved context but no tools.

Hosted: virtual-context.com runs the same engine as a service with the dashboard and cost reports; you change your base URL and nothing else.

Integrations

Claude Code

export ANTHROPIC_BASE_URL=http://127.0.0.1:5757

Tool output (file reads, searches, command results) is collapsed into restorable stubs, and when Claude Code trims its own history, the missing turns are restored from storage.

Any client with a base URL (Cursor, Continue, your own app)

client = anthropic.Anthropic(base_url="http://127.0.0.1:5757")
client = OpenAI(base_url="http://127.0.0.1:5757/v1")

OpenClaw: use the OpenClaw plugin. Settings for direct proxy use are in docs/install.md.

Python SDK

from virtual_context import VirtualContextEngine

engine = VirtualContextEngine(config_path="./virtual-context.yaml")
assembled = engine.on_message_inbound(message=user_message, conversation_history=messages)  # before the LLM call
report = engine.on_turn_complete(messages)                                                  # after it

MCP server: eight tools (recall_context, compact_context, expand_topic, recall_all, remember_when, find_quote, search_summaries, domain_status) plus topic resources, for Claude Desktop, Cursor or any MCP client.

Features

Commands in any conversation

Type these as ordinary messages in a connected client; the proxy handles them without calling the model.

VCATTACH is a durable redirect: nothing is deleted and old references keep resolving, so two clients (or two agents) can share one memory. Details: docs/commands.md.

Running it in production

• Storage: SQLite by default; PostgreSQL (pip install "virtual-context[postgres]") for multi-worker deployments. With pgvector installed, virtual-context admin migrate-semantic-vectors prepares in-database vector search.
• Multi-worker safety: compactions run under leased, fenced operations, lifecycle changes are epoch-guarded, and a backlog sweeper compacts conversations whose traffic never triggers it inline.
• Operator tooling: virtual-context admin has 27 guarded backfill, repair and migration commands with dry-run modes.
• Security: dashboard endpoints are open until you set VC_DASHBOARD_TOKEN; the default bind is loopback only.
• Observability: per-stage timing logs, request captures in the dashboard, and per-call cost telemetry.

Architecture, including the canonical-turn model and the REST prepare/ingest surface: docs/architecture.md.

Benchmarks

On 100 questions from LongMemEval-500, with the same reader model (Claude Sonnet 4.5):

These are historical results: the original run's provenance is incomplete and they are not a measurement of the current pipeline. Configuration, results by category, the per-question table and how new runs are recorded: docs/benchmarks.md.

Documentation

Development

git clone https://github.com/virtual-context/virtual-context.git
cd virtual-context
python -m venv .venv && source .venv/bin/activate
python -m pip install uv==0.9.30
uv sync --locked --extra all --extra dev
.venv/bin/python scripts/check_contracts.py

License

AGPL-3.0, Copyright Y. Ahmed Kidwai. For commercial licensing: ahmed@kidw.ai