_{virtual-context cloud: a 3 million token virtual window served at 80K real tokens}

virtual-context

Virtual memory for LLM context. Give your agent a context window of tens or hundreds of millions of tokens; the model only ever sees the part that matters.

Your client sets contextWindow: 20000000 (or 200,000,000). Your model's real window is 200K. virtual-context sits between them and makes it work, the same way an operating system lets a process address more memory than physically exists. The client keeps sending its whole conversation; virtual-context stores all of it, organizes it by topic, and forwards the model a compact window of what matters for the current turn, at a size you choose. Nothing is discarded unless someone removes it on purpose, with a forget command, an admin edit or by deleting the conversation, and anything older can be paged back in, down to its full transcript. The dashboard above shows a live 3 million token conversation served at 80K real tokens.

It runs as an HTTP proxy, so integration is a base-URL change; a Python SDK and an MCP server are there for direct use.

Why

• Memory that lasts. What the user said at turn 12 is still recallable at turn 1,000, across sessions, clients and models.
• Better answers from less context. A focused window of relevant summaries and facts beats a raw window of everything, where the one fact you need is buried.
• Lower cost. Smaller payloads, arranged to keep providers' prompt caches warm.
• Group chats that know who said what. Every message keeps its sender, facts keep the author their source shows, and search can be scoped to one person.
• One memory everywhere. Build context in Claude Code, continue it from Telegram, query it from Cursor.

How it works

client ──> virtual-context proxy ──> model provider
             │   before the call: retrieve topics and facts for this turn,
             │                    assemble a bounded window
             │   after the call:  store the turn, tag it, compact when needed
             v
           store (SQLite or PostgreSQL)
             canonical turns   every message verbatim, with sender and channel
             segments + facts  per-topic summaries and structured facts
             topic summaries   one digest per topic, the bird's-eye view

Recent messages stay verbatim. Older turns are grouped by topic, summarized and mined for facts. Each turn, the model gets recent turns plus the topics and facts relevant to the question, and a topic index telling it what else it can open with built-in tools (vc_expand_topic, vc_find_quote, vc_recall_all, and others).

The conversation is the record

virtual-context is built like virtual memory. The conversation is the backing store: every turn is kept verbatim, with who said it, where, when and to whom. Everything else is a cache layer above it, there to put the right part of the conversation in front of the model, not to replace it.

Paging happens on every turn. Before each model call, virtual-context pages in what the current question needs: the recent turns, the summaries of the relevant topics, the most relevant facts, and an index of the other topics. Facts and summaries are selected for the same question, and every fact carries a pointer to the topic it came from. When the model needs more, it pages deeper itself: vc_expand_topic opens a topic from its summary down to its full transcript, vc_find_quote searches the verbatim record, vc_remember_when scopes recall to a time range, and vc_restore_tool puts back a tool output, or a turn's whole tool-call chain, that the proxy replaced with a stub in the request.

So a fact is not a final answer. It is a cached copy of something that was said, telling the model what it can assume and where the full discussion lives. That is where the questions a memory has to answer are settled:

When a cached fact and the conversation disagree, the conversation wins, and it is always one page-in away.

Quick start

pip install "virtual-context[proxy,embeddings]"    # Python 3.11+

virtual-context uses a language model of your choice to tag turns and write summaries. Pick where it runs:

# A local model server (Ollama, LM Studio, llama.cpp, vLLM): tagging and summaries stay on your machine
virtual-context init recommended-local

# Or a hosted model through OpenRouter
virtual-context init recommended
export OPENROUTER_API_KEY=...

Either writes virtual-context.yaml; set tag_generator.model and summarization.model to the model you want. Then start the proxy and point your client at http://127.0.0.1:5757 instead of the provider:

virtual-context proxy --upstream https://api.anthropic.com

A dashboard runs at http://127.0.0.1:5757/dashboard.

The two presets are the hosted service's configuration on one machine and differ only in where the model runs: SQLite storage, paging tools on, fact curation on. Answer quality depends on the model you pick. scripts/smoke_recommended.sh checks a preset end to end from a clean install. The optional parts (PostgreSQL vector search, the judgment layer, group attribution) are each a section of docs/configuration.md; virtual-context presets list shows every preset, and docs/install.md covers install options and running it as a service.

The proxy accepts Anthropic Messages, OpenAI Responses, OpenAI Chat Completions and Gemini requests, and injects the paging tools for all four.

Hosted: virtual-context.com runs the same engine as a service with the dashboard and cost reports; you change your base URL and nothing else.

Integrations

Claude Code

export ANTHROPIC_BASE_URL=http://127.0.0.1:5757

Tool output (file reads, searches, command results) is collapsed into restorable stubs, and when Claude Code trims its own history, the missing turns are restored from storage.

Any client with a base URL (Cursor, Continue, your own app)

client = anthropic.Anthropic(base_url="http://127.0.0.1:5757")
client = OpenAI(base_url="http://127.0.0.1:5757/v1")         # proxy started with --upstream https://api.openai.com

OpenClaw: use the OpenClaw plugin. Settings for direct proxy use are in docs/install.md.

Python SDK

from virtual_context import VirtualContextEngine

engine = VirtualContextEngine(config_path="./virtual-context.yaml")
assembled = engine.on_message_inbound(message=user_message, conversation_history=messages)  # before the LLM call
report = engine.on_turn_complete(messages)                                                  # after it

MCP server: eight tools (recall_context, compact_context, expand_topic, recall_all, remember_when, find_quote, search_summaries, domain_status) plus topic resources, for Claude Desktop, Cursor or any MCP client.

Features

Commands in any conversation

Type these as ordinary messages in a connected client; the proxy handles them without calling the model.

VCATTACH is a durable redirect: nothing is deleted and old references keep resolving, so two clients (or two agents) can share one memory. Details: docs/commands.md.

Running it in production

• Storage: SQLite by default; PostgreSQL (pip install "virtual-context[postgres]") for multi-worker deployments. With pgvector installed, virtual-context admin migrate-semantic-vectors prepares in-database vector search.
• Multi-worker safety: compactions run under leased, fenced operations, lifecycle changes are epoch-guarded, and the stores find and claim conversations whose compaction backlog their traffic never triggered inline, for a sweeper to compact.
• Operator tooling: virtual-context admin has 28 guarded commands for reviewing and editing the record, backfills, repairs and migrations, with dry-run modes.
• Security: dashboard endpoints are open until you set VC_DASHBOARD_TOKEN; the default bind is loopback only.
• Observability: per-stage timing logs, request captures in the dashboard, and per-call cost telemetry.

Architecture, including the canonical-turn model and the prepare/ingest path a host exposes over REST: docs/architecture.md.

Benchmarks

On 100 questions from LongMemEval-500, with the same reader model (Claude Sonnet 4.5):

Configuration, results by category, the per-question table and how new runs are recorded: docs/benchmarks.md.

Known limits

A cache can be wrong. What we know and how it is handled:

• A wrong cached fact can steer an answer. If extraction misreads what was said, the model may act on that fact before paging in its source. Several things limit this: facts are curated per question rather than shown wholesale, every fact names its topic so the source is one page-in away, the topic's summary is selected for the same question, facts keep their author and dates, and supersession retires a fact once a newer one replaces it. A later correction in the conversation always becomes part of the record. Better ways to catch a stale fact before it is used are ongoing work.

Documentation

Development

git clone https://github.com/virtual-context/virtual-context.git
cd virtual-context
python -m venv .venv && source .venv/bin/activate
python -m pip install uv==0.9.30
uv sync --locked --extra all --extra dev
.venv/bin/python scripts/check_contracts.py

License

AGPL-3.0-or-later, Copyright Y. Ahmed Kidwai. For commercial licensing: ahmed@kidw.ai