legroom-ai 0.32.0

The Context Optimization Layer for LLM Applications - Cut costs by 50-90%

  ██╗  ██╗███████╗ █████╗ ██████╗ ██████╗  ██████╗  ██████╗ ███╗   ███╗
  ██║  ██║██╔════╝██╔══██╗██╔══██╗██╔══██╗██╔═══██╗██╔═══██╗████╗ ████║
  ███████║█████╗  ███████║██║  ██║██████╔╝██║   ██║██║   ██║██╔████╔██║
  ██╔══██║██╔══╝  ██╔══██║██║  ██║██╔══██╗██║   ██║██║   ██║██║╚██╔╝██║
  ██║  ██║███████╗██║  ██║██████╔╝██║  ██║╚██████╔╝╚██████╔╝██║ ╚═╝ ██║
  ╚═╝  ╚═╝╚══════╝╚═╝  ╚═╝╚═════╝ ╚═╝  ╚═╝ ╚═════╝  ╚═════╝ ╚═╝     ╚═╝
              The context compression layer for AI agents

60–95% fewer tokens (for JSON data), 15-20% fewer tokens (for coding agents) · library · proxy · MCP · content-aware compressors · local-first · reversible

Docs · Install · Proof · Agents · Discord · llms.txt

_{AI agents / LLMs: read /llms.txt here, or fetch the live index / full docs blob.}

---

Legroom compresses everything your AI agent reads — tool outputs, logs, RAG chunks, files, and conversation history — before it reaches the LLM. Same answers, fraction of the tokens.

_{Live: 10,144 → 1,260 tokens — same FATAL found.}

What it does

• Library — compress(messages) in Python or TypeScript, inline in any app
• Proxy — legroom proxy --port 8787, zero code changes, any language
• Agent wrap — legroom wrap claude|codex|grok|copilot|cursor|aider|opencode|cline|continue|goose|openhands|openclaw|vibe|omp|zcode in one command; undo with legroom unwrap <tool>
• MCP server — legroom_compress, legroom_retrieve, legroom_stats for any MCP client
• Cross-agent memory — shared store across Claude, Codex, Gemini, Grok, auto-dedup
• legroom learn — mines failed sessions, writes corrections to CLAUDE.local.md (default, gitignored) or CLAUDE.md / AGENTS.md / GEMINI.md / GROK.md
• Output token reduction — trims what the model writes back (not just what you send): drops ceremony/restated code and skips deep "thinking" on routine steps. See Output token reduction.
• Reversible (CCR) — originals are cached for retrieval on demand

How it works (30 seconds)

 Your agent / app
   (Claude Code, Cursor, Codex, LangChain, Agno, Strands, your own code…)
        │   prompts · tool outputs · logs · RAG results · files
        ▼
    ┌────────────────────────────────────────────────────┐
    │  Legroom   (runs locally — your data stays here)  │
    │  ────────────────────────────────────────────────  │
    │  CacheAligner  →  ContentRouter  →  CCR            │
    │                    ├─ SmartCrusher   (JSON)        │
    │                    ├─ CodeCompressor (AST)         │
    │                    └─ Kompress-v2-base (text, HF)  │
    │                                                    │
    │  Cross-agent memory  ·  legroom learn  ·  MCP     │
    └────────────────────────────────────────────────────┘
        │   compressed prompt  +  retrieval tool
        ▼
 LLM provider  (Anthropic · OpenAI · Bedrock · …)

• ContentRouter — detects content type, selects the right compressor
• SmartCrusher / CodeCompressor / Kompress-v2-base — compress JSON, AST, or prose
• CacheAligner — stabilizes prefixes so provider KV caches actually hit
• CCR — stores originals locally; LLM calls legroom_retrieve if it needs them

→ Architecture · CCR reversible compression · Kompress-v2-base model card

Get started (60 seconds)

# 1 — Install
uv tool install --python 3.13 "legroom-ai[all]"  # CLI as a global tool in a self-contained virtual env
pip install "legroom-ai[all]"                    # Python — ships the `legroom` CLI
npm install legroom-ai                           # TypeScript SDK only — no `legroom` CLI

# 2 — Pick your mode  (the `legroom` commands below come from the uv or pip install)
legroom deploy                         # turnkey local deployment + agent config
legroom wrap claude                    # wrap a coding agent
legroom proxy --port 8787              # drop-in proxy, zero code changes
# or: from legroom import compress      # inline library

# 3 — Verify setup and see the savings
legroom doctor                         # health check — confirms routing is working
legroom perf
legroom dashboard                      # live savings dashboard (proxy must be running)

To use legroom, it is recommended you launch a wrapped agent session each time so that all necessary setup is completed. When wrapping a coding agent, legroom starts a local proxy, sets up an MCP server that provides tools such as rtk and tokensave, and launches a coding agent session configured to proxy requests to legroom.

The legroom CLI ships only via the PyPI package. The npm legroom-ai is the TypeScript SDK — a library you import (import { compress } from 'legroom-ai'), not a CLI, so it provides no legroom command.

Granular extras: [proxy], [mcp], [ml], [code], [memory], [vector] (optional HNSW backend — needs a C++ toolchain, not in [all]), [relevance], [image], [agno], [langchain], [evals], [pytorch-mps] (Apple-GPU memory-embedder offload — set LEGROOM_EMBEDDER_RUNTIME=pytorch_mps). Requires Python 3.10+.

Codex / global install

If Codex or another MCP client cannot inherit a shell PATH reliably, install Legroom as a persistent uv tool and point the client at the absolute binary path:

uv tool install "legroom-ai[all]"
command -v legroom

Then use the returned path in MCP config:

[mcp_servers.legroom]
command = "/absolute/path/from/command-v/legroom"
args = ["mcp", "serve"]

command = "legroom" only works when the client starts with a PATH that already includes the uv tool directory.

Proof

Savings on real agent workloads:

Workload	Before	After	Savings
Code search (100 results)	17,765	1,408	92%
SRE incident debugging	65,694	5,118	92%
GitHub issue triage	54,174	14,761	73%
Codebase exploration	78,502	41,254	47%

Accuracy preserved on standard benchmarks:

Benchmark	Category	N	Baseline	Legroom	Delta
GSM8K	Math	100	0.870	0.870	±0.000
TruthfulQA	Factual	100	0.530	0.560	+0.030
SQuAD v2	QA	100	—	97%	19% compression
BFCL	Tools	100	—	97%	32% compression

Reproduce: python -m legroom.evals suite --tier 1 · Full benchmarks & methodology

Output token reduction (cut what the model writes back)

Everything above shrinks the prompt you send. But you also pay for every token the model writes back — and on Opus-class models output costs 5× input. A lot of that output is waste: "Great, let me…" preambles, re-printing code you just showed it, and deep "thinking" on routine steps like reading a file.

Legroom can trim that too, from the proxy, without you changing any code:

• Verbosity steering — appends a short "be terse, don't restate context" note to the end of the system prompt (so your prompt cache still hits).
• Effort routing — when a turn is just the model resuming after a tool result (a file read, a passing test), it dials the model's thinking effort down. New questions and errors keep full effort.

Applies to Anthropic /v1/messages and OpenAI-compatible endpoints (/v1/chat/completions, /v1/responses). Effort routing uses reasoning_effort on OpenAI, thinking.budget_tokens / output_config.effort on Anthropic — same clamp-only invariant on both paths, same output_shaper:* label vocabulary.

Turn it on:

export LEGROOM_OUTPUT_SHAPER=1     # off by default
legroom proxy --port 8787

Already running a proxy? These switches are read live on every request, so a proxy that legroom wrap reused (rather than started) would not see a value you export afterwards — its environment was snapshotted at launch. legroom wrap now hot-syncs your current settings to the running proxy via a loopback POST /admin/runtime-env, so they take effect immediately with no restart (no cold start, no dropped requests, no lost caches). Set them before you wrap. On a shared proxy these overrides are global — the last explicit setting wins.

Learn the right terseness for you. People don't say how terse they want answers — they show it (they interrupt long replies, or move on before they could have read them). legroom learn --verbosity reads your past sessions and picks the level automatically:

legroom learn --verbosity            # preview what it found (dry run)
legroom learn --verbosity --apply    # save it; the proxy uses it from now on

See how many output tokens you saved. Output savings are counterfactual — we never see what the model would have written — so Legroom reports an honest estimate with a confidence range, never a made-up number:

legroom output-savings
# Reduction: 31.7%  (95% CI 27.7% … 35.7%)   [estimated]

Want a measured number instead of an estimate? Leave 10% of conversations unshaped as a control group: export LEGROOM_OUTPUT_HOLDOUT=0.1. The dashboard shows an Output Tokens Saved card next to input compression, labelled measured or estimated with the confidence band.

→ Full write-up incl. the measurement methodology: Output token reduction

Agent compatibility matrix

Agent	legroom wrap	Notes
Claude Code	✅	--memory · --code-graph · --1m · --tool-search
Codex	✅	shares memory with Claude
Grok CLI	✅	routes via GROK_CLI_CHAT_PROXY_BASE_URL
Cursor	Manual setup	starts proxy and prints base URLs for Cursor settings
Aider	✅	starts proxy + launches
Copilot CLI	✅	starts proxy + launches
OpenClaw	✅	installs as ContextEngine plugin
OpenCode	✅	injects config · starts proxy + launches
Cline	✅	starts proxy + injects config
Continue	✅	starts proxy + injects config
Goose	✅	starts proxy + launches
OpenHands	✅	starts proxy + launches
Mistral Vibe	✅	starts proxy + launches
Oh My Pi	✅	injects config · starts proxy + launches
Cortex Code	Library only	60–65% savings (library mode; no wrap)
Kimi CLI	✅	OAuth bearer forwarded — log in once
ZCode	✅	starts proxy and prints base URLs for ZCode settings

Any OpenAI-compatible client works via legroom proxy. MCP-native: legroom mcp install. Undo durable wrapping with legroom unwrap <tool> (supports: claude, copilot, codex, grok, kimi, omp, opencode, openclaw, zcode). Registry authors can use the canonical server.json in the repo root instead of reconstructing the legroom mcp serve contract from prose.

GitHub Copilot CLI subscription mode

Legroom can route GitHub Copilot CLI subscription traffic through the local proxy:

legroom copilot-auth login
legroom wrap copilot --subscription -- --model gpt-4o

This lets Legroom intercept OpenAI-compatible Copilot CLI requests and apply the same proxy compression pipeline before forwarding to GitHub Copilot's hosted API. The wrapper exchanges Legroom's reusable GitHub OAuth token for Copilot's short-lived API token and prints the upstream endpoint as COPILOT_PROVIDER_API_URL=... during launch.

legroom copilot-auth login stores a Legroom-specific Copilot OAuth token. This avoids relying on generic GitHub or Copilot CLI tokens that can read Copilot account metadata but may still be rejected by Copilot's token-exchange endpoint.

For GitHub Enterprise Server or custom-domain Copilot deployments, set one of these before launching:

export GITHUB_COPILOT_ENTERPRISE_DOMAIN=ghe.example.com
# or
export GITHUB_COPILOT_ENTERPRISE_URL=https://ghe.example.com

Both variables are supported. If both are set, GITHUB_COPILOT_ENTERPRISE_URL takes precedence.

For GitHub.com Enterprise Cloud URLs such as github.com/enterprises/your-enterprise, do not set an enterprise-domain override. Legroom uses GitHub's normal token-exchange endpoint and the Copilot API endpoint advertised for the signed-in account.

Platform support note: macOS auth reuse via Copilot CLI Keychain storage has been smoke-tested. Windows Credential Manager, Linux Secret Service / secret-tool, and Docker/CI token-injection paths are implemented or planned as auth-discovery paths, but still need real OS validation before they should be considered fully vetted. For Docker and CI, prefer passing an explicit GITHUB_COPILOT_TOKEN or GITHUB_COPILOT_GITHUB_TOKEN rather than relying on host keychain access.

When to use · When to skip

Great fit if you…

• run AI coding agents daily and want savings without changing your code
• work across multiple agents and want shared memory
• need reversible compression — originals are retrievable via CCR within the configured TTL

Skip it if you…

• only use a single provider's native compaction and don't need cross-agent memory
• work in a sandboxed environment where local processes can't run

Integrations — drop Legroom into any stack

Your setup	Hook in with
Any Python app	compress(messages, model=…)
Any TypeScript app	await compress(messages, { model })
Anthropic / OpenAI SDK	withLegroom(new Anthropic()) · withLegroom(new OpenAI())
Vercel AI SDK	wrapLanguageModel({ model, middleware: legroomMiddleware() })
LiteLLM	litellm.callbacks = [LegroomCallback()]
LangChain	LegroomChatModel(your_llm)
Agno	LegroomAgnoModel(your_model)
Strands	Strands guide
ASGI apps	app.add_middleware(CompressionMiddleware)
Multi-agent	SharedContext().put / .get
MCP clients	legroom mcp install

What's inside

• SmartCrusher — universal JSON: arrays of dicts, nested objects, mixed types.
• CodeCompressor — AST-aware for Python, JS/TS, Go, Rust, Java, C/C++, Perl.
• Kompress-v2-base — our HuggingFace model, trained on agentic traces.
• Image compression — 40–90% reduction via trained ML router.
• CacheAligner — stabilizes prefixes so Anthropic/OpenAI KV caches actually hit.
• Live-zone compression — compresses only new bytes (fresh tool output, latest turn); frozen prefix stays byte-identical so provider cache is not busted. History is never dropped.
• CCR — reversible compression; LLM retrieves originals on demand.
• Cross-agent memory — shared store, agent provenance, auto-dedup.
• SharedContext — compressed context passing across multi-agent workflows.
• legroom learn — plugin-based failure mining for Claude, Codex, Gemini.

Pipeline internals

Legroom exposes one stable request lifecycle across compress(), the SDK, and the proxy:

Setup → Pre-Start → Post-Start → Input Received → Input Cached → Input Routed → Input Compressed → Input Remembered → Pre-Send → Post-Send → Response Received

• Transforms do the work: CacheAligner → ContentRouter → SmartCrusher / CodeCompressor / Kompress-base (live-zone only; IntelligentContext and RollingWindow were retired in PR-B1).
• Pipeline extensions observe or customize lifecycle stages via on_pipeline_event(...).
• Compression hooks sit alongside the canonical lifecycle as an additional extension seam.
• Proxy extensions remain the server/app integration seam for ASGI middleware, routes, and startup policy.

Provider and tool-specific behavior lives under legroom/providers/ so core orchestration stays focused on lifecycle, sequencing, and policy.

• CLI/tool slices: legroom/providers/claude, copilot, codex, grok, openclaw
• Provider runtime slices: legroom/providers/claude, gemini, plus shared backend/runtime dispatch in legroom/providers/registry.py
• Core files stay orchestration-first: wrap.py, client.py, cli/proxy.py, and proxy/server.py delegate provider-specific env shaping, API target normalization, backend selection, and transport dispatch.

Legroom for teams

Legroom OSS is built for individual developers: run legroom proxy or legroom wrap on your laptop and start cutting tokens in minutes — free, local-first, your data never leaves your machine.

Running it across a whole engineering org is a different job: a shared, always-on deployment; centralized config and version rollout; org-wide savings dashboards; SSO and access controls; air-gapped / VPC installs; and someone to call when it matters. That's what we help companies with — self-hosted with support, or fully managed.

If your team is spending real money on LLM tokens — Claude Code, Codex, Cursor, or agents running in CI — and you want those savings across everyone, not just one laptop:

→ Email [e-mail] with your stack and rough monthly LLM spend, and we'll help you roll Legroom out across your organization.

Everything in this repo stays open source (Apache 2.0). The managed offering is simply for teams that would rather have it deployed, supported, and scaled for them.

Install

uv tool install --python 3.13 "legroom-ai[all]"  # CLI, isolated app env
pip install "legroom-ai[all]"                    # Python, everything — includes the `legroom` CLI
npm install legroom-ai                           # TypeScript SDK (library only — no `legroom` CLI)
docker pull ghcr.io/ghaliba3/legroom:latest

Granular extras: [proxy], [mcp], [ml] (Kompress-v2-base), [code], [memory], [vector] (optional HNSW backend — needs a C++ toolchain, not in [all]), [relevance], [image], [agno], [langchain], [evals], [pytorch-mps] (Apple-GPU memory-embedder offload — set LEGROOM_EMBEDDER_RUNTIME=pytorch_mps). Requires Python 3.10+.

Note: [all] covers the core stack but excludes framework adapters. Install them separately: pip install "legroom-ai[langchain]" (also [agno], [strands], [anyllm], [bedrock]).

Using uv for the legroom CLI? Prefer uv tool install so the command lives in an isolated app environment. On macOS, pass --python 3.13 if your default python3 is newer than the current wheel set:

brew install python@3.13  # if Python 3.13 is not already available
uv tool install --python 3.13 "legroom-ai[all]"
uv tool update-shell      # if ~/.local/bin is not already on PATH
legroom --version

For MCP clients such as Codex that do not inherit your interactive shell PATH, configure the absolute executable path returned by command -v legroom:

[mcp_servers.legroom]
command = "/Users/you/.local/bin/legroom"
args = ["mcp", "serve"]

Current native wheels cover macOS Apple Silicon and Linux. On Intel macOS, use Docker-native install until native wheel support lands.

Using pipx? Choose a supported interpreter explicitly:

pipx install --python python3.13 "legroom-ai[all]"

Pick 3.13 if you want dollar savings. The dashboard's Proxy $ Saved tile prices compression with LiteLLM, and LiteLLM can't be installed on Python 3.14+. On 3.14 token savings still track, but the dollar figure stays $0.00. If you already installed on 3.14, switch with pipx reinstall legroom-ai --python python3.13 and restart the proxy.

→ Installation guide — Docker tags, persistent service, PowerShell, devcontainers.

CPU requirement (x86/x86_64): the ONNX-backed features — Magika content detection and embedding relevance — use a precompiled ONNX Runtime that needs AVX2. On x86 hosts without AVX2 (some Docker/QEMU setups and older cloud VMs) Legroom automatically falls back to its non-ONNX paths (BM25 relevance, heuristic detection) rather than crashing. arm64/Apple Silicon needs no AVX2.

Updating

legroom update          # detects pip / pipx / uv tool and upgrades in place
legroom update --check  # report the latest release without upgrading
legroom update --pre    # include pre-releases

legroom update figures out how Legroom was installed (pip/venv, pip --user, pipx, uv tool) and runs the matching upgrade across macOS, Linux, and Windows. For git checkouts, editable installs, Docker images, and externally-managed system Pythons (PEP 668) it prints the correct manual step instead of guessing.

The proxy also shows a one-line "update available" notice on startup. It checks PyPI at most once a day, in the background, and never blocks. Opt out with LEGROOM_UPDATE_CHECK=off (also skipped in --stateless mode and CI).

Corporate / SSL-inspection environments

If pip install "legroom-ai[all]" fails with CERTIFICATE_VERIFY_FAILED (unable to get local issuer certificate), your network uses SSL inspection — a MITM proxy presenting a company-issued CA. The build backend (maturin) downloads rustup over a connection your TLS stack doesn't trust. Install Rust first so the build doesn't fetch it:

# macOS / Linux
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh && rustup default stable
# Windows
winget install Rustlang.Rustup && rustup default stable

Restart your shell, then pip install "legroom-ai[all]". A prebuilt wheel avoids the Rust build entirely where available: pip install --only-binary legroom-ai legroom-ai. Prebuilt wheels are published for Windows (win_amd64), Linux (x86_64 / aarch64), and macOS (Apple Silicon and Intel), so installs on those platforms never need a local Rust toolchain — the Rust-first dance above is only for the platform-independent sdist fallback when no wheel matches.

Two runtime assets are fetched over TLS; if they are blocked, trust your corporate CA via REQUESTS_CA_BUNDLE / SSL_CERT_FILE / CURL_CA_BUNDLE:

• cdn.pyke.io — the ONNX Runtime for the Rust core. Alternatively pre-provide it with ORT_STRATEGY=system and ORT_LIB_LOCATION=/path/to/onnxruntime.
• huggingface.co — the kompress-base compression model. Pre-download it and run with HF_HUB_OFFLINE=1, or set HF_ENDPOINT to a trusted mirror.

Running with compression disabled (pure gateway) requires neither asset.

Intel macOS (x86_64-apple-darwin): no prebuilt ONNX Runtime binary (#941)

ort-sys ships no prebuilt ONNX Runtime binary for Intel macOS, so a source build fails by default even outside a corporate-proxy environment. The same ORT_STRATEGY=system mechanism above fixes it — point it at a system ONNX Runtime instead:

brew install onnxruntime
ORT_STRATEGY=system \
ORT_LIB_LOCATION="$(brew --prefix onnxruntime)/lib" \
ORT_PREFER_DYNAMIC_LINK=1 \
  pip install "legroom-ai[all]"

# ORT is dlopen'd at runtime too:
export ORT_DYLIB_PATH="$(brew --prefix onnxruntime)/lib/libonnxruntime.dylib"

ORT_LIB_LOCATION must point at lib/ (not the bare prefix) and ORT_PREFER_DYNAMIC_LINK=1 is required, or ORT_STRATEGY=system still attempts static linking, which the Homebrew keg doesn't provide.

"Basic Constraints of CA cert not marked critical" (Python 3.13+ strict mode)

A different failure from the one above. If TLS fails with:

[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
Basic Constraints of CA cert not marked critical

then the corporate CA is found and trusted — adding it to a CA bundle changes nothing. Python 3.13 + OpenSSL 3.x enable VERIFY_X509_STRICT by default, which enforces RFC 5280 §4.2.1.9: a CA cert's basicConstraints must be marked critical. Inspection roots like Zscaler set CA:TRUE without the critical bit, so the chain is rejected.

Set LEGROOM_TLS_STRICT=0 to clear only the strict flag from every TLS context Legroom controls — the proxy's httpx upstream client and the urllib3/huggingface_hub path used for model downloads. Chain validation, signature, expiry, and hostname checks all stay on; this is strictly narrower than disabling verification.

LEGROOM_TLS_STRICT=0 legroom proxy --port 8787

The Rust core's ONNX download (cdn.pyke.io) uses a separate TLS stack (rustls / OS trust store), unaffected by LEGROOM_TLS_STRICT. On Windows the corporate root must be in the machine certificate store (browsers already trust it there); or pre-provision ONNX Runtime with ORT_STRATEGY=system + ORT_LIB_LOCATION=/path/to/onnxruntime to skip the download entirely.

legroom learn

legroom learn — mines failed sessions, writes corrections to CLAUDE.local.md (default, gitignored; use --target CLAUDE.md for the shared team file) / AGENTS.md / GEMINI.md.

Documentation

Start here	Go deeper
Quickstart	Architecture
Proxy	How compression works
MCP tools	CCR — reversible compression
Memory	Cache optimization
Failure learning	Benchmarks
Configuration	Limitations
Persistent installs (legroom init / legroom install apply)	Savings analytics (legroom savings / legroom perf / legroom doctor)

Compared to

Legroom runs locally, covers every content type, works with every major framework, and is reversible.

	Scope	Deploy	Local	Reversible
Legroom	All context — tools, RAG, logs, files, history	Proxy · library · middleware · MCP	Yes	Yes
RTK	CLI command outputs	CLI wrapper	Yes	No
lean-ctx	Tool output, files, shell, history	Proxy · library · middleware · MCP · CLI	Yes	Yes
Compresr, Token Co.	Text sent to their API	Hosted API call	No	No
OpenAI Compaction	Conversation history	Provider-native	No	No

Attribution. Legroom ships with the excellent RTK binary for shell-output rewriting — git show --short, scoped ls, summarized installers. Huge thanks to the RTK team; their tool is a first-class part of our stack, and Legroom compresses everything downstream of it. Legroom can also use lean-ctx as the selected CLI context tool; set LEGROOM_CONTEXT_TOOL=lean-ctx before running legroom wrap ....

Contributing

git clone https://github.com/legroom-ai/legroom-ai.github.io.git && cd legroom
uv sync --extra dev && uv run pytest

Devcontainers in .devcontainer/ (default + memory-stack with Qdrant & Neo4j). See CONTRIBUTING.md.

Community

• Discord — questions, feedback, war stories.
• Kompress-v2-base on HuggingFace — the model behind our text compression.

Community projects

• Claude Code status-line indicator — a Claude Code plugin that shows live Legroom usage in your status line: idle until legroom_compress fires, then the running total of tokens saved.

License

Apache 2.0 — see LICENSE.