hypercompress

Query-aware, meaning-first context compression for LLM applications. Same objective as a commercial learned-model compressor — cut input tokens before they hit the model, keep the meaning — with a different set of engineering bets:

The honest caveat: a well-trained learned scorer can beat lexical scoring on paraphrase-heavy queries (where the question shares few exact words with the evidence). That is the one axis we don't claim to win — run the included head-to-head harness on your traffic and let the data decide.

Benchmark (reproduce: python benchmarks/run_benchmark.py)

60 synthetic cases, ~50 near-topic distractor paragraphs each, evidence placed at a random position, retention = ALL answer spans present verbatim in the compressed output:

Read these numbers for what they are: the questions share topic vocabulary and exact identifiers (codes, names, figures) with their evidence — the realistic case for support/RAG/incident queries, and exactly where lexical scoring shines. Paraphrase-only queries will score lower; the risk signal is designed to catch that (weak coverage → high → caller sends the original context).

vs LLMLingua / LongLLMLingua (measured, reproduce with benchmarks/vs_llmlingua.py)

24 standard + 16 paraphrase cases, all systems local on the same CPU ("safe savings" = savings on cases where every answer span survived):

The hybrid wins BOTH suites on safe savings with equal-or-better retention: a lexical fast path (~1ms) serves confidently-answerable queries, and a false-confidence guard (top-block score spike detection) routes everything else to the semantic tier — 256-dim static embeddings (model2vec, ~30MB, numpy-only) rank paragraphs by cosine similarity, then a fingerprint sweep re-adds identifier-bearing paragraphs the ranking missed. Answers carry codes, dates and amounts; distractor prose does not — the sweep is what turns a decent ranking into 100% retention. A heavier LongLLMLingua-style neural tier ([neural], ~2GB) remains available and is preferred when HYPERCOMPRESS_PREFER_NEURAL=1. Latency honesty: competitor numbers above are CPU; on an Apple-GPU (MPS) re-run LLMLingua-2 drops to ~37ms and LongLLMLingua to ~480ms — retention/savings unchanged, and still 15–500× slower than the hybrid. These are synthetic suites — validate on your own traffic (benchmarks/my_data_benchmark.py).

from hypercompress import compress_context_hybrid   # pip install ".[semantic]"
result = compress_context_hybrid(context, question) # 1ms fast path, semantic tier on declines

Install

pip install .                 # core: zero dependencies
pip install ".[server]"       # + self-hosted API (FastAPI/uvicorn)
pip install ".[semantic]"     # + 30MB paraphrase tier (recommended)
pip install ".[mcp]"          # + MCP server for coding agents
pip install ".[exact-tokens]" # + tiktoken for exact token counts

Library (drop-in for common compression-client conventions)

from hypercompress import compress_context

result = compress_context(context, question)          # adaptive mode
result = compress_context(context, question, 0.3)     # fixed 30% budget

result.compressed_text     # send this to the LLM
result.tokens_saved_pct    # e.g. 78.3
result.compression_risk    # "low" | "medium" | "high" -> fall back if high
result.kept_blocks         # audit trail: every block, score, reasons

Message-list form (system prompt + latest user message always verbatim):

from hypercompress import compress_for_turn
messages = compress_for_turn(messages)

The interactive demo — watch a compression happen, on your own files

pip install "hypercompress[server,documents]"
uvicorn hypercompress.server:app --port 8765
# open http://localhost:8765/demo

What it's for. The demo is the live proof surface: it runs the exact production defaults (no knobs the library doesn't have), on your own documents, and shows every decision the engine makes. Nothing is mocked — every view is the real audit trail returned by the API.

Inputs. Upload a PDF, Word, Excel, markdown, log, JSON, CSV or code file — or pick a built-in example (incident log, meeting notes, chat history, JSON data; an "edge cases" toggle adds the stress tests: a question worded with none of the document's vocabulary, and an input too small to compress — which the engine correctly refuses to touch). Type the question, optionally list comma-separated facts that MUST survive, and hit Compress.

The five-step walkthrough, with plain-English narration at each hand-off:

1. Input — what the app was about to send, and what it costs.
2. Split — the text cut along natural seams: log runs (errors isolated), sections with their headers, chat turns, JSON branches, spreadsheet tables.
3. Score — every block ranked with its recorded reasons (BM25, exact identifiers, phrases, severity, recency), the explicit keep-line rule (max(0.38 × top, 0.35)) computed for this run, and each block marked above/below the line. Survey-style questions are detected from the question form and narrated: coverage is widened automatically.
4. Decide — kept blocks in green (✓ KEPT), dropped ones struck through, completeness rules for tables (matching rows kept in full under "all/every/how many" questions), and the tier decision: whether the fast lexical path was confident (three checks, real numbers shown) or the semantic/neural tier produced the final text, and why.
5. Send — the compressed text with […] elision marks, a with-vs-without impact table (input tokens, est. cost per request and per month, prefill work, quota headroom — editable price/volume assumptions; token counts measured, not estimated), and the quality proofs below.

Quality proofs.

• Facts check — every fact you named is verified to survive verbatim in the compressed output (the same retention criterion the benchmark suites use). A loss without risk="high" is a reportable bug.
• Answer quality with automated judge — with an API key configured (ANTHROPIC_API_KEY in the server environment, or the macOS Keychain entry hypercompress-anthropic; the key never reaches the browser), one click asks the same model the same question twice: once with the full document, once with the compressed one. A third call then judges the two answers — anonymized and in randomized order, so the judge cannot know or positionally favor either side — and the demo renders the verdict: the winning column is highlighted green with a ✓ and the judge's one-sentence reason is shown; a too-close-to-call verdict renders as a neutral gray tie. The judge can and does pick the full-context side when that answer is genuinely better — that honesty is the point, and one such verdict caught (and fixed) a brevity bias in the comparison harness itself.

A Reset button clears everything for the next run. The side-by-side answer columns are fixed at 50/50 width with word-wrap, so long answers render fully on both sides.

Self-hosted API (hosted-API compatible)

uvicorn hypercompress.server:app --port 8765
# optional auth: export HYPERCOMPRESS_API_KEY=hc_your_key

curl -X POST localhost:8765/api/v1/compress \
  -H 'content-type: application/json' \
  -d '{"context":"...long context...","query":"what failed?"}'

Response schema matches hosted compression services (compressed_text, original_tokens, kept_tokens, tokens_saved_pct, important_kept_pct, compression_risk, kept_blocks, dropped_blocks, policy_name), and the same auth headers (X-API-Key / Authorization: Bearer) are accepted — existing hosted-API client code migrates by changing one URL.

Integrations (full hosted-API parity)

OpenAI — hypercompress/wrappers/openai_wrapper.py

from openai import OpenAI
from hypercompress.wrappers.openai_wrapper import HyperCompressOpenAI
client = HyperCompressOpenAI(OpenAI())
client.chat.completions.create(model="gpt-4o-mini", messages=msgs)
client.stats.tokens_saved_pct   # verified savings, not vendor claims

Anthropic — hypercompress/wrappers/anthropic_wrapper.py

from hypercompress.wrappers.anthropic_wrapper import HyperCompressAnthropic
client = HyperCompressAnthropic(anthropic.Anthropic())

LangChain — hypercompress/wrappers/langchain_hook.py

from hypercompress.wrappers.langchain_hook import compress_lc_messages
chain.invoke(compress_lc_messages(messages))

Express / Next.js — js/src/index.js

const { expressMiddleware } = require("hypercompress");
app.post("/chat", expressMiddleware(), handler);   // compresses req.body.messages

Vercel AI SDK — provider-agnostic

const { wrapGenerateText } = require("hypercompress");
const gen = wrapGenerateText(generateText);
await gen({ model, messages });

MCP (Claude Code / Cursor / Codex / Windsurf)

pip install ".[mcp]"
claude mcp add hypercompress -- python -m hypercompress.mcp_server

Exposes compress_context and compress_file tools so agents can pull query-relevant slices of big files instead of whole files.

Guardrails (identical across every integration)

1. System prompts and the latest user message are never compressed.
2. Fail open — errors, timeouts, and compression_risk == "high" all send the original context. Compression must never break an answer.
3. Elisions are marked with […] so the model knows content was removed.
4. Savings are measured and logged locally; nothing here asks you to trust a marketing number.

How it works

splitter.py cuts context into structure-aware blocks (code fences atomic, markdown sections, chat turns, log runs with ERROR lines isolated). scoring.py ranks blocks with Okapi BM25 plus exact-identifier boosts (error codes, numbers, dotted names), bigram matches, log severity, and chat recency; headers inherit their best child's score so surviving sections keep their titles. core.py selects adaptively (keep while marginal relevance is meaningful) or under a fixed budget, stitches ±1 neighbor blocks for local coherence, reassembles in original order with […] gap markers, and computes compression_risk from measured query-term coverage.

Tests

python -m pytest tests/   # 14 tests: core behavior, wire compat, wrappers

Author

Built by Natarajan Venkatasubramaniam (Natarajan.Venkatasubramaniam@wissen.com).

License

MIT © 2026 Natarajan Venkatasubramaniam. See LICENSE.