hypercompress

Query-aware, meaning-first context compression for LLM applications. Same objective as a commercial learned-model compressor — cut input tokens before they hit the model, keep the meaning — with a different set of engineering bets:

The honest caveat: a well-trained learned scorer can beat lexical scoring on paraphrase-heavy queries (where the question shares few exact words with the evidence). That is the one axis we don't claim to win — run the included head-to-head harness on your traffic and let the data decide.

Benchmark (reproduce: python benchmarks/run_benchmark.py)

60 synthetic cases, ~50 near-topic distractor paragraphs each, evidence placed at a random position, retention = ALL answer spans present verbatim in the compressed output:

Read these numbers for what they are: the questions share topic vocabulary and exact identifiers (codes, names, figures) with their evidence — the realistic case for support/RAG/incident queries, and exactly where lexical scoring shines. Paraphrase-only queries will score lower; the risk signal is designed to catch that (weak coverage → high → caller sends the original context).

vs LLMLingua / LongLLMLingua (measured, reproduce with benchmarks/vs_llmlingua.py)

24 standard + 16 paraphrase cases, all systems local on the same CPU ("safe savings" = savings on cases where every answer span survived):

The hybrid wins BOTH suites on safe savings with equal-or-better retention: a lexical fast path (~1ms) serves confidently-answerable queries, and a false-confidence guard (top-block score spike detection) routes everything else to the semantic tier — 256-dim static embeddings (model2vec, ~30MB, numpy-only) rank paragraphs by cosine similarity, then a fingerprint sweep re-adds identifier-bearing paragraphs the ranking missed. Answers carry codes, dates and amounts; distractor prose does not — the sweep is what turns a decent ranking into 100% retention. A heavier LongLLMLingua-style neural tier ([neural], ~2GB) remains available and is preferred when HYPERCOMPRESS_PREFER_NEURAL=1. Latency honesty: competitor numbers above are CPU; on an Apple-GPU (MPS) re-run LLMLingua-2 drops to ~37ms and LongLLMLingua to ~480ms — retention/savings unchanged, and still 15–500× slower than the hybrid. These are synthetic suites — validate on your own traffic (benchmarks/my_data_benchmark.py).

from hypercompress import compress_context_hybrid   # pip install ".[semantic]"
result = compress_context_hybrid(context, question) # 1ms fast path, semantic tier on declines

Install

pip install .                 # core: zero dependencies
pip install ".[server]"       # + self-hosted API (FastAPI/uvicorn)
pip install ".[semantic]"     # + 30MB paraphrase tier (recommended)
pip install ".[mcp]"          # + MCP server for coding agents
pip install ".[exact-tokens]" # + tiktoken for exact token counts

Library (drop-in for common compression-client conventions)

from hypercompress import compress_context

result = compress_context(context, question)          # adaptive mode
result = compress_context(context, question, 0.3)     # fixed 30% budget

result.compressed_text     # send this to the LLM
result.tokens_saved_pct    # e.g. 78.3
result.compression_risk    # "low" | "medium" | "high" -> fall back if high
result.kept_blocks         # audit trail: every block, score, reasons

Message-list form (system prompt + latest user message always verbatim):

from hypercompress import compress_for_turn
messages = compress_for_turn(messages)

Self-hosted API (hosted-API compatible)

uvicorn hypercompress.server:app --port 8765
# optional auth: export HYPERCOMPRESS_API_KEY=hc_your_key

curl -X POST localhost:8765/api/v1/compress \
  -H 'content-type: application/json' \
  -d '{"context":"...long context...","query":"what failed?"}'

Response schema matches hosted compression services (compressed_text, original_tokens, kept_tokens, tokens_saved_pct, important_kept_pct, compression_risk, kept_blocks, dropped_blocks, policy_name), and the same auth headers (X-API-Key / Authorization: Bearer) are accepted — existing hosted-API client code migrates by changing one URL.

Integrations (full hosted-API parity)

OpenAI — hypercompress/wrappers/openai_wrapper.py

from openai import OpenAI
from hypercompress.wrappers.openai_wrapper import HyperCompressOpenAI
client = HyperCompressOpenAI(OpenAI())
client.chat.completions.create(model="gpt-4o-mini", messages=msgs)
client.stats.tokens_saved_pct   # verified savings, not vendor claims

Anthropic — hypercompress/wrappers/anthropic_wrapper.py

from hypercompress.wrappers.anthropic_wrapper import HyperCompressAnthropic
client = HyperCompressAnthropic(anthropic.Anthropic())

LangChain — hypercompress/wrappers/langchain_hook.py

from hypercompress.wrappers.langchain_hook import compress_lc_messages
chain.invoke(compress_lc_messages(messages))

Express / Next.js — js/src/index.js

const { expressMiddleware } = require("hypercompress");
app.post("/chat", expressMiddleware(), handler);   // compresses req.body.messages

Vercel AI SDK — provider-agnostic

const { wrapGenerateText } = require("hypercompress");
const gen = wrapGenerateText(generateText);
await gen({ model, messages });

MCP (Claude Code / Cursor / Codex / Windsurf)

pip install ".[mcp]"
claude mcp add hypercompress -- python -m hypercompress.mcp_server

Exposes compress_context and compress_file tools so agents can pull query-relevant slices of big files instead of whole files.

Guardrails (identical across every integration)

1. System prompts and the latest user message are never compressed.
2. Fail open — errors, timeouts, and compression_risk == "high" all send the original context. Compression must never break an answer.
3. Elisions are marked with […] so the model knows content was removed.
4. Savings are measured and logged locally; nothing here asks you to trust a marketing number.

How it works

splitter.py cuts context into structure-aware blocks (code fences atomic, markdown sections, chat turns, log runs with ERROR lines isolated). scoring.py ranks blocks with Okapi BM25 plus exact-identifier boosts (error codes, numbers, dotted names), bigram matches, log severity, and chat recency; headers inherit their best child's score so surviving sections keep their titles. core.py selects adaptively (keep while marginal relevance is meaningful) or under a fixed budget, stitches ±1 neighbor blocks for local coherence, reassembles in original order with […] gap markers, and computes compression_risk from measured query-term coverage.

Tests

python -m pytest tests/   # 14 tests: core behavior, wire compat, wrappers

Author

Built by Natarajan Venkatasubramaniam (Natarajan.Venkatasubramaniam@wissen.com).

License

MIT © 2026 Natarajan Venkatasubramaniam. See LICENSE.