Paritok

A non-destructive compression gateway for coding agents — cut your input-token bill without changing your agent.

Paritok sits between your agent and the LLM as a drop-in proxy. On every request it strips the tool-schema bloat, compresses tool results and file reads, and summarizes stale history — then forwards upstream, billed on the compressed tokens. Nothing is ever permanently discarded: the agent pulls back any exact original on demand. Works with Claude Code, Cursor, Codex, OpenHands, and any agent that honors BASE_URL — you don't change a line of your agent.

Powered by the first open-source 4B compression model trained specifically for coding agents (45K real trajectories). Cut input token bills from ~25% on turn one to past 85% in long, context-saturated sessions, and fit ~3× more turns in the same context window.

What it does · The three levers · Compounding savings · Cost · Pricing · Quick Start · The engine (model) · Team

📢 News

• 2026-07-31   v1.3.0 — stability release: edit-recovery, read_original API rename (from expand_context).
• 2026-07-19   v1.2.0 ships the embedding-based tool filter — the biggest single-turn lever (~29K → ~8K on a typical Claude Code turn), unlocking prompt-cache-friendly tool selection with gateway_search_tools recall.
• 2026-07-15   Paritok gateway v1.0.0 open-sourced — the proxy/middleware that turns the 4B model into a drop-in Claude Code / Cursor / Codex compression layer.
• 2026-07-14   Paritok-4B-v1 released on Hugging Face Hub with full SWE-bench Lite end-to-end evaluation.
• 2026-06-25   Finished training. 45K teacher-distilled samples on the Qwen3-4B backbone.

🚪 What Paritok does

Paritok runs as a middle layer between your agent and the LLM API — your agent points at Paritok instead of Anthropic/OpenAI, and everything else stays the same.

Your Agent (Claude Code / Cursor / Codex)
  → builds request (tool schemas + history + tool results / file reads)
     ★ Paritok gateway rewrites the request here ★
  → forwarded to Anthropic / OpenAI  (billed on the compressed tokens)
  ← response flows back unchanged; compressed refs expand on demand

The token bill for a coding agent is dominated by input you re-send every turn: dozens of tool schemas, an ever-growing message history, and big file-read / tool-output blocks. Paritok attacks all three — and because it's a non-destructive gateway, anything it compresses or filters is still recoverable on demand. It's lossy on the wire, fully recoverable when it counts.

🎚️ The three levers

Paritok saves tokens through three independent mechanisms. They stack, and they hit different parts of the bill:

1. Tool-schema filter — the biggest single-turn win

Coding agents expose dozens of tools — often 70+ once you add MCP servers — in full JSON schema on every request. Most are irrelevant to the task at hand. Paritok filters them semantically (tool_discovery.strategy: embedding), keeping only the handful relevant to the user's intent in full schema and stubbing the rest.

• This is the largest single-turn lever. On a typical Claude Code turn the tool block alone is ~29K tokens; filtered it drops to ~8K — a saving no amount of file compression matches on a single turn.
• Prompt-cache friendly. The selection is frozen per conversation, so the tools[] block stays byte-stable turn-to-turn and never invalidates the LLM's KV cache.
• Never destructive. Anything filtered is recoverable — the model calls gateway_search_tools and gets the full schema back.
• Runs a small open embedding model (BAAI/bge-small-en-v1.5, MIT, ~130MB) entirely locally on CPU — no API, no per-token fee.
• An agent's core execution tool (shell / exec / apply_patch) is never stubbed, so agents like Codex that expose only a handful of tools always keep the one they can't work without.

2. Content compression — file reads, tool output, history

Each tool_result, file read, and (once the window fills) stale history turn is compressed by the 4B model down to ~26% of its original size, tagged [REF:id]. This is where the trained model earns its keep: it knows a function signature from a debug line, so it protects identifiers, paths, and error strings while dropping the noise.

• Single-turn this is the smaller lever (a few % — most of a turn's cost is the fixed prefix). Its power shows up across a session — see below.
• Every compressed segment is recoverable: the agent calls read_original / expand_context to pull back the exact untouched bytes, locally and instantly.

3. History summarization — keep long sessions under the window

Turns beyond the recent window are summarized once the context fills up, so a long session stays inside the model's context window instead of overflowing (or forcing an aggressive client-side compaction that drops detail).

📈 Savings compound over a session

This is the part a single-turn benchmark hides. In a real multi-turn session the two levers grow at different rates, and together they compound.

We ran the same read-only "find the bug" task for 5 consecutive turns in one Claude Code session (Sonnet, GPU model). Important: in this A/B the tool filter was left on for both sides, so the delta below isolates content compression alone — it does not include the tool-schema saving.

Content compression only (tool filter on both sides):

But that's only one of the three levers. The real "do I use Paritok or not?" comparison must add the tool filter back — and without Paritok the agent sends the entire ~29K tool-schema block every turn, not the filtered ~8K. Folding that ~21K/turn back onto the no-Paritok side:

Full stack (filter + compression vs. no Paritok at all):

So the 4.6% / 22.2% above is the floor (content only). Against a real no-Paritok baseline it's ~25% on turn 1, past ~39% by turn 5 — because the tool filter saves a fixed ~21K every turn on top of the compounding content compression.

Why it grows: every file you read stays in history and is re-sent (cache-read) every subsequent turn — so content compression keeps paying off turn after turn, while the tool filter adds a fixed cut on top.

• Content compression → quadratic. Cumulative saving ≈ 3,350 × N² — each turn's compressed reads keep paying off on every later turn.
• Tool filter → linear. Cumulative saving ≈ 21,000 × N — a fixed block saved every turn.
• Crossover ≈ turn 6: early on the tool filter dominates; past ~turn 6 content compression overtakes it and the gap widens.

Plugging these formulas into a range of N (baseline ~96,500 tokens/turn), capped at a ~200K context budget (typical Sonnet-tier configuration; larger-context models like Opus 1M push the flatten point out proportionally):

_{Capped at a ~200K context budget. The quadratic only holds while history is still growing. Once the accumulated context fills the configured budget (around turn ~8–12 here on Sonnet-tier ~200K), client-side compaction holds it flat, per-turn content saving stops growing (freezes at ~48K/turn), and % saved plateaus toward the ~72% default ceiling instead of diverging. Turns 1–5 match the measured tables above; later rows are in-window projections.}

Ceilings shift with deployment shape:

_{The projection above uses the default ceiling. MCP-heavy setups earn a bigger per-turn tool-filter cut; context-saturated sessions get an even larger effective saving because they're now compared against a compacted, information-lossy baseline.}

Honest cap: the quadratic doesn't run forever — whatever session context budget you configure bounds it. In practice a ~200K budget (typical for Sonnet-tier deployments) makes the curve flatten around turn ~12–20 as the window fills; larger-context models like Opus 1M push the flatten point out proportionally. But that ceiling is itself a feature: because each turn's prefix is smaller, the agent fits more turns before hitting the budget — Paritok effectively buys back context length.

Same budget, more room to think. Because each turn's prefix is smaller, the agent fits far more turns in the same window before compaction kicks in.

That's roughly 3× the runway on any budget, giving the agent more turns to think and finish the task before hitting the wall.

One line: use more, save more. Compression frees up the window and lets the agent go deeper and longer in the same session. Strongest on long, multi-turn, read-heavy work (auditing, Q&A over a big codebase, long debugging sessions).

💰 Cost Impact

The 74% figure is Paritok's content compression rate — file reads, tool output, and history shrink to ~26% of their size. End-to-end token savings vary by session length and workload:

Dollar impact at Claude Sonnet input pricing ($3 / M input tokens):

_{Turn size assumed ~15K. Session-length reference: edit-heavy ~1-3 turns, mixed ~5-10 turns, read-heavy ~15+ turns. Read-heavy (~75%) reflects typical MCP-heavy deployments (~78% ceiling); the default ~40-tool projection tops out at ~72%. See the compounding table above for the turn-by-turn breakdown, and the deployment-scenarios table for how the ceiling shifts with tool count and session saturation.}

Deployment overhead pays for itself in days, not weeks — and there's no lock-in: it's your own 4B model on your own hardware.

💵 Pricing

Self-host (Apache 2.0): Free forever. The gateway, the 4B model, and every training script ship open. Run on your own hardware. No telemetry, no key, no fees.

Hosted GPU (paritok.com): $0.30 per 1M tokens processed. No GPU required. Free through end of August as we launch, with full pricing kicking in September 1st.

🚀 Quick Start

Fastest path (self-host, no clone needed)

Everything ships in the PyPI package — you do not need to git clone the repo. In a fresh environment (with Ollama installed):

pip install "paritok[proxy]"     # the gateway + CLI
paritok up                       # pulls the model if missing, then starts the proxy

paritok up checks Ollama, ollama pulls paritok/paritok-4b-v1 and tags it as the local paritok-4b-v1 if it isn't already there (~2.5GB, first run only), then serves on port 8080.

Leave that terminal running. The proxy is a foreground server — every agent request flows through it, so it must stay up for the whole session. Open a separate terminal for the next step.

In the shell that launches your agent:

export ANTHROPIC_BASE_URL=http://127.0.0.1:8080   # then start Claude Code / Cursor / …

No config file is needed — paritok up and paritok proxy run on built-in defaults. Run paritok init to drop a starter paritok.yaml only if you want to tweak settings.

q4 vs full precision. paritok up uses the q4 model (:latest, ~2.5GB) by default. For full precision, run paritok up --registry-model paritok/paritok-4b-v1:f16 (~8GB). If you already ollama pulled either variant, up auto-detects which one you have — no re-download.

1. Install

pip install "paritok[proxy]"
# or, from a clone of this repo:
pip install -e ".[proxy]"

Enable the tool-schema filter (recommended — it's the biggest single-turn lever):

pip install "paritok[toolselect]"   # adds sentence-transformers (CPU-only)

First request warms up the embedding model (~10–15s — downloads bge-small once, then caches locally). Every request after is instant (~15 ms).

2. Pick a backend — self-host or the GPU server

One boolean in paritok.yaml decides where compression runs:

use_gpu_server: false   # ← the only switch that matters

Option A — self-host (false, default). Run the open 4B model on your own machine. No key, nothing leaves your box. Simplest is Ollama:

ollama pull paritok/paritok-4b-v1                # one-time, ~2.5GB
ollama cp   paritok/paritok-4b-v1 paritok-4b-v1  # tag it as the runtime name

Default paritok.yaml already points local_model at Ollama — nothing else to set.

pip install vllm
vllm serve Qwen/Qwen3-4B-Instruct-2507 \
  --enable-lora \
  --lora-modules paritok-4b-v1=paritok/paritok-4b-v1 \
  --max-lora-rank 32 \
  --port 8000

Option B — Paritok GPU server (true). No GPU required. Create an API key at paritok.com → dashboard → API keys, then:

use_gpu_server: true
gpu_server:
  api_key: "pk_live_..."   # or: export PARITOK_API_KEY=pk_live_...

3. Start the proxy

paritok proxy --port 8080 --config-file paritok.yaml

Keep this terminal open — the proxy must stay running for the whole session; run your agent from a separate terminal. On startup it checks the backend and warns (never aborts) if it can't reach one.

4. Point your agent at it

Set the base URL in the shell that launches your agent, then start the agent:

# macOS / Linux
export ANTHROPIC_BASE_URL=http://127.0.0.1:8080   # Claude Code
export OPENAI_BASE_URL=http://127.0.0.1:8080      # Cursor / OpenAI-SDK agents

# Windows PowerShell
$env:ANTHROPIC_BASE_URL = "http://127.0.0.1:8080"
$env:OPENAI_BASE_URL    = "http://127.0.0.1:8080"

Keep your real provider API key set as usual (ANTHROPIC_API_KEY / OPENAI_API_KEY) — the proxy only rewrites the request body and forwards your headers upstream. Compressed prompts go upstream; original responses come back unchanged.

Codex is the exception — it does not read OPENAI_BASE_URL. The Codex CLI takes its endpoint from ~/.codex/config.toml. See Codex setup.

Any OpenAI-compatible upstream (Groq, Gemini, OpenRouter, …)

The gateway is independent of the upstream LLM — point the OpenAI path at any OpenAI-Chat-Completions-compatible endpoint with --openai-url:

paritok proxy --openai-url https://api.groq.com/openai        # Groq
paritok proxy --openai-url https://openrouter.ai/api/v1       # OpenRouter

Then point your OpenAI-SDK agent at the proxy (OPENAI_BASE_URL=http://127.0.0.1:8080), set OPENAI_API_KEY to the provider's key, and use that provider's model names.

Codex CLI

Codex ignores OPENAI_BASE_URL — it only reads its endpoint from ~/.codex/config.toml, so Paritok writes that file for you:

codex:
  enabled: true            # paritok writes ~/.codex/config.toml on `paritok up`
  model: gpt-5             # any model your key can call
  api_key: "sk-..."        # your OpenAI key (or leave "" to use env OPENAI_API_KEY)

paritok up     # writes ~/.codex/config.toml (backs up any existing one)
codex          # in another shell — now routed through Paritok

Codex custom providers only speak the responses wire protocol, which the proxy serves at /v1/responses.

5. Check it's working

curl http://127.0.0.1:8080/health   # {"status":"ok","version":"..."}
curl http://127.0.0.1:8080/stats    # live compression totals

/stats reports cumulative savings across the session:

{
  "total_requests": 42,
  "input_tokens_original": 512340,
  "input_tokens_compressed": 138221,
  "compression_ratio": 0.27,
  "tokens_saved": 374119,
  "estimated_cost_saved_usd": "$1.01"
}

These numbers are scoped to what Paritok actually intervenes in — the content it compresses plus the tool schemas it stubs. Everything it can't affect (your system prompt, the model's output) is excluded. estimated_cost_saved_usd is cache-aware: compressed content is priced at the full base rate (new input each turn), while the frozen tool block is priced as a prompt-cache hit after turn one — counting it at full list price would massively overstate savings.

SDK mode (alternative)

import anthropic
import paritok

client = paritok.ParitokClient(anthropic.Anthropic())
resp = client.messages.create(
    model="claude-sonnet-4-20250514", max_tokens=4096, messages=[...]
)
print(resp._paritok_savings.saved_tokens, resp._paritok_savings.ratio)

🧩 When to use it

Paritok is most useful when:

• Your sessions are long and multi-turn. Savings compound from ~25% at turn 1 to 60%+ by turn 20.
• You're read-heavy or mixed: auditing, Q&A, debugging over a large codebase.
• You have many MCP tools (40+). The tool filter alone drops ~21K tokens per turn.
• You're paying premium input rates (Claude Sonnet $3/M, Opus $5/M, GPT-5 $10/M).
• You want to fit deeper sessions in the same context window and avoid forced client-side compaction.

🧠 The engine: 4B compression model

The gateway's content compression is powered by Paritok-4B-v1, the first open-source compression model trained end-to-end on real coding-agent trajectories. This section is the engine spec — you don't need it to run the gateway, but it's why the compression keeps what matters.

Highlights

• 🎨 Code-native. Trained on real coding-agent trajectories (file_read, bash_command, log_output, …). It knows what an import statement is worth vs a debug line, so it protects function names, paths, and error strings while compressing.
• 🚀 Compresses each segment to 25.7% of original — 2× harder than gpt-4.1-mini (50.2% CR) and 2.4× harder than gpt-5 (61.9% CR).
• 🎯 Retains 86.5% of full-context solve quality on SWE-bench Lite — matching gpt-4.1-mini as compressor at less than half the token spend.
• 🪶 Small & self-hostable — 4B LoRA adapter, bf16, runs on a single 24GB GPU. No SaaS, no lock-in, no per-token compressor fee.
• 🔓 Fully open — Apache 2.0 weights, reproducible data pipeline, real end-to-end SWE-bench numbers.

Benchmark: SWE-bench Lite

Real end-to-end evaluation. An agent scaffold receives its context through each compressor, then attempts to resolve the issue. Primary metric is quality retained (solve rate normalized to the uncompressed baseline).

_{¹ Quality retained = compressor solve rate ÷ uncompressed baseline solve rate. Higher is better.}

The benchmark is a floor, not a ceiling. The 86.5% measures the raw 4B model with no recall enabled — compressed output fed straight to the agent. What you actually deploy is the gateway: every segment is tagged [REF:id] and the agent can call read_original to pull back the exact bytes at any time. Nothing is permanently discarded, so real-world deployment recovers quality the raw benchmark leaves on the table. We publish the raw-model number because it's the honest, reproducible floor.

How Paritok compares

🗺️ Roadmap

• 🎯 Paritok-4B-v2. Next-gen pipeline pushing compression under 20% while closing the gap to uncompressed solve rate.
• 📈 Frontier-scale backbones. Larger models (10B+) for multi-day sessions with 100K+ token histories.
• 🌍 Multi-language expansion. First-class TypeScript, Rust, Go, Java, C++, Kotlin — v1 is Python-heavy but the architecture is language-agnostic.
• 🔌 Native integrations. Drop-in mcp add paritok plugin for Claude Code and Cursor.
• ⚙️ Adaptive compression. Per-segment auto-selection of aggressiveness based on age, kind, and downstream intent — no manual tuning.

👥 Team

Paritok is built by two engineers — no big lab, no external funding, just months of GPU budget and eval iteration.

• Jiayu Shi — training, modeling, reward design, data pipeline.
• Luzhuo Chen — evaluation, deployment, product, data pipeline.

Reach us: paritok9@gmail.com · X @Paritok

📖 Citation

@misc{paritok-4b-v1,
  author       = {Paritok Team},
  title        = {Paritok-4B-v1: An Open-Source Compression Model for AI Coding Agents},
  year         = {2026},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/Paritok-official/paritok-4b-v1}},
}

📄 License

Apache 2.0 — see LICENSE. The base model, Qwen3-4B-Instruct-2507, is released under its own license.

💬 Community & Support

• 👥 Join the community → Discord
• 🐛 Bug reports & feature requests → GitHub Issues
• 💭 Discussion → 🤗 HF Model discussions
• 📧 Contact → paritok9@gmail.com