tokensqueeze 0.2.0

Reduce LLM conversation tokens by up to 70%

  _        _                                             
 | |_ ___ | | _____ _ __  ___  __ _ _   _  ___  ___ _______ 
 | __/ _ \| |/ / _ \ '_ \/ __|/ _` | | | |/ _ \/ _ \_  / _ \
 | || (_) |   <  __/ | | \__ \ (_| | |_| |  __/  __// /  __/
  \__\___/|_|\_\___|_| |_|___/\__, |\__,_|\___|\___/___\___|
                                 |_|                         

Reduce LLM conversation tokens by up to 91%. Drop-in compression for any OpenAI-compatible API.

---

Why

Every turn in an LLM conversation resends the entire history. A 30-turn conversation can burn 50,000+ tokens per request — most of it redundant context the model has already processed.

That's wasted money and wasted context window.

tokensqueeze compresses your conversation history before each API call. One line of code, 91% fewer tokens.

Real-World Results

Tested on 20 real Claude Code conversations (1,200 messages):

  114,481 total tokens → 10,146 tokens (91.1% savings)

Conversation	Messages	Before	After	Saved
Coding session #1	60	7,686	261	96.6%
Coding session #2	60	11,356	377	96.7%
Coding session #3	60	6,524	246	96.2%
Average (20 conversations)	60	5,724	507	91.1%

These aren't synthetic benchmarks — they're actual development conversations with tool calls, file reads, and multi-step debugging.

Install

pip install tokensqueeze

For LLM-powered compression (recommended), install Ollama and pull a small model:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2:3b   # 2GB, runs on any machine

Quick Start

Use with Claude Code, Cursor, or any AI tool

tokensqueeze runs as a local proxy — no code changes needed:

# Terminal 1: start the proxy
tokensqueeze proxy

# Terminal 2: use Claude Code through it
ANTHROPIC_BASE_URL=http://localhost:8080 claude

That's it. Every API call now gets compressed automatically. The proxy logs savings in real time:

[tokensqueeze] /v1/messages — 12,847 → 3,854 tokens (70% saved)
[tokensqueeze] /v1/messages — 18,231 → 5,102 tokens (72% saved)

Works with anything that lets you set a base URL: Claude Code, Cursor, OpenAI SDK apps, LangChain, etc.

For maximum compression (requires Ollama):

tokensqueeze proxy --use-llm

Use as a Python library

If you're building your own app, one line:

from openai import OpenAI
from tokensqueeze import squeeze

client = OpenAI()
messages = [...]  # your conversation history

result = squeeze(messages, use_llm=True)
print(result)  # CompressionResult(5,724 → 507 tokens, saved 91.1%)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=result.messages  # compressed
)

How It Works

Four strategies applied in sequence, each targeting a different type of waste:

messages → [Dedup] → [Trim] → [Condense] → [Summarize] → compressed

#	Strategy	What it does	Savings
1	Deduplication	Removes repeated content across messages (re-pasted code, duplicate tool results)	5-15%
2	Tool result trimming	Shrinks large file reads and API responses that have already been processed	15-25%
3	Condensation	Compresses individual verbose messages in place via local LLM	20-40%
4	LLM summarization	Compresses old turns into structured fact-preserving notes	30-50%

Strategies are composable — use the defaults, pick specific ones, or write your own.

What compressed context looks like

[Conversation context — 14 earlier messages]
- Topic: Troubleshooting Flask application issues
- ERROR: ImportError: cannot import name celery_app → Fix: separate celery_config.py
- ERROR: ValueError: invalid literal for int() → FIX: @app.route(/users/<int:id>)
- ISSUE: stale cache data → FIX: cache.delete(f"user:{user_id}") after commit
- ERROR: .exe upload accepted → FIX: extension allowlist + python-magic MIME check
- ISSUE: race condition on inventory → FIX: SELECT FOR UPDATE with_for_update()

Exact error messages, function names, and decisions are preserved — not just a vague summary.

Two Modes

	Extractive	LLM (Ollama)
How	Keyword extraction + sliding window	Local LLM condensation + structured summarization
Speed	<1ms	1-15s
Requires	tiktoken only	Ollama + llama3.2:3b (free)
Savings	40-70%	85-97%
Quality	81/100	79-89/100

# Extractive — fast, no dependencies
result = squeeze(messages)

# LLM — much better compression, requires Ollama
result = squeeze(messages, use_llm=True)

If Ollama isn't running, use_llm=True falls back to extractive automatically.

CLI

# Start the compression proxy
tokensqueeze proxy                              # Anthropic (default)
tokensqueeze proxy --target openai              # OpenAI
tokensqueeze proxy --use-llm                    # LLM compression via Ollama
tokensqueeze proxy --port 9000                  # Custom port

# Compress a conversation file
tokensqueeze compress conversation.json -o compressed.json
tokensqueeze compress --use-llm --stats-only conversation.json
tokensqueeze compress --use-llm --score conversation.json

Quality Scoring

tokensqueeze includes a built-in quality judge that measures whether compressed conversations still produce equivalent LLM responses:

result = squeeze(messages, use_llm=True)
quality = result.score_quality()

print(quality)         # QualityScore(overall=79/100, faith=80, comp=70, coher=90)
print(quality.grade)   # "good"

Score	Grade	Meaning
90-100	Lossless	All meaningful context preserved
70-89	Good	Minor details lost, core meaning intact
50-69	Degraded	Noticeable information loss
<50	Broken	Too much context destroyed

Advanced Usage

Custom strategy pipeline

from tokensqueeze import squeeze, LLMSummarizeStrategy, ToolResultTrimStrategy, CondenseStrategy

result = squeeze(
    messages,
    strategies=[
        ToolResultTrimStrategy(max_tokens=100, preserve_recent=1),
        CondenseStrategy(min_tokens=50),
        LLMSummarizeStrategy(window_size=2, ollama_model="llama3.2:3b"),
    ]
)

Compressor instance with timeout

from tokensqueeze import Compressor

compressor = Compressor(model="gpt-4o", max_time=30.0)  # 30s pipeline timeout
result = compressor.compress(messages)

print(f"Saved {result.saved_tokens:,} tokens ({result.savings_pct:.1f}%)")
for strategy, saved in result.strategy_stats.items():
    print(f"  {strategy}: -{saved:,}")

Custom strategy

from tokensqueeze import Strategy

class MyStrategy(Strategy):
    def compress(self, messages, **kwargs):
        # Your compression logic here
        return messages

Remote or alternative LLM

result = squeeze(
    messages,
    use_llm=True,
    ollama_model="mistral:7b",
    ollama_url="http://your-server:11434",
)

Any OpenAI-compatible endpoint works — Ollama, vLLM, LM Studio, or a remote server.

Input Format

Standard OpenAI message format:

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "Hello"},
  {"role": "assistant", "content": "Hi there!"},
  {"role": "user", "content": "..."}
]

How tokensqueeze Compares

Other tools in the token reduction space solve different parts of the problem:

	tokensqueeze	Caveman	RTK
Compresses	Conversation history (input)	LLM responses (output)	Shell command output (input)
Technique	Dedup + condense + LLM summarize	Prompt engineering ("be terse")	Per-command Rust filters
Reduction	67-97% input tokens	~65% output tokens	60-90% tool output
Target	Any LLM app or tool (proxy mode)	AI coding agents	AI coding agents
Integration	One env var or one line of code	Config file install	Shell hook

These tools are complementary. Use all three together for maximum savings:

• RTK compresses tool results as they enter the conversation
• Caveman makes the LLM respond more concisely
• tokensqueeze compresses the accumulated history before each API call

Measuring Your Savings

tokensqueeze reports savings on every call:

result = squeeze(messages, use_llm=True)
print(result)                 # CompressionResult(5,724 → 507 tokens, saved 91.1%)
print(result.strategy_stats)  # per-strategy breakdown

For tracking overall token costs across your workflow, ccusage is a useful companion — it reads usage logs from coding agents and generates cost reports:

npx ccusage@latest
ccusage claude daily  # compare before and after adding tokensqueeze

License

MIT