tokensqueeze 0.2.0

  _        _                                             
 | |_ ___ | | _____ _ __  ___  __ _ _   _  ___  ___ _______ 
 | __/ _ \| |/ / _ \ '_ \/ __|/ _` | | | |/ _ \/ _ \_  / _ \
 | || (_) |   <  __/ | | \__ \ (_| | |_| |  __/  __// /  __/
  \__\___/|_|\_\___|_| |_|___/\__, |\__,_|\___|\___/___\___|
                                 |_|                         

Reduce LLM conversation tokens by up to 91%. Drop-in compression for any OpenAI-compatible API.

Why

Every turn in an LLM conversation resends the entire history. A 30-turn conversation can burn 50,000+ tokens per request — most of it redundant context the model has already processed.

That's wasted money and wasted context window.

tokensqueeze compresses your conversation history before each API call. One line of code, 91% fewer tokens.

Real-World Results

Tested on 20 real Claude Code conversations (1,200 messages):

  114,481 total tokens → 10,146 tokens (91.1% savings)

These aren't synthetic benchmarks — they're actual development conversations with tool calls, file reads, and multi-step debugging.

Install

pip install tokensqueeze

For LLM-powered compression (recommended), install Ollama and pull a small model:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2:3b   # 2GB, runs on any machine

Quick Start

Use with Claude Code, Cursor, or any AI tool

tokensqueeze runs as a local proxy — no code changes needed:

# Terminal 1: start the proxy
tokensqueeze proxy

# Terminal 2: use Claude Code through it
ANTHROPIC_BASE_URL=http://localhost:8080 claude

That's it. Every API call now gets compressed automatically. The proxy logs savings in real time:

[tokensqueeze] /v1/messages — 12,847 → 3,854 tokens (70% saved)
[tokensqueeze] /v1/messages — 18,231 → 5,102 tokens (72% saved)

Works with anything that lets you set a base URL: Claude Code, Cursor, OpenAI SDK apps, LangChain, etc.

For maximum compression (requires Ollama):

tokensqueeze proxy --use-llm

Use as a Python library

If you're building your own app, one line:

from openai import OpenAI
from tokensqueeze import squeeze

client = OpenAI()
messages = [...]  # your conversation history

result = squeeze(messages, use_llm=True)
print(result)  # CompressionResult(5,724 → 507 tokens, saved 91.1%)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=result.messages  # compressed
)

How It Works

Four strategies applied in sequence, each targeting a different type of waste:

messages → [Dedup] → [Trim] → [Condense] → [Summarize] → compressed

Strategies are composable — use the defaults, pick specific ones, or write your own.

What compressed context looks like

[Conversation context — 14 earlier messages]
- Topic: Troubleshooting Flask application issues
- ERROR: ImportError: cannot import name celery_app → Fix: separate celery_config.py
- ERROR: ValueError: invalid literal for int() → FIX: @app.route(/users/<int:id>)
- ISSUE: stale cache data → FIX: cache.delete(f"user:{user_id}") after commit
- ERROR: .exe upload accepted → FIX: extension allowlist + python-magic MIME check
- ISSUE: race condition on inventory → FIX: SELECT FOR UPDATE with_for_update()

Exact error messages, function names, and decisions are preserved — not just a vague summary.

Two Modes

# Extractive — fast, no dependencies
result = squeeze(messages)

# LLM — much better compression, requires Ollama
result = squeeze(messages, use_llm=True)

If Ollama isn't running, use_llm=True falls back to extractive automatically.

CLI

# Start the compression proxy
tokensqueeze proxy                              # Anthropic (default)
tokensqueeze proxy --target openai              # OpenAI
tokensqueeze proxy --use-llm                    # LLM compression via Ollama
tokensqueeze proxy --port 9000                  # Custom port

# Compress a conversation file
tokensqueeze compress conversation.json -o compressed.json
tokensqueeze compress --use-llm --stats-only conversation.json
tokensqueeze compress --use-llm --score conversation.json

Quality Scoring

tokensqueeze includes a built-in quality judge that measures whether compressed conversations still produce equivalent LLM responses:

result = squeeze(messages, use_llm=True)
quality = result.score_quality()

print(quality)         # QualityScore(overall=79/100, faith=80, comp=70, coher=90)
print(quality.grade)   # "good"

Advanced Usage

from tokensqueeze import squeeze, LLMSummarizeStrategy, ToolResultTrimStrategy, CondenseStrategy

result = squeeze(
    messages,
    strategies=[
        ToolResultTrimStrategy(max_tokens=100, preserve_recent=1),
        CondenseStrategy(min_tokens=50),
        LLMSummarizeStrategy(window_size=2, ollama_model="llama3.2:3b"),
    ]
)

from tokensqueeze import Compressor

compressor = Compressor(model="gpt-4o", max_time=30.0)  # 30s pipeline timeout
result = compressor.compress(messages)

print(f"Saved {result.saved_tokens:,} tokens ({result.savings_pct:.1f}%)")
for strategy, saved in result.strategy_stats.items():
    print(f"  {strategy}: -{saved:,}")

from tokensqueeze import Strategy

class MyStrategy(Strategy):
    def compress(self, messages, **kwargs):
        # Your compression logic here
        return messages

result = squeeze(
    messages,
    use_llm=True,
    ollama_model="mistral:7b",
    ollama_url="http://your-server:11434",
)

Input Format

Standard OpenAI message format:

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "Hello"},
  {"role": "assistant", "content": "Hi there!"},
  {"role": "user", "content": "..."}
]

How tokensqueeze Compares

Other tools in the token reduction space solve different parts of the problem:

These tools are complementary. Use all three together for maximum savings:

• RTK compresses tool results as they enter the conversation
• Caveman makes the LLM respond more concisely
• tokensqueeze compresses the accumulated history before each API call

Measuring Your Savings

tokensqueeze reports savings on every call:

result = squeeze(messages, use_llm=True)
print(result)                 # CompressionResult(5,724 → 507 tokens, saved 91.1%)
print(result.strategy_stats)  # per-strategy breakdown

For tracking overall token costs across your workflow, ccusage is a useful companion — it reads usage logs from coding agents and generates cost reports:

npx ccusage@latest
ccusage claude daily  # compare before and after adding tokensqueeze

License

MIT