TokenCrush

LLM token optimization CLI - compress prompts, save costs.

TokenCrush uses LLMLingua-2 to compress your prompts before sending them to LLMs, reducing token usage by 50-80% while preserving response quality.

Features

• Prompt Compression: Reduce tokens by 2-5x using Microsoft's LLMLingua-2
• Multi-Provider Support: Works with OpenAI, Anthropic (Claude), and Google (Gemini)
• Simple CLI: One command to compress and chat
• Cost Savings: Significantly reduce API costs

Installation

pip install tokencrush

Or with uv:

uv add tokencrush

Quick Start

1. Configure API Keys

# Set API keys (stored in ~/.config/tokencrush/config.toml)
tokencrush config set openai sk-your-openai-key
tokencrush config set anthropic sk-ant-your-anthropic-key
tokencrush config set google your-google-api-key

# Or use environment variables
export OPENAI_API_KEY=sk-your-key

2. Compress a Prompt

tokencrush compress "Your very long prompt text here..."

Output:

┌─────────────────────────────────────┐
│        Compression Result           │
├──────────────────┬──────────────────┤
│ Original Tokens  │ 500              │
│ Compressed Tokens│ 150              │
│ Compression Ratio│ 30.0%            │
│ Tokens Saved     │ 350              │
└──────────────────┴──────────────────┘

3. Chat with Compression

# Compress and send to GPT-4
tokencrush chat "Explain quantum computing in detail" --model gpt-4

# Use Claude
tokencrush chat "Write a poem" --model claude-3-sonnet

# Use Gemini
tokencrush chat "Summarize this" --model gemini-1.5-pro

CLI Reference

tokencrush compress

Compress text to reduce tokens.

tokencrush compress <text> [OPTIONS]

Options:
  -r, --rate FLOAT    Compression rate (0.0-1.0). Default: 0.5
  -g, --gpu           Use GPU for faster compression

tokencrush chat

Compress and send to LLM.

tokencrush chat <prompt> [OPTIONS]

Options:
  -m, --model TEXT    Model to use. Default: gpt-4
  -r, --rate FLOAT    Compression rate. Default: 0.5
  -g, --gpu           Use GPU for compression
  --no-compress       Skip compression

tokencrush config

Manage API keys.

tokencrush config set <provider> <key>   # Save API key
tokencrush config show                    # Show configured keys

Supported Models

How It Works

1. Compression: TokenCrush uses LLMLingua-2's trained model to identify and remove redundant tokens while preserving meaning.
2. API Call: The compressed prompt is sent to your chosen LLM provider.
3. Response: You get the same quality response with fewer input tokens = lower cost.

Development

# Clone the repository
git clone https://github.com/xodn348/tokencrush.git
cd tokencrush

# Install with dev dependencies
uv sync --all-extras

# Run tests
uv run pytest -v

# Build package
uv build

License

MIT License - see LICENSE for details.

Acknowledgments

• LLMLingua by Microsoft Research
• LiteLLM for multi-provider support