⚡ TokenSafe (tokensafe)

Autonomous Zero-Loss Token Economy & Context Optimizer for AI Coding Agents.
Slashes LLM token costs by up to 95% with zero quality loss. Built for Google Antigravity, OpenAI Codex, Cursor, Claude Desktop, Z.ai Code, Windsurf & Cline.

🌐 Official Website: https://tokensafe.dev

⚡ 1-Click Universal Install

Connect TokenSafe to all detected AI coding agents on your machine in seconds:

# Option 1: Zero-Install via npx (Recommended for Cursor & Claude Desktop)
npx tokensafe

# Option 2: Python / PyPI
pip install tokensafe && tokensafe

# Option 3: Direct Shell Script (macOS / Linux)
curl -fsSL https://tokensafe.dev/install.sh | bash

🔌 Universal MCP Server Configuration

Connect TokenSafe directly to Claude Desktop (claude_desktop_config.json) or Cursor MCP Settings:

{
  "mcpServers": {
    "tokensafe": {
      "command": "npx",
      "args": ["-y", "tokensafe", "mcp"]
    }
  }
}

🚀 The Breakthrough: How It Saves 60%–90% Tokens

Traditional AI code interactions suffer from massive context bloat:

1. The BPE Token Penalty: Verbose natural language prompts and filler waste 40%–70% of ingress context.
2. Bulk Tool Dumps: Reading whole 800+ line files burns 5,000+ tokens per turn when only 20 lines are needed.
3. Stuck Loops & Bloated Diffs: Terminal git diff and repetitive failed commands spiral context into overflow.

TokenSafe Solves All Three Autonomously:

• Graph-Guided AST Skeletonization: Folds inactive function bodies into signatures, keeping target functions and 1-hop dependencies full while collapsing unrelated code (-85% to -92%).
• Prompt Cache Prefix Normalizer: Strips volatile timestamps and aligns system prompts to 64-token boundaries. Injects Anthropic ephemeral breakpoints (cache_control: {"type": "ephemeral"}) and DeepSeek chunking for 100% provider cache hits and 90% cost discounts.
• Context Window Governor & Token Ceiling: Enforces deterministic token budgets via 3-stage cascading autothrottling (AST skeletonization ➔ TOCP user message history compaction ➔ sliding state ledger) to prevent catastrophic context overflows.
• Subagent Deduplication Firewall: Detects repeated file reads across subagents and agent swarms, serving cached AST slices directly from SQLite session memory (SUBAGENT_DEDUP) with zero redundant overhead.
• AST Diff Slicer & Lockfile Stripper: Compresses code review diffs by stripping noisy lockfiles (package-lock.json, pnpm-lock.yaml, yarn.lock, Cargo.lock) and minified assets, condensing diffs to 2 lines of context (git diff -U2) for up to 96% token savings.
• God Nodes & Architectural Hub Detection: Calculates Degree Centrality to identify structural project hubs and injects topology awareness into agent sessions without expensive file crawling.
• TOCP Prompt Distillation: Intercepts prompts and distills intent into lean S-Expressions via Google Gemini (2.0 Flash-Lite), OpenAI ChatGPT (gpt-4o-mini), Anthropic Claude, or DeepSeek-Flash (-40% to -70%). Automatically detects GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY, or DEEPSEEK_API_KEY.
• API Surface Map Generator: Generates high-density tokensafe-out/API_SURFACE.md (~1.5k tokens) for instant agent codebase onboarding.
• Predictive Dependency Prefetcher: Attaches imported symbol signatures to eliminate downstream exploratory read_file roundtrips.
• Live SSE Streaming Dashboard: Real-time reactive Web UI streaming token reductions, live event logs, and USD savings via Server-Sent Events (/api/events).
• Friday Team ROI Digest: Automatically aggregates 7-day token economics every Friday and emails team leads an executive HTML report with engineer savings leaderboards.
• Environment Overhead Auditor (tokensafe doctor): Inspects active MCP servers and Antigravity skills, measuring system prompt token tax and pruning redundant tool schemas.
• MCP Tool Schema Bloat Guard: Stops heavy MCP servers (50–200+ tools) from injecting 5,000–15,000 hidden schema tokens into every API turn. 1-click toggle (tokensafe mcp disable <name>) and real-time in-chat bloat alerts.
• Data Retention & Compact Rollups (tokensafe prune / vacuum): Automated sliding-window log cleanup with lifetime metric rollups, cutting telemetry database disk footprint by up to 40%.
• Local Secret Redactor: Sanitizes Stripe, AWS, JWT, and database credentials before network transmission.
• Output Guard & Loop Guard: Blocks destructive full-file rewrites on files $\ge 40$ lines and halts command loops.

🛡️ The Hidden Context Killer: MCP Tool Schema Bloat

Why do AI coding agent quotas (like Antigravity's 5-Hour Limit or Claude/OpenAI monthly token budgets) evaporate after just a few prompts?

The Hidden System Prompt Tax

Whenever an MCP server is configured in your editor (Cursor, Claude Desktop, Antigravity, Windsurf), the complete JSON schema for every single tool is injected into the LLM's system prompt on EVERY API TURN — even if you never use that tool!

┌────────────────────────────────────────────────────────────────────────┐
│  SYSTEM PROMPT ANATOMY (EVERY SINGLE TURN)                            │
├────────────────────────────────────────────────────────────────────────┤
│  Base System Instructions                                ~1,500 tokens │
│  Active Workspace Skills & Prompts                       ~8,000 tokens │
│  Regular MCP Tools (context7, prisma, etc.)                ~800 tokens │
│  🚨 Heavy MCP Tools (200+ schemas from cloud/re tools)   ~6,000 tokens │
├────────────────────────────────────────────────────────────────────────┤
│  TOTAL HIDDEN BASE TAX BEFORE YOU TYPE A SINGLE CHARACTER:             │
│  🔥 ~16,300 tokens PER TURN × 15 turns = 244,500 tokens burned!       │
└────────────────────────────────────────────────────────────────────────┘

• 1 Server with 50 tools = ~1,500 – 2,500 tokens of static tax on every turn.
• 1 Server with 200+ tools = ~5,000 – 10,000 tokens per turn!
• In a 15-turn coding task, 75,000 to 150,000 tokens are burned on tool schemas alone.

How TokenSafe Eliminates MCP Bloat:

1. tokensafe doctor (Context Tax Auditor): Scans all configured MCP servers and active skills, measures the exact token overhead per turn, and flags servers with $\ge 40$ tools as CRITICAL BLOAT.
2. 1-Click MCP Management: Instantly toggle heavy servers without editing JSON files manually:

   tokensafe mcp disable rea     # Prunes 208 schemas from system prompt
   tokensafe mcp enable rea      # Re-enables when actively needed
   

3. PreInvocation Bloat Alerts: Live in-chat warning badges in the IDE when critical MCP bloat is detected on session start.
4. Data Retention & Compact Rollups: Automatic rolling-window event cleanup (tokensafe prune and tokensafe vacuum) prevents SQLite telemetry from ballooning on disk.

⚡ TokenSafe vs. Graphify: Dynamic In-Flight Defense vs. Static Reports

How TokenSafe complements or supersedes static graph tools like Graphify (graphify-labs/graphify):

Complements & Replaces: If you already use Graphify for repository mapping, TokenSafe works alongside it by compressing real-time agent tool reads on the wire. Or use TokenSafe standalone for a zero-configuration, zero-cost, sub-30ms architecture engine out of the box.

🤖 Universal LLM Distillation Providers

TokenSafe supports any key you already have:

Check active providers anytime with:

tokensafe providers

📊 Zero-Loss SWE-bench Benchmark Results

Verified across 8 real-world production software engineering scenarios (131,604 baseline tokens down to 7,251 tokens, -94.5% net token reduction, +28.1% turn latency speedup with 100% Zero-Loss accuracy retention):

💎 Pricing & 7-Day Free Trial

👉 Start your 7-Day Free Trial on our website

🛠️ CLI Quickstart

# Analyze repository God Nodes and dependency topology
tokensafe graph

# Generate compact tokensafe-out/API_SURFACE.md interface snapshot
tokensafe map

# Run environment health check and audit MCP/skill context token tax
tokensafe doctor

# View live savings dashboard in your terminal
tokensafe stats

# Open interactive Apple dark-mode HTML dashboard
tokensafe html

# Generate or preview Friday team ROI digest email
tokensafe digest

# Manage or disable high-overhead MCP tools
tokensafe mcp list
tokensafe mcp disable <server>

# Optimize SQLite metrics storage
tokensafe vacuum

# Run the 5-scenario benchmark suite
tokensafe benchmark

# Run the local transparent AI proxy (0.0.0.0:8765/v1)
tokensafe proxy

# Activate your Pro License key (TS2 Asymmetric Ed25519 Key)
tokensafe activate TS2.eyJlbWFpbCI6...

🐳 Docker Deployment (Multi-Provider Support)

TokenSafe is fully containerized with a hardened multi-stage production Docker image, non-root security (uid: 1000), and native multi-provider LLM routing.

1. Multi-Provider Environment Keys

You can configure any or all of your preferred LLM provider keys:

2. Docker Compose Quickstart

Run TokenSafe as a centralized microservice or team gateway:

version: '3.8'

services:
  tokensafe:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: tokensafe
    restart: unless-stopped
    ports:
      - "8765:8765"
    environment:
      - TOKENSAFE_HOST=0.0.0.0
      - TOKENSAFE_PORT=8765
      - TOKENSAFE_DATA_DIR=/app/data
      # Choose your provider(s):
      - DEEPSEEK_API_KEY=${DEEPSEEK_API_KEY:-}
      - OPENAI_API_KEY=${OPENAI_API_KEY:-}
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY:-}
      - GEMINI_API_KEY=${GEMINI_API_KEY:-}
      - TOKENSAFE_LICENSE_KEY=${TOKENSAFE_LICENSE_KEY:-}
    volumes:
      - ./data:/app/data
    healthcheck:
      test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8765/api/health')"]
      interval: 30s
      timeout: 5s
      retries: 3

Start the container in detached mode:

docker compose up -d

3. Verification & Live SSE Streaming

• Health check: curl http://localhost:8765/api/health
• Live SSE Event Stream: curl -N http://localhost:8765/api/events
• Point your AI Agents: Configure your IDE or Agent to use http://localhost:8765/v1 as the upstream base URL.

🔒 Security Architecture & Cloud API Endpoints

TokenSafe Cloud Backend enforces enterprise zero-trust security across all communications:

1. Cryptographic Authentication & Licensing

• Asymmetric RFC 8032 Ed25519 Signatures (TS2.*): Client instances verify digital signatures offline using an embedded 32-byte public key. The private signing key never leaves the secure cloud enclave.
• JWT Session Tokens: Cloud endpoints authenticate sessions via standard HS256 JWT bearer tokens or Clerk session tokens.
• Dual Bearer Support: Client endpoints accept either a signed license key (TS2.* / TS1.*) or a JWT in the Authorization: Bearer <token> header.
• Zero Hardcoded Secrets: All SMTP, Stripe, Admin, and signing keys are strictly injected via environment variables.

2. Cloud API Endpoint Reference

3. Server Configuration & Environment Variables

🧪 Unit Testing

python3 -m unittest discover tests

Zero external runtime dependencies — runs on standard Python 3.10+.

📄 License

MIT License. Commercial enterprise licenses available for on-premise B2B deployments.

Export Markdown Report

./bin/agy-tokens markdown

📂 Project Structure

agy-token-optimizer/
├── plugin.json                 # Antigravity plugin manifest
├── hooks.json                  # Lifecycle hook declarations (PreInvocation, PreToolUse, PostToolUse)
├── Dockerfile                  # Hardened multi-stage container deployment
├── docker-compose.yml          # Production Docker compose with multi-provider routing
├── rules/
│   └── AGENTS.md               # TOCP Protocol directives for LLM (prompt-cached)
├── skills/
│   └── token-optimizer/
│       └── SKILL.md            # On-demand Antigravity skill
├── proto/
│   └── tocp.proto              # Protocol Buffers contract
├── src/
│   ├── cache_normalizer.py     # Prompt Cache Prefix Normalizer (Anthropic & DeepSeek)
│   ├── governor.py             # Context Window Governor & Token Ceiling (Cascading reduction)
│   ├── agent_firewall.py       # Cross-Agent Deduplication Cache (Subagent Firewall)
│   ├── diff_slicer.py          # AST Diff Slicer & Lockfile Stripper (`git diff -U2`)
│   ├── weekly_digest.py        # Friday Team ROI Digest & Engineer Leaderboard
│   ├── environment_auditor.py  # MCP Server & Skill context token tax auditor
│   ├── proxy.py                # Multi-provider transparent AI proxy & SSE broadcaster
│   ├── graph_engine.py         # Sub-30ms AST parser, call graphs & God Node centrality
│   ├── tokenizer.py            # Zero-dependency calibrated BPE token counter
│   ├── compactor.py            # Semantic compactor (Natural Language -> TOCP S-Expr)
│   ├── slicer.py               # AST & Line windowing tool optimizer
│   ├── metrics.py              # Persistent SQLite metrics engine
│   ├── dashboard.py            # Visual ANSI terminal, Markdown & reactive SSE web dashboard
│   ├── license_manager.py      # Ed25519 cryptographic license verification
│   ├── mailer.py               # Transactional email & Friday ROI digest dispatcher
│   └── cloud_backend.py        # Cloud REST backend, telemetry sync & Stripe webhooks
├── hooks/
│   ├── pre_invocation.py       # PreInvocation hook entry point
│   ├── pre_tool_use.py         # PreToolUse hook entry point
│   └── post_tool_use.py        # PostToolUse hook entry point
├── frontend/                   # Modern React / Vite / Tailwind Web App & Dashboard
├── bin/
│   └── agy-tokens              # Executable CLI (`tokensafe`)
├── tests/                      # Unit test suite (100% passing)
├── install.sh                  # One-click installer for Antigravity
└── README.md

🛠️ Installation & Antigravity Setup

Run the installer:

./install.sh

This symlinks the plugin directly into ~/.gemini/config/plugins/agy-token-optimizer.
Antigravity automatically loads it on startup.

🧪 Running Unit Tests

python3 -m unittest discover tests

No external packages required — runs on standard Python 3.10+.