Skip to main content

⚡ TokenSafe (tokensafe)

npm version PyPI version License: MIT SWE-bench: Verified Python: 3.10+ Trial: 7 Days Free

Autonomous Zero-Loss Token Economy & Context Optimizer for AI Coding Agents.
Slashes LLM token costs by up to 95% with zero quality loss. Built for Google Antigravity, OpenAI Codex, Cursor, Claude Desktop, Z.ai Code, Windsurf & Cline.

🌐 Official Website: https://tokensafe.dev


⚡ 1-Click Universal Install

Connect TokenSafe to all detected AI coding agents on your machine in seconds:

# Option 1: Zero-Install via npx (Recommended for Cursor & Claude Desktop)
npx tokensafe

# Option 2: Python / PyPI
pip install tokensafe && tokensafe

# Option 3: Direct Shell Script (macOS / Linux)
curl -fsSL https://tokensafe.dev/install.sh | bash

🔌 Universal MCP Server Configuration

Connect TokenSafe directly to Claude Desktop (claude_desktop_config.json) or Cursor MCP Settings:

{
  "mcpServers": {
    "tokensafe": {
      "command": "npx",
      "args": ["-y", "tokensafe", "mcp"]
    }
  }
}

🚀 The Breakthrough: How It Saves 60%–90% Tokens

Traditional AI code interactions suffer from massive context bloat:

  1. The BPE Token Penalty: Verbose natural language prompts and filler waste 40%–70% of ingress context.
  2. Bulk Tool Dumps: Reading whole 800+ line files burns 5,000+ tokens per turn when only 20 lines are needed.
  3. Stuck Loops & Bloated Diffs: Terminal git diff and repetitive failed commands spiral context into overflow.

TokenSafe Solves All Three Autonomously:

  • Graph-Guided AST Skeletonization: Folds inactive function bodies into signatures, keeping target functions and 1-hop dependencies full while collapsing unrelated code (-85% to -92%).
  • God Nodes & Architectural Hub Detection: Calculates Degree Centrality to identify structural project hubs and injects topology awareness into agent sessions without expensive file crawling.
  • TOCP Prompt Distillation: Intercepts prompts and distills intent into lean S-Expressions via Google Gemini (2.0 Flash-Lite), OpenAI ChatGPT (gpt-4o-mini), or DeepSeek-Flash (-40% to -70%). Automatically detects GEMINI_API_KEY, OPENAI_API_KEY, or DEEPSEEK_API_KEY.
  • API Surface Map Generator: Generates high-density tokensafe-out/API_SURFACE.md (~1.5k tokens) for instant agent codebase onboarding.
  • Predictive Dependency Prefetcher: Attaches imported symbol signatures to eliminate downstream exploratory read_file roundtrips.
  • Local Secret Redactor: Sanitizes Stripe, AWS, JWT, and database credentials before network transmission.
  • Output Guard & Loop Guard: Blocks destructive full-file rewrites on files $\ge 40$ lines and halts command loops.

⚡ TokenSafe vs. Graphify: Dynamic In-Flight Defense vs. Static Reports

How TokenSafe complements or supersedes static graph tools like Graphify (graphify-labs/graphify):

Capability Graphify TokenSafe
Primary Focus Static knowledge graph reports & MCP queries In-flight real-time token compression & defense
Indexing Cost Expensive LLM tokens / heavy graph extraction 0 tokens, 100% deterministic local AST (< 30ms)
Execution Latency Offline / CLI-initiated build Sub-millisecond inline proxy & lifecycle hooks
Active Slicing Static file output (GRAPH_REPORT.md) Dynamic call-graph skeletonization (85–92% saved)
Multi-Project Workspace Requires manual directory targeting Auto-detects project roots per file on the fly
Agent Prompt Optimization None TOCP S-Expression prompt distillation (-40% to -70%)
Safety & Runaway Prevention None Output rewrite guard, loop detector, secret scrubbing

Complements & Replaces: If you already use Graphify for repository mapping, TokenSafe works alongside it by compressing real-time agent tool reads on the wire. Or use TokenSafe standalone for a zero-configuration, zero-cost, sub-30ms architecture engine out of the box.


🤖 Universal LLM Distillation Providers

TokenSafe supports any key you already have:

Provider Supported Model Environment Variable Config Key
Google Gemini gemini-2.0-flash-lite, gemini-2.0-flash GEMINI_API_KEY gemini_api_key
OpenAI ChatGPT gpt-4o-mini, gpt-4o OPENAI_API_KEY openai_api_key
DeepSeek deepseek-flash, deepseek-chat DEEPSEEK_API_KEY deepseek_api_key

Check active providers anytime with:

tokensafe providers

📊 Zero-Loss SWE-bench Benchmark Results

Verified across 8 real-world production software engineering scenarios (131,604 baseline tokens down to 7,251 tokens, -94.5% net token reduction, +28.1% turn latency speedup with 100% Zero-Loss accuracy retention):

Benchmark Scenario Task Complexity Baseline Tokens TokenSafe Tokens Reduction Turn Speedup Accuracy
Zero-Token Architecture Discovery Full repo God Node indexing 49,716 486 -99.0% 6 ➔ 1 turns (+83%) 100% Pass
Graph-Guided AST Symbol Slicing Target function + 1-hop callees 8,016 451 -94.4% 4 ➔ 2 turns (+50%) 100% Pass
Monorepo Lockfile Grep Leak git grep across lockfiles 10,111 491 -95.1% 2 turns 100% Pass
Monorepo Lockfile Git Diff Monorepo branch diff 18,911 761 -96.0% 2 turns 100% Pass
Multi-Turn History Compaction 8-turn session history fold 31,010 3,130 -89.9% 8 turns 100% Pass
Race Condition / Distributed Lock AST query & Redis redlock 5,074 404 -92.0% 4 ➔ 3 turns (+25%) 100% Pass
OAuth JWT Expiration Triage Controller bug investigation 3,757 401 -89.3% 3 ➔ 2 turns (+33%) 100% Pass
CI Test Failure Extraction Full test log & stack trace 5,009 1,127 -77.5% 3 turns 100% Pass

💎 Pricing & 7-Day Free Trial

Tier Price Highlights
Hacker (Open-Core) $0 (Free Forever) 100k tokens/mo, Local Regex compactor, use own DeepSeek / Gemini key
Solo Developer $9 / month 7-Day Free Trial, Unlimited Managed DeepSeek-Flash / Gemini Flash-Lite, AST Prefetcher, Output Guards
Engineering Team $29 / dev / mo 7-Day Free Trial, Team Budget Caps, Centralized Ledger, B2B Docker Proxy
Enterprise Custom On-Premise Nuitka C-compiled Docker Proxy, Custom Compliance, SAML SSO

👉 Start your 7-Day Free Trial on our website


🛠️ CLI Quickstart

# Analyze repository God Nodes and dependency topology
tokensafe graph

# Generate compact tokensafe-out/API_SURFACE.md interface snapshot
tokensafe map

# View live savings dashboard in your terminal
tokensafe stats

# Open interactive Apple dark-mode HTML dashboard
tokensafe html

# Run the 5-scenario benchmark suite
tokensafe benchmark

# Run the local transparent AI proxy (127.0.0.1:8765/v1)
tokensafe proxy

# Activate your Pro License key (TS2 Asymmetric Ed25519 Key)
tokensafe activate TS2.eyJlbWFpbCI6...

🔒 Security Architecture & Cloud API Endpoints

TokenSafe Cloud Backend enforces enterprise zero-trust security across all communications:

1. Cryptographic Authentication & Licensing

  • Asymmetric RFC 8032 Ed25519 Signatures (TS2.*): Client instances verify digital signatures offline using an embedded 32-byte public key. The private signing key never leaves the secure cloud enclave.
  • JWT Session Tokens: Cloud endpoints authenticate sessions via standard HS256 JWT bearer tokens or Clerk session tokens.
  • Dual Bearer Support: Client endpoints accept either a signed license key (TS2.* / TS1.*) or a JWT in the Authorization: Bearer <token> header.
  • Zero Hardcoded Secrets: All SMTP, Stripe, Admin, and signing keys are strictly injected via environment variables.

2. Cloud API Endpoint Reference

Method Endpoint Auth Required Headers / Parameters Description
GET /api/health Public None Healthcheck and service version status.
GET /api/dashboard Bearer Auth Authorization: Bearer <jwt|key> or ?license=<key> Authenticated user metrics, token savings, and license status.
POST /api/team/seat/create Bearer Auth Authorization: Bearer <jwt|key>, JSON { owner_email, member_email } Generates a cryptographically signed seat license for a team member.
POST /api/team/seat/revoke Bearer Auth Authorization: Bearer <jwt|key>, JSON { owner_email, license_key } Revokes an active team seat license key.
POST /api/telemetry/sync License Auth JSON { license_key, email, events: [...] } Syncs batch telemetry events (enforces max 50 events/batch).
POST /webhook/stripe HMAC Sig Stripe-Signature: t=...,v1=... Verifies Stripe webhook payload against STRIPE_WEBHOOK_SECRET.
GET /api/admin/overview Admin Only X-Admin-Secret or Admin JWT Global overview of MRR, seats, active licenses, and system health.
POST /api/admin/license/issue Admin Only X-Admin-Secret or Admin JWT Manually issues or provisions a custom tier license.
POST /api/admin/license/revoke Admin Only X-Admin-Secret or Admin JWT Immediately revokes a license.
POST /api/admin/license/extend Admin Only X-Admin-Secret or Admin JWT Extends an active license duration.

3. Server Configuration & Environment Variables

Variable Required Default / Description
TOKENSAFE_PRIVATE_KEY Production 64-character hex Ed25519 seed for issuing TS2. licenses.
ADMIN_SECRET_KEY Yes High-entropy secret for administrative API access (X-Admin-Secret).
ADMIN_EMAILS Optional Comma-separated admin emails (e.g. nekosin4eg@gmail.com).
STRIPE_WEBHOOK_SECRET Production Webhook secret (whsec_...) used to verify Stripe webhooks.
SMTP_PASS Production Password/token for transactional mail delivery via ImprovMX SMTP.
JWT_SECRET Optional Shared HMAC secret for verifying custom JWT tokens.

🧪 Unit Testing

python3 -m unittest discover tests

Zero external runtime dependencies — runs on standard Python 3.10+.


📄 License

MIT License. Commercial enterprise licenses available for on-premise B2B deployments.

Export Markdown Report

./bin/agy-tokens markdown

📂 Project Structure

agy-token-optimizer/
├── plugin.json                 # Antigravity plugin manifest
├── hooks.json                  # Lifecycle hook declarations (PreInvocation, PreToolUse, PostToolUse)
├── rules/
│   └── AGENTS.md               # TOCP Protocol directives for LLM (prompt-cached)
├── skills/
│   └── token-optimizer/
│       └── SKILL.md            # On-demand Antigravity skill
├── proto/
│   └── tocp.proto              # Protocol Buffers contract
├── src/
│   ├── tokenizer.py            # Zero-dependency calibrated BPE token counter
│   ├── compactor.py            # Semantic compactor (Natural Language -> TOCP S-Expr)
│   ├── slicer.py               # AST & Line windowing tool optimizer
│   ├── metrics.py              # Persistent SQLite metrics engine
│   └── dashboard.py            # Visual ANSI terminal & Markdown generator
├── hooks/
│   ├── pre_invocation.py       # PreInvocation hook entry point
│   ├── pre_tool_use.py         # PreToolUse hook entry point
│   └── post_tool_use.py        # PostToolUse hook entry point
├── bin/
│   └── agy-tokens              # Executable CLI
├── tests/                      # Unit test suite (100% passing)
├── install.sh                  # One-click installer for Antigravity
└── README.md

🛠️ Installation & Antigravity Setup

Run the installer:

./install.sh

This symlinks the plugin directly into ~/.gemini/config/plugins/agy-token-optimizer.
Antigravity automatically loads it on startup.


🧪 Running Unit Tests

python3 -m unittest discover tests

No external packages required — runs on standard Python 3.10+.

Metadata

Release files for tokensafe 1.3.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tokensafe 1.3.0
File Size Uploaded Provenance Actions
tokensafe-1.3.0.tar.gz 75.5 kB Without provenance Details for tokensafe-1.3.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for tokensafe 1.3.0
File Interpreter ABI Platform Actions
tokensafe-1.3.0-py3-none-any.whl Python 3 none any Details for tokensafe-1.3.0-py3-none-any.whl

Total release size: 144.1 kB

Release files / tokensafe-1.3.0.tar.gz

Download URL tokensafe-1.3.0.tar.gz
Size 75.5 kB
Tags Source
SHA-256 checksum
How to use checksums
a621be75eab4bd4b1cb80eb926befa70126d493d10982c71963cc6c19002a51c
BLAKE2b-256 checksum
How to use checksums
ea9323b0ff019b989a7125e0cebac8f8f414a559e77d2bbdf0e6bb246590fd49
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.5

Release files / tokensafe-1.3.0-py3-none-any.whl

Download URL tokensafe-1.3.0-py3-none-any.whl
Size 68.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a097e34e9a552f0634564037766953fa7ff71cad320fe340a87963a179706775
BLAKE2b-256 checksum
How to use checksums
c61bd0a95b55e5d591dcfa16518653a5d28ea4eade9f179a02366c05eefdb22e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.5

Security for tokensafe 1.3.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.3.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page