English | Türkçe

🍯 TokenJar

Put tokens back in your jar. Save 70-95% tokens for AI coding assistants without losing functionality.

MCP server that saves 70-95% tokens for AI coding assistants — without losing functionality.

TokenJar sits between your AI coding assistant and your codebase, intelligently compressing code reads, terminal outputs, and file operations to dramatically reduce token consumption, context compaction, and latency.

Works with Claude Code, Cursor, Antigravity (AGY), Windsurf, Continue.dev, and any MCP-compatible AI assistant.

✨ Features & Architecture

🛡️ Built-in Guardrails & Reliability

• Lockfile & Giant Asset Shield: Prevents context window destruction from 50,000-line lockfiles; supports 5-line surgical version queries.
• L1 RAM + L2 SQLite Persistent Cache: Survives MCP server restarts and IDE reboots (~/.tokenjar/cache.db with WAL mode).
• Fallback Safety Guard: If a test or command fails (exit_code != 0), TokenJar guarantees tracebacks and error contexts are preserved intact.
• Tiny File Anomaly Guard: If a diff header would consume more tokens than the file itself, the full content is returned to prevent token inflation.
• Runaway Stream Protection: Protects host memory from infinite loops by capping raw terminal buffers at 2MB with graceful truncation.
• SQLite Database Bloat Guard: Files larger than 5MB are cached by hash reference without bloating disk space.

🚀 Quick Start & Installation (v1.0.1 GA)

TokenJar is distributed in two official editions:

1. 🦀 Rust Native Engine (Recommended): High-performance, self-contained single binary with microsecond AST, 14 MB RAM, and zero Python dependencies.
2. 🐍 Python Edition: Pure Python FastMCP package for pip and virtual environments.

📥 1-Click Direct Downloads (Precompiled Binaries)

Click your operating system below to download the latest v1.0.1 release:

⚡ Option 1: Rust Native Engine (1-Click Terminal Install)

Best for: Highest speed, 14 MB RAM, microsecond tree-sitter AST, and zero Python dependency.

Copy and paste one line into your terminal to install and add tokenjar to your PATH automatically:

Windows (PowerShell):

iwr -useb https://raw.githubusercontent.com/Farukes/TokenJar/main/install.ps1 | iex

# Linux & macOS (Bash):
curl -fsSL https://raw.githubusercontent.com/Farukes/TokenJar/main/install.sh | bash

Or install via Cargo (crates.io):

cargo install tokenjar

🐍 Option 2: Python Edition (pip)

Best for: Python-centric environments, custom script integration, or pip workflows.

# Install from PyPI
pip install tokenjar

# Or install directly from GitHub main:
pip install git+https://github.com/Farukes/TokenJar.git

📊 Proven Performance & Stress Test Benchmark

Empirical results from our rigorous 100-Step Real-Life Developer Stress Test and 50-Cycle MCP Head-to-Head Benchmark comparing Standard Raw AI vs TokenJar Python vs TokenJar Rust Native Engine:

Auto-Configure Agent Steering Rules

Automatically inject TokenJar optimization instructions into your repository rules:

# Injects rules into AGENTS.md, .cursorrules, .windsurfrules, and CLAUDE.md
tokenjar init-rules

🎛️ Output Optimization Controls (CLI & Terminals)

Switch between compact surgical output and default unrestricted output with crystal-clear commands:

# 🟢 Enable compact surgical diffs & zero-truncation quality mandate
tokenjar output on

# ⚪ Revert AI assistant to default unrestricted output settings
tokenjar output off

# 📊 Check current output configuration status
tokenjar output

Slash commands are also supported in your AI assistant chat (/tokenjar output on, /tokenjar output off).

🔌 Setup with Your AI Assistant

⚡ 1-Click Automatic Setup (Recommended)

Automatically detects and configures TokenJar MCP server in Claude Desktop, Cursor, Windsurf, Claude Code, and VS Code with automated backups:

# 🟢 Configure all detected IDEs in one command
tokenjar install-mcp

# ⚪ Cleanly revert at any time (preserves all other servers you added!)
tokenjar uninstall-mcp

Manual Configuration

If you prefer to configure manually or use other clients:

claude mcp add tokenjar -- python -m tokenjar

{
  "mcpServers": {
    "tokenjar": {
      "command": "python",
      "args": ["-m", "tokenjar"],
      "env": { "PYTHONUNBUFFERED": "1" }
    }
  }
}

{
  "mcpServers": {
    "tokenjar": {
      "command": "python",
      "args": ["-m", "tokenjar"],
      "env": { "PYTHONUNBUFFERED": "1" }
    }
  }
}

{
  "mcpServers": {
    "tokenjar": {
      "command": "python",
      "args": ["-m", "tokenjar"]
    }
  }
}

mcpServers:
  - name: tokenjar
    command: python
    args: ["-m", "tokenjar"]

🛠️ Available MCP Tools

• find_symbol_global(query, root_path=".", exact=False): Search for functions, methods, or classes across the entire codebase by name without reading multiple files.
• find_symbol_references(symbol_name, root_path=".", max_results=25): Blast radius reference analyzer. Finds all callers, imports, and usages across the entire codebase before editing or refactoring code.
• tool_get_code_skeleton(file_path): Extract structural skeleton of a file — classes, function signatures, docstrings, and type annotations with bodies replaced by .... (Supports Python, JS/TS, Go, Rust, Java, C/C++, C#, Ruby, PHP, Kotlin).
• tool_get_symbol(file_path, symbol_name): Extract the full implementation of a specific class or function by name after inspecting its skeleton.
• read_file_smart(file_path, force_full=False, query="", start_line=None, end_line=None): Differential file reader with session caching, targeted line range slicing, and Lockfile Shield. Supports start_line and end_line (1-indexed, inclusive) to surgically inspect specific line ranges with line numbers instead of loading entire large files. Returns [CACHED] unchanged (~3 tokens) or unified diffs on edits. For lockfiles (package-lock.json, Cargo.lock, etc.), pass query="package-name" for surgical 5-line version blocks instead of 50,000 lines.
• run_command_smart(command, cwd="."): Executes shell commands and prunes verbose logs from pytest, jest, npm, cargo, and git.
• filter_output(output, output_type="auto"): Pure text filter for test runners, build pipelines, and version control logs without executing commands.
• get_repo_map_tool(root_path=".", max_tokens=1000): Graph centrality codebase map prioritized by cross-file import relationships.
• get_directory_tree_tool(root_path=".", max_depth=4): Lightweight directory tree honoring .gitignore and skipping binary folders.
• cache_stats(): Inspect session read hits, misses, diffs, and aggregate token savings.

📦 MCP Resources & Prompts

• Resources:
  • tokenjar://stats: Live cumulative token and financial savings dashboard.
  • tokenjar://guide: AI assistant best-practice optimization guidelines.
  • tokenjar://config: Active project configuration and ignore settings.
• Prompts:
  • optimize_coding_task(task_description): System prompt template steering assistants toward token-efficient workflows.

⚙️ Project Configuration (tokenjar.toml)

Create an optional tokenjar.toml in your repository root to customize exclusions and budgets:

[general]
ignore_patterns = ["tests/fixtures/*", "legacy/*", "*.bak"]
max_cacheable_bytes = 5242880 # 5 MB

[cache]
ttl_days = 30
max_entries = 5000

[repo_map]
default_budget = 1000

💻 CLI Commands & Shell Hooks

TokenJar also functions as an interactive command-line utility for human developers and local shell automation:

# 🎨 Launch On-Demand Control Dashboard (Zero Background RAM UI)
tokenjar ui

# 📊 Check comprehensive live operational status of TokenJar across IDEs
tokenjar status

# ⚡ 1-Click auto-configure MCP across Claude Desktop, Cursor, Windsurf, VS Code
tokenjar install-mcp

# ⚪ Safely remove TokenJar MCP configuration and restore exact original state
tokenjar uninstall-mcp

# View cumulative savings dashboard (tokens saved, money saved, operations)
tokenjar stats

# Run any shell command through intelligent filtering
tokenjar run "pytest tests/ -v"
tokenjar run "npm test"

# Temporary bypass: see 100% of raw output when you need full logs
RAW=1 tokenjar run "pytest"
tokenjar run "pytest --raw"

# Prune expired or excess entries from L2 SQLite cache
tokenjar cache-prune --ttl-days 30 --max-entries 5000

# Install transparent shell hooks (so pytest/npm are automatically filtered)
tokenjar hook

# Cleanly and safely uninstall all shell hooks
tokenjar unhook

# 🟢 Enable TokenJar for THIS project (default)
tokenjar on

# ⚪ Disable TokenJar for THIS project (keeps other projects active)
tokenjar off

# 🌐 Enable TokenJar MCP globally across all detected IDEs
tokenjar on --global

# 🔴 Disable TokenJar MCP globally and cleanly revert IDE settings
tokenjar off --global

# 📝 Alias: Inject steering rules into the current project
tokenjar init
tokenjar init --clean

# 🎨 Open interactive Web Dashboard (Zero Background RAM)
tokenjar ui

# 🧹 Completely clear L2 SQLite cache
tokenjar cache-clear

# ⚠️ Completely uninstall TokenJar from host (IDEs, project rules, hooks, cache, and PATH)
tokenjar uninstall
# or skip confirmation prompt:
tokenjar uninstall --yes

# Install /tokenjar slash commands for AGY CLI and Claude Code
tokenjar setup-commands

# Reset metrics counter
tokenjar reset-stats

🔒 Enterprise Privacy & Security Guarantee

TokenJar is built strictly under a Zero-Telemetry, 100% Localhost design philosophy:

• 100% Local Execution: All parsing (Tree-sitter), caching (SQLite), and output filtering happen locally in-process on your CPU.
• Zero External Network Calls: No telemetry servers, no analytical trackers, no outbound pings, and no cloud dependencies whatsoever.
• Air-Gapped Compatible: Safely operates in classified, offline, or air-gapped corporate enterprise environments.
• Local Data Isolation: Persistent cache (~/.tokenjar/cache.db) and statistics (~/.tokenjar/telemetry.json) reside exclusively in your user directory and can be purged at any time with tokenjar reset-stats or by deleting the directory.
• Non-Invasive Architecture: Never modifies your project code without explicit assistant direction.

🦀 Enterprise & High-Performance Native Engine (Rust Edition)

For enterprise environments, massive monorepos (50,000+ files), CI/CD pipelines, or developer systems without a Python runtime, TokenJar provides an ultra-fast, zero-dependency native Rust binary (tokenjar.exe / standalone executable).

Why the Enterprise Native Engine?

• Zero Runtime Dependencies: No Python, pip, Node.js, or virtual environments required. Single standalone binary.
• Ultra-Low Latency: Instant startup (~3ms cold start vs 300ms Python startup) for zero-delay MCP tool responses.
• High-Concurrency Indexing: True multithreaded (Rayon + Tokio) parallel code parsing and symbol extraction.
• Embedded 15-Language AST Engine: Built-in Tree-sitter parsers for Rust, C, C++, Go, C#, Java, Python, JavaScript, TypeScript, PHP, Ruby, Bash, HTML, CSS, JSON statically linked inside the binary.
• Minimal Memory Footprint: Consumes only ~8-15 MB RAM under active load.

Enterprise Quick Start (Standalone Binary)

Download the precompiled binary from GitHub Releases or build directly with Cargo:

# Build optimized native release binary from source
cargo build --release --workspace

# The standalone binary is ready:
./target/release/tokenjar.exe status

Enterprise MCP Configuration (claude_desktop_config.json / Cursor)

Point directly to the native binary without any Python wrapper:

{
  "mcpServers": {
    "tokenjar": {
      "command": "C:\\path\\to\\tokenjar.exe"
    }
  }
}

🌍 Supported Languages

TokenJar uses Tree-sitter for AST parsing and supports 130+ programming languages out of the box, including:

Python · TypeScript · JavaScript · Go · Rust · Java · C# · C / C++ · Ruby · PHP · Swift · Kotlin · Scala · Dart · Lua · Elixir · Haskell · and more.

🧪 Development & Quality Assurance

TokenJar maintains dual test suites ensuring 100% parity across both implementations:

# Python (Community Edition & MCP SDK)
pip install -e ".[dev]"
pytest tests/ -v           # 65 tests passing

# Rust (Enterprise Native Engine)
cargo test --workspace    # 35 tests passing

📄 License & Intellectual Property

Copyright © 2026 Ömer Faruk Eskitürk. All rights reserved.

Licensed under the Business Source License 1.1 (BSL 1.1) with an automatic transition to the Apache License, Version 2.0.

• Free Use: Free for all personal, educational, research, evaluation, and internal business use.
• Commercial Restrictions: Cannot be hosted or provided as a paid commercial service or SaaS competing with the Licensor.
• Sunset to Apache 2.0: Converts automatically to 100% open-source Apache 2.0 on 2030-01-01.

See LICENSE for full legal terms.