🦾 Token-optime

A middleware MCP server that sits between Claude Desktop and your downstream MCP servers — reducing token consumption through semantic caching, dynamic tool selection, and response trimming.

LLM Backend: Groq | Vector Store: ChromaDB | Dashboard: Streamlit

🎯 What Problem This Solves

Every Claude Desktop session sends the full schema of every connected MCP tool with every prompt. With 23+ tools registered, that's thousands of tokens injected per query — even when only one tool is needed.

Token-optime optimises at three levels:

1. Semantic Tool Selection — Only the most relevant tool schema is sent to Claude per query (ChromaDB similarity search)
2. Semantic Caching — Repeated or similar queries skip tool calls entirely and return cached answers (similarity ≥ 0.8)
3. Response Trimming — Verbose tool responses are trimmed to ≤200 tokens before entering the context window

Real result from testing: 91.8% schema token reduction per call.

📁 Project Structure

.
├── README.md                       
├── config.py                      # Settings & env loading
├── token.sh                       # One-command installer for new users
├── requirements.txt               # Python dependencies
├── Dockerfile                     # For EC2 dashboard deployment
├── pyproject.toml                 # PyPI package definition
│
├── src/
│ ├── front.py                     # Streamlit dashboard
│ ├── dash_api.py                  # Lightweight API for EC2 deployment
│ ├── core/
│ │ ├── cache.py                   # Semantic cache
│ │ ├── trim.py                    # Response trimmer
│ │ ├── tool_selection.py          # Tool selection
│ │ ├── client.py                  # Groq/Claude client
│ │ ├── document_search.py         # RAG (index + search)
│ │ └── db.py                      # SQLite audit logging
│ │
│ └── mcp/
│ ├── server.py                    # Main MCP server
│ └── server_http.py               # FastAPI + metrics
│
└── storage/
├── chroma_db/                     # ChromaDB persistent store
└── token_audit.db                 # SQLite audit logs

Our Test Case

1. list files on /Users/../Desktop

2. list files on /Users/../Desktop

3. index /Users/../Desktop/leave_policy.pdf doc_id leave

4. what is the leave policy?

5. what is the leave policy?

6. read a file

🏗️ System Architecture

                    Claude Desktop
                          │
                          ▼
┌──────────────────────────────────────────────────────────┐
│ Token-Optimised MCP Server                               │
│ ┌─────────────────────────────────────────────────────┐  │
│ │ Exposed Tools                                       │  │
│ │ • find_tool (orchestrator)                          │  │
│ │ • index_document / ask_document                     │  │
│ │ • search_all_documents / index_documents_folder     │  │
│ └─────────────────────────────────────────────────────┘  │
│ ┌─────────────────────────────────────────────────────┐  │
│ │ Optimization Layer                                  │  │
│ │ ┌──────────────┐ ┌──────────────────────────────┐   │  │
│ │ │ Semantic     │ │ Tool Selection               │   │  │
│ │ │ Cache        │ │ (ChromaDB → top-k tools)     │   │  │
│ │ │ (sim ≥ 0.8)  │ │                              │   │  │
│ │ └──────────────┘ └──────────────────────────────┘   │  │
│ │ ┌──────────────┐ ┌──────────────────────────────┐   │  │
│ │ │ Response     │ │ Token Audit                  │   │  │
│ │ │ Trimmer      │ │ (SQLite audit log)           │   │  │
│ │ │ (≤500 tok)   │ │                              │   │  │
│ │ └──────────────┘ └──────────────────────────────┘   │  │
│ └─────────────────────────────────────────────────────┘  │
└────────────┬──────────────────┬──────────────────────────┘
             │                  │
             ▼                  ▼
      ┌─────────────┐    ┌──────────────┐
      │ Filesystem  │    │ Memory MCP   │
      │ MCP         │    │ (remote)     │
      │ (14 tools)  │    │ (9 tools)    │
      └─────────────┘    └──────────────┘

🚀 Install — One Command

Prerequisites

• Python 3.10+
• Node.js + npx
• Groq API key — free at console.groq.com
• Claude Desktop (latest)

New users — two commands

pip install token-optimise
token-optimise

The installer will:

• Check Python and Node are installed
• Ask for your Groq API key
• Write the MCP config into Claude Desktop automatically
• Restart Claude Desktop

Existing users / developers

git clone https://github.com/yourrepo/tom.git
cd tom
cp .env.example .env      # add your GROQ_API_KEY
./token.sh start

token.sh commands

./token.sh start     # start
./token.sh stop      # stop
./token.sh restart   # restart
./token.sh status    # check if running

📊 Dashboard

Local (your metrics only)

http://localhost:7738

🔧 Ports

These are intentionally non-default to avoid conflicts with other projects.

💬 For Best Results — Tell Claude to Use Token-optime

Claude Desktop has its own built-in tools (memory, web search) that it may prefer by default. For consistent routing through Token-optime:

Add Instructions for Claude :

Paste it here Settings → Instruction for Claude

You have access to a token MCP server. Follow these rules strictly:
For ALL tasks — files, Gmail, Notion, memory, or anything else — always use token:execute first before calling any other tool.
For PDF questions always call token:list_indexed_documents first then token:ask_document.
After every response call token:wick_track.

DOCUMENT/PDF TASKS:
- ALWAYS call list_indexed_docs first, then ask_document immediately.
- NEVER ask the user for clarification.

AFTER EVERY RESPONSE: Call wick_track.

You can start your conversation with :

Use token:execute for every task. Never call other tools directly.

For PDF tasks specifically:

First call token:list_indexed_documents, then token:ask_document.

Why is this needed? Claude decides which tool to call — Token-optime can't force it. The system prompt in the config nudges Claude, but an explicit instruction in the chat is the most reliable way to ensure Token-optime is used. This is an honest limitation of how Claude Desktop works, not a bug in Token-optime.

Dashboard Metrics

🐛 Troubleshooting

Server not starting

tail -50 server_out.log

Port 7737 in use

lsof -i :7737
kill <PID>
./token.sh start

Claude not using token:execute

• Check Settings → Developer → MCP Servers — token should show green
• Add explicit instruction at start of conversation

Dashboard offline

./token.sh status
./token.sh restart

⚠️ Known Limitations

• Claude's built-in tools take priority — memory and web search bypass Token-optime because Claude prefers its native tools
• Cost estimates are approximate — based on Sonnet 4.6 pricing; varies by model
• Windows not supported — Mac and Linux only
• Tilde paths — if a tool call fails with "file not found", use the full path e.g. /Users/name/Desktop/file.pdf

📖 Docs

• docs/COMPONENTS.md — Detailed components guide
• docs/QUICKSTART.md — 5-minute quick start